It may of course happen (at age 50+), but if it did, you would have lots of other symptoms as well, such as
leg pain when walking, chest
pain during exercise, etc. Narrowing of arteries affect all arteries, not just the penile artery.
This is good, because a simple test may make you rule this out. A carotid ultrasound. Not sure if US health insurance covers it, but in my country, it's cheap (about $100) and it can rule out narrowed arteries by 98-99% certainty.