I had a heart attack two years ago at 45 years young! No history in the family but I had really abused myself during my early years. I actually just woke up in the morning one day and did it. I had one stent fitted and my recovery has been good. Fortunately no muscle damage and no residual angina. Cardiac arrests whilst exercising are quite rare (but not unheard of obviously). I'd say at your age your ok if you've not got any symptoms. Even the top guys struggle up the hills, they just make it look prettier than us haha
I had really subtle symptoms prior to the actual event, jaw ache and neuralgia as well as a dull ache in the center of my chest during exercise. If you're not getting any of that then I'd say your good to go. Unless you have a history of premature CVD in the family (heart attack/stroke etc) in which case might be an idea to get yourself checked over anyway.
Not exercise is counter productive to a healthy heart. What I would say is this though, if you're not enjoying it don't do it! Unless your training for an event why put yourself through it?
I've been back cycling now 18 months. I'm far fitter than I was was prior the event, I can cycle further and harder than before and I love a hill! Trust me I huff and puff up them whilst the fitter lads fly past but I don't care! I get up any hill through sheer determination knowing that I'm doing myself the world of good whilst doing it and its a great challenge. If I didn't enjoy it, I wouldn't do it! I've nothing to prove!