Can I go against conventional wisdom?
I started on my son's Tesco mountain bike and had exactly the same experience. I was hammering along getting overtaken by OAPs on road bikes who just hissed by, laughing and blethering to each other as they went. I thought I was fit through regular gym sessions but I just couldn't match their speeds so I bought a road bike, a Specialized Secteur.
My - what a difference! it felt light as a feather and seemed to just fly along. Great!! I even entered a 50 mile event and thoroughly enjoyed it!
But, nobody warns you about how the knife edge saddle nearly cuts you in two when you go over a pothole. The pain is excruciating so over time, you learn to stand on the pedals at the slightest bump. But that's the other thing - these bikes are so lightweight and flimsy that they can only really be used on tarmac like a good woman - silky smooth and freshly laid preferably. Forget about darting along the odd cycle trail. Many NCN routes involve muddy sections which effectively debar road bikes. Far from gaining the freedom of the road on your sleek new flyer, suddenly you start planning your routes more carefully, on roads that you inevitably have to share with trucks and boy racers. You'll start getting more punctures, because those whisker-thin tyres are actually manufactured from regurgitated rice paper painted black and they pop at a passing glance. Every rough section of road has you anxiously looking betwixt your legs to spot unexpected deflations (of the tyre too).
After a year of this I switched to a Tricross, a cyclo-cross type bike. It looks exactly like a road bike except it's a bit more sturdily made and you can choose to fit slick tyres or chunky go-anywhere tyres as you please. Every route is now open to you, you can even (shock horror) ride it over a kerb without cutting yourself in half and bending the bike into a pretzel.
If you can only have one bike, don't have one that limits your horizons is my advice.