There's 2 different discussions going on here (well, 3 if you count Jim's, but I lost the manual); one is about improving fitness and the other is appropriate gearing.
My contention is simply this: most off-the-peg "racing" bikes don't have gearing with wide enough range for recreational cyclists. It's always been this way.
I remember, back in '78, when I was working in a bike shop that would actually cater to real cyclists and we'd modify stock cranks for customers (we did this for the first Canadians to enter PBP, in '79).
Then, one day, an old chap from Herefordshire (I think it was) stopped by. He was on a tour up into northern British Columbia and his bike sported a T.A. Cyclotouriste chainset, with a tiny inner ring -- 26 I think it was.
This was a novelty to me, who, by style-conscious default, insisted on keeping my 42-52 Campagnolo Neovo Record cranks, in spite of the fact that this necessitated a ridiculous, badly shifting 13-32 cluster, whenever I went touring.
I remember this wise old touriste saying something along the lines of "I'm no hero; I just like to get over the hills without undue pain, or walking."
It took me another few years to get over my hero complex and learn how to apply the correct technology to the conditions and my abilities.
The other thing that springs to mind reading this thread is the fact that -- besides age, average fitness level and tolerance for suffering -- most normal cyclists' fitness waxes and wanes with the season.
I don't know about you, but -- though I have several to choose from -- I like to be able to ride
all my bikes knowing that I have gearing options for most eventualities.
Even so, in the winter, when I need to plow through fresh snow, the only option is my Blizzard, with a low of 22".