You absolute tart, how could you give up, and ruin a pair of cleats into the bargain. You should be ashamed of yourself, a wee lassie could probably have finished out the distance with you on her back!!!
Nah, seriously, that's good going, I had a similar upward curve when I started, unable to complete 3.5 miles without pushing up a hill, to 50 miles in a short period. It does improve quite quickly, after two rides of just over 50 I was on to 100 miles in one leap. The 100, barring some knee pain from poor saddle position fixed for me by others on the ride, was far less painful than the first 50.
Get out with those guys and you'll pick up the distance in no time, company makes it go easier as well.