Hey there Rooster, I haven't been cycling too long, but I have been trying to train myself to go harder for longer. Twice daily 5 times a week I commute pretty much as hard as my hybrid will take me to work at 5.3 mile distance (Not much but my fitness isn't brill), My legs started to seize up and stiffen because I wasn't pre/post stretching on my commutes. I have had the same problems as you when I cycle harder I get slower, I went from near enough passing out after 10 minutes, to 4 hours easy in 2 months. In my opinion, the best thing for keeping your legs feeling fresh and keeping your motivation up to push yourself, shower then soak in a piping hot bath whilst massaging your legs *you can find guides for pressure points easily online*. Also go for a relaxing swim at a local pool and it works the muscles you use to cycle as well as all the muscles around them, commuting cyclists who do not go to the gym or core train at home tend to neglect these, this would cause you to slow down as your legs are not working properly as they should with all muscle groups.
Also, a nice 30-45 minute massage every other week definitely helps, not only physically but psychologically keeps me positive and feeling great.
I hope others would agree