Another consideration. When do you start to get saddle discomfort? If in the first few miles, something is clearly wrong with the saddle.
If it's something that just builds up over a few hours in the saddle, it could be just lack of leg strength. As your legs tire, more weight rests on the saddle and pain increases.
May just need to increase distances gradually.
Good luck with your planned rides.