Let me just turn that on its head. You are saying that we must put on some outlandish specialised riding gear just to go out and do that most ordinary of things, ie riding a bike?
When I was a club cyclist many years ago I was not a roadie. I didn't race. But I led tours and did weekend rides and it was a nice thing then to have club colours because it gave us a feeling of belonging. It wasn't lycra, in those days, but some sort of wool/synthetic mix. When I was on my own, on tour, or whatever, I wore the same stuff because it was worn in and comfortable. since then, having worn them out, I have found that I can cycle in ordinary clothes just as well, though purpose designed cycling trousers are better if you are going further, or maybe for several days, to avoid discomfort in the underpants department. I do have a pair of lycra cycling trousers for longer trips from that famous designer cycle wear emporium, ALDI, to that end (pardon the pun). I am not a club cyclist any more. I can please myself.
It was occasionally commuting to work over the years that made me realise that I could manage perfectly well for longer trips in appropriate ordinary clothes.
As far as I am concerned, if you are a sporty cyclist and the appropriate gear meets your needs, fill your boots, we can wear what we like, but I know what works for me. Lycra seems to be a hate word for the likes of the Daily Wail and its ilk but I am willing to bet that even the editor has lycra in his socks. If some other wonder fabric were to appear and be adopted by cyclists no doubt that would become their next target.
So there you go.