No offence, but you two seem obsessed by what divides us, as opposed to what unites us. Cycling clubs are all about cycling (or at least they should be - yours are obviously the exception) - if you are riding a bike, then that should be all thats matters.
I'm trying hard to sympathise with you, but I can't help thinking that the issues you mention could all be easily be resolved by you simply 'not giving a sh1t what anyone else thinks and getting on with your life in any way you like.'
The cycling's fine. It's when some of the people riding the bikes express what they're thinking out loud and it turns out that they think you're a sub-species, a second class citizen, the butt of the joke, and so on and so forth. I can tell myself I don't care and some of the time I may convince myself to swallow that lie but I do care.
Do you know why?
Because making someone feel that they are worth less because of their race or gender or sexuality or disability or class or religion isn't a particularly nice thing to do, and the time is long since past when it's ok for for society (and societies - pun intended) to stand by and let it happen.
You're right, of course. It's not about what people think. It's what they say and what they do. It hurts and it's not right. Furthermore, some people use their fists or other weapons to express their thoughts. There is a link between one person cracking a homophobic "joke" and someone else massacring people in a gay club. It's different ends of the spectrum but it's the same spectrum nonetheless.
I'm sorry that you seem to lack to empathy to understand why diversity matters, why some of us might lack the confidence that others will accept us for who we are but I guarantee that anyone who expresses this concern has previously been on the receiving end of some kind of abuse, and I doubt it's just once. It's bullying, really, and there comes a point where you lose faith in other people: it's beaten out of you with every jibe, every comment or sideways look, every time someone doesn't want to sit next to you on the bus, or sniffs in disgust, or thinks it's ok to make you the butt of the joke or objectifies you in some other way. Stuff happens and people don't always mean it - but it's especially difficult when you try to explain why it matters and you're dismissed out of hand and told that you're the one with the problem!