Well, I'm lumping bicycles in with motorcars as vehicles (albeit human powered ones) and I would add mobility scooters (while moving) to the list as well. Pedestrians, mainly I'm avoiding that can of worms because it truly would be political suicide for someone to try but I agree the argument could be made. Also, in my head we're talking on the carriageway predominantly where as I presume you picture tootling along on a cycle path.
With vehicle speeds, all of them are limited to something, so like the Hummers argument (which I have to say I don't quite get what group you're lumping Hummers in to, big American SUVs?) I don't think they have the same weight to them. Buses and Taxis make sense in bus lanes, if anything adding bicycles is the anathema (not that I'd like to see that stopped).
If you want to debate for the hell of it then alcohol would probably be the obvious one to question why there's no law against it. It definitely contributes to more injuries than mobile phones in anyone's hands and so if we're arguing civil liberties then that would be the obvious one, but that's SCP territory and there's a reason I try not to get involved in that one.
I think he's just pointing out that All Things Are Not The Same. Restricting the freedoms of motorists does not in any way imply that we should restrict the freedoms of pedestrians or cyclists.