There are plenty of fast flat bars out there, Planet X do a rather tasty light carbon number, Dolan would probably build you something, as would Pearsons,
Condor etc
To be honest, I'm not a lot quicker on my road bikes or flat bar or indeed single speed when I do my circuits around home due to the traffic. The flat bar does feel a bit sluggish now with mudguards, a rack and marathon + tyres. When naked with slicker tyres, it was certainly more sprightly. When I get out on to the open roads, probably faster on the road bikes, but that might be just because I'm riding them more away from home. Roadbikes do offer more ways to alter your hand and body position on the bike, but you can always add ergo grips and bar ends. I rarely use the drops for any distance for example
To be honest, I'll never be quick - my age, weight, gender, lack of athleticism and crap joints will always hold me back. That's fine, I enjoy riding all of the bikes when I do
(as long as the idiot drivers behave
)
I'm not sure why you need to be ashamed of doing pilates and yoga, forget the bloody stereotypes. I'm not into pink or flower arranging ether!