Everytime you get home! (after all, commuting to work is a ride and it would not be practical to wash your kit between arriving and leaving!)
There is a world of difference between going on a 100 mile ride and then washing your kit than going on 5 x 20 mile rides (therefore 100 miles) and washing your kit, due to the fact that bacteria will breed and stay on the material.
Sorry, but no excuses, it's just plain minging not to wash your kit and, dare i say it, a typical man's attitude. you would never hear a woman say "shall i wash my kit, because i've only worn it for 20 sweaty miles?"
put your shorts in the washing machine everytime you get home. to put it bluntly ... they've been around your sweaty nads!
And while we're on the subject of washing... same goes with underwear! one day is enough! don't wear them the next day as well !