but they're probably fiddling the VAT etc like most remaining cash only businesses.
I think it's really unfair to suggest that if a business doesn't want to be dependent on a bank for all of its transactions it is naturally fiddling the books. I took a sabbatical from work a few years ago and opened a shop and didn't open a merchant account (you needed to do this back then to have card facilities). As a retailer, you know where you are with cash - no chance of a chargeback a month later. Plus no extra fees. It is more straightforward for everyone.
I like cash. I don't like places that are card only, it's harder to know how much you're spending. On more than one occasion I've been charged twice for rounds of drinks, for example. You have to either write it off as an expensive night out, or go back with your bank statement.
As to bank accounts being closed on "moral grounds", whose morals are we talking about? Look at the massive Pornhub website and the way the credit card companies managed to get them to remove most of the content by removing their banking facilities. I'm not defending what companies like Pornhub do, but It is a very powerful lever and banks don't have enough oversight or accountability to hold it.
As to bank accounts, a long time ago I tried living without one for a while working cash in hand. It didn't work for a lot of things, and was very expensive for others. If I hadn't lived with my mum and dad I wouldn't have managed.