I hear what you are saying and I am sure that quality modern set-ups are 'light' years ahead of the stuff most of us are basing our opinions on but, how much does such a set-up cost (including having a wheel built with the dedicated hub)?
The OP sounds as though he is struggling with a product straight out of the 70s or 80s and based on that assumption I will stand by my advice to chuck it in the bin and get some 'light' modern LED lights.
I'm kind of with you. That bike over there has battery powered LEDs, from B&M. New and shiny and very good.
That bike over there has a mounting bracket for a bottle dynamo but I now use a hub dynamo with five or six year old shiny LED light.
I don't know how long the batteries will last as I've only had the light a week. I'm hoping for a whole Friday Night Ride's worth, but will take spares. The dynamo just goes. It doesn't have a standlight (due to an ordering mix up when I first bought it) but such things are available.
To the OP, my bottle dynamo was mounted on the right fork even though it was a left fork dynamo. This meant the bracket was on the rear face of the fork, so the spinning action of the wheel tended to pull the dynamo into the tyre. It could be this is what you're experiencing. (note that dynamos mounted back to front as I had it must be done up securely. If it comes loose, it'll jam between fork and wheel, stopping the bike dead).