I was touring the Isle of Wight earlier this year, was a bit lost and some chap pulled up along side and riding no-handed, pulled a map out his backpack and read it and gave us directions. I was quite impressed.
Bit sad, but I've been practicing since, not so bad myself now and it's a useful skill tbh. Like has been said for wiping glasses, ajusting backpack, just to relax and take in the view. I'm not up to reading maps on the move however. Perhaps I'm wrong, but I get the impression it uses slightly different muscles to pedal when bolt upright, seems to give legs a bit fo a breather or feels like it anyhow.
Just do it whenever someplace quiet, gradually gets easier and easier and should be able to turn and manuver a bit eventually.