I use panniers for commuting and obviously for touring, but when my commuter was off the road last year, I used the 'best' bike and a rucksack instead. It wasn't a super dooper rucksack, so I did get a sweaty back. I also felt that it restricted breathing by being tight across the chest, gave me neck and shoulder aches and I found it more difficult to turn when performing the safety glances behind.
Now this is very unscientific, because obviously we're talking two different bikes, but I felt that in terms of riding efficiency, I was faster with the rucksack than I am with the panniers. Obviously the commuter is heavier and has a rack and guards etc, and the 'best' bike is carbon, but even taking that into account, the greater manoeuvrability offered by a 'naked' bike even when carrying weight on the person seems to offer speed and agility advantages.
So if my commute was about getting to work as fast as possible then I'd rucksack all day long, but the most comfortable option for me is to use the panniers - my suit gets less creased, and I can take additional tools in my rack bag, and I don't get as many physical complaints. I also actually enjoy commuting on a heavier, less agile bike. I carry all sorts of stuff I don't need to and treat my commute as training miles, hoping that come the summer club rides when the carbon gets dusted off, I'll feel the benefits of my fitness, a lighter frame, improved aerodynamics, and more agility to a greater extent.