I ride single track roads, aware that at certain times of the year tractor use will be higher than at other times. When I feel it's safe* to let them past I'll let them. They're working, me I'm using my primary means of transport. Be it for fun or because I need to.
On the bike it's a case of looking a lot further ahead than on a wider road, and using your ears more. You can anticipate when an overtaking opportunity can arise, and by the simple action of making eye contact with the drivers can let them know you know they're there, and you're not holding them up on purpose. Often letting them past before it becomes an issue, or stopping for a few seconds because you know they're headed your way(eyes or ears), has the added benefit of slowing traffic coming the other way. Fewer would argue with a tractor than a cyclist, especially if there's anything mounted on the front of the tractor.
*Safe for me and them.