I concur with others who diagnosed a bent hanger. However, bending it back is not all that easy. Firstly, it has to be bent to a very precise position and this is best done with the right tool, usually not part of every home mechanic's arsenal due to cost.
Secondly, the sacrificial hanger is made from aluminium. This material work-hardens as it bends. It means that bending it back doesn't reverse the bend but places another bend adjacent to the first one. Imagine forming an S in two stages. To bend it back to the original position without an S requires you to remove it and hammer it on a flat anvil.
For all that trouble, most people will simply buy a new hanger. Buy two, one for spare. Remember that real, hard-core cyclists wear the spare one on a leather thong around the neck, as some sort of gnarly jewelry. Those in the know will recognize it in the pub and buy you beer.
If your frame is made from steel, it probably doesn't have a sacrificial hanger. Don't fiddle with this, just take it to a bike shop who can prove to you that they have the right tool.