You've got a little bit of play in the headset.
To fix it, start by unscrewing the pinch bolts. Unscrew them a little at a time so you don't overload a screw. Now your handlebars should be able to move independently of the front wheel.
Next, tighten down the top cap until play is removed from the headset, but no more. It is possible to overtighten this bolt, so only tighten it enough to remove the play.
Now tighten the pinch bolts back up, again a little at a time, alternating between each bolt. If you have any carbon on your bike, or if you know you have a carbon steerer, it's best to do this with a torque wrench to the manufacturers specifications. If your steerer is metal (a Triban owner should be able to clear this up) then you can just torque it down by hand with some mechanical sympathy.
As long as your headset isn't worn then this should fix the issue. If it is worn then you may need new bearings.