I had to walk downhill once, which I didn't enjoy at all! (steep, slippery cobbles)
One of my new year's resolutions this year was to get better at hills and I've enjoyed fulfilling it. This was because I was embarrassed by the loudness of my breathing at the top of a hill last November on my first group ride.
I've done a couple of hilly Skyrides which have been great because everyone is so supporting and encouraging. It's so much easier to keep going when you have people with you.
The down hills are so much fun and are earnt on the ups.
I still struggle on hills sometimes, but I'd rather stop for a breather, take photos, have a drink etc, then set off again, than walk.
You have to do what works for you. Everything gets easier the more miles you do. Just keep enjoying your cycling.