The first time I tried goat cheese it was Norwegian gjettost. It tastes more like a toffee than a cheese. I like it in small quantities. It is very rich. So, I knew that I liked goat cheese. The first time I had goat milk was a surprise. I was on vacation in Puerto Soller on Mallorca. I asked for a glass of milk and they gave me goat milk. I was startled, to say the least. But, once I got over the shock, it was pretty darned good. I tried some goat cheese from a health food store in Chicago, back in the 1970s. It was awful. I smelled like billy goat and the taste seemed to match the smell. I threw it out.
When goat cheese first started being available here in Greater Montreal, I tried some. Again with the billy goat smell, but not quite as strong. So, I didn't buy it for a long time. Then, a goat cheese maker told me that her farm didn't have any billy goats, just so they wouldn't taint the smell of the milk. Once a year she paid for stud service. This has apparently become quite common at goat dairy farms. Some, just keep any billy goats far away from the females. I cautiously tried goat cheese again and it was good. I don't buy it very often, but I do get some spreadable goat cheese every now and again. I also get feta made from goat milk and another one made with sheep milk. They are slightly different and I can't choose which I prefer.
My point is that you should give some mild goat cheese another try. It may have improved since you last tasted goat cheese.