"Good-quality forages are the cornerstone of goat nutrition. Supplement grain based on the body condition and reproductive and growth stage of the animal. Most goats should be able to thrive on pasture and hay." http://pubs.ext.vt.edu/412/412-501/412-501.html
However, like people, goats will eat stuff they shouldn't.
"I have heard outrageous things people feed to goats. Just because a goat will eat something does not mean you should feed it to them."
There follows a list of foods and non-foods goats will eat but shouldn't: paper, cigarettes, dog food, cat food.
Goats are reputed to be willing to eat almost anything, even tin cans. While goats do not in fact eat tin cans, goats are browsing animals, not grazers like cattle and sheep, and as such will attempt to consume just about any plant matter.[12]
A domestic goat feeding in a field of capeweed, a weed which is toxic to most stock animalsContrary to this reputation, they are quite fastidious in their habits, preferring to browse on the tips of woody shrubs and trees, as well as the occasional broad-leaved plant. However, it can fairly be said that their plant diet is extremely varied, and includes some species which are otherwise toxic.[13] They will seldom consume soiled food or contaminated water unless facing starvation. This is one of the reasons why goat rearing is most often free ranging, since stall-fed goat rearing involves extensive upkeep and is seldom commercially viable.