The sustainability of society hinges on the future of agriculture. Though alternatives to unsustainable, high-input industrial agriculture are available, agricultural systems have been slow to transition to them. Much of the resistance to adopting alternative techniques stems from the perceived costs of alternative agriculture, mainly in terms of yields. The general assumption is that agriculture that is less harmful to people and wildlife directly will be indirectly more harmful because of yield losses that lead to food shortages in the short-term and agricultural extensification in the long-term. Though the yield gap between industrial and alternative forms of agriculture is often discussed, does industrial agriculture actually produce the highest yields? In addition, to what aspects of the food system is yield relevant? We review the evidence for differences in crop yields between industrial and alternative systems and then evaluate the contribution of yields in determining whether people are fed, the land in production, and practices farmers will adopt. In both organic and conservation agriculture, different combinations of crops, climate and diversification practices outperformed industrial agriculture, and thus we find little evidence that high input systems always outperform alternative forms of agriculture. Yield, however, is largely irrelevant to determining whether people are fed or the amount of land in production. A focus on increasing yields alone to feed the world or protect biodiversity will achieve neither goal. To promote sustainable agriculture, we must move past focusing on these oversimplified relationships to disentangling the complex social and ecological factors, and determine how to provide adequate nutrition for people while protecting biodiversity.