With the continuous increase in computer power, computational scientists are able to devise models of the physical world with ever increasing complexity and detail. Despite the accuracy of such sophisticated models though, the "forest" of underlying basic principles often gets lost within the "trees" of details of the model. In this talk I will illustrate the benefits of the reverse direction: With examples ranging from materials science, protein folding and snow physics, I will show that intricate scientific questions can often better be understood by simplified models where unimportant details are systematically peeled off (reductionism). In particular, the investigated topics are: (i) Growth-induced polarity formation in molecular crystals; (ii) statistical physics questions in proteins and polymers; (iii) snow depth distribution and smoothing in complex terrain. Besides giving insight into fundamental behavior of complex physical phenomena, I will demonstrate that simplified models surprisingly often pose also a considerable computational challenge asking for sophisticated algorithms.