Importance of 'smallness' in a category, and functor categories

My impression (as an outsider) is that the smallness assumptions are not considered to be important at all inside category theory itself. They are there only to make sure the results can be formalized in standard set theory, and don't encode any particular intuitive insight. In most (probably all) concrete applications of the results, it is easy to argue that all of the categories involved are small enough to make things work anyway.

Like dicatorships traditionally call themselves the Democratic Republic of $X$, because countries are generally expected to want to be democracies, category theorists dutifully keep track of their smallness assumptions, because mathematical disciplines are generally expected to want to be expressible in ZFC. But it's not as if any particular attention is paid to this internally.

Smallness conditions in category theory appear in several situations. One situation is to assure that certain constructions exist. This includes the most elementary cases of (as you mentioned) the forming of functor categories but also for the constructions of adjoints (i.e., the solution set condition in the Freyd adjoint functor theorem, with a huge emphasis here on the word 'set'). A more advanced application of a categorical construction involving set conditions is in homotopical algebra. Establishing a Quillen model structure can be very hard. It is simplified enormously (yet typically remains hard) to construct a Quillen model structure by means of a cofibrant generation. There one needs to provide sets (and not just classes) of certain arrows with certain properties. If these sets exists the model structure is guaranteed. If such classes of arrows are proper classes and not sets then a model structure is not guaranteed.

Another aspect of smallness conditions is to assure that certain categorical conditions do not force degeneration. For instance, any category that admits all products, not just set indexed products, is known to be a poset. That is why one usually considers small complete (and small cocomplete) categories.

To conclude, smallness in category theory plays a crucial role in different ways. For some of these aspects a tacit assumption of Grothendieck universes is sufficient to hide all the size issues under the carpet and happily go on with your business. In other situations size issues play a very important role that can't be 'pushed away to a higher universe'.