The colloquial use of the pronoun "you" followed by "adjectives"

It’s just a special kind of noun phrase where a pronoun is described by a follow-on noun (or adjective being used like a noun). The grammatical terms are dense and frankly over my head, but there is a discussion of this in Foundations of Cognitive Grammar where it gives other examples like “we linguists” and “you three”.

The personal pronouns are actually grounding predications; although they normally stand alone as nominals, we and you are occasionally elaborated by a head noun: we linguists; you bastard!”

So you can use it with a wide variety of nouns and pronouns. While most of the common ones involving “you” that spring to mind are negative, they don’t have to be. “You saint!” is just as valid as “You liar!”

As Low Powah pointed out, "you silly" doesn't really fit the pattern. It's more something you'd say to a child, or in a joking childlike fashion.