How did "meta" come to mean self-referential?

Solution 1:

It originally meant something like "after". Possibly the most prominent use of this sense is the title of Aristotle's Metaphysics, so called because it constitutes those things which are to be studied after one has learned physics.

This text wound up founding the discipline which we today call "metaphysics", and one way to describe what this subject encompasses is that it covers things at a level of abstraction above physics.

This sort of meaning became applied retroactively to the prefix, and we began building words in that sense, often by applying the root word to itself. Take for example metaknowledge, knowledge at a higher level of abstraction than standard knowledge, or knowledge about knowledge. Eventually, this reflexive sense became a new usage in its own right.