Can you demystify the Power Law?

As Ross Millikan points out, a function $p(x)$ obeys a power law if

$$p(x) \propto x^\alpha, \ \ \alpha \in \mathbb{R}$$

There are many real-world examples of power law functions. For example, Newtonian gravity obeys a power law ($F = G \frac{m_1 m_2}{r^2}$, where $\alpha=2$ in this case), Coulombs's law of electrostatic potential ($F = \frac{Q_1 Q_2}{4 \pi r^2 \varepsilon_0}$, again $\alpha=2$), critical exponents of phase transitions near the critical point ($C \propto |T_c - T|^\alpha$), earthquake magnitude vs. frequency (this is why we measure earthquakes on a Ricther-scale ) and the length of the coastline of Britain ( $L(G) = M G^{1-D}$, where $1-D$ is the exponent in the power law), to name just a few.

There are many ways to generate power law distributions. Reed and Hughes show they can be generated from killed exponential processes, Simkin and Roychowdhury give an overview of how power laws have been rediscovered many times and Brian Hayes gives a nice article on 'Fat-Tails'. See Mitzenmacher for a review of other generative models to create power law distributions.

Of the ways to generate power law distributions, I believe the one that is most descriptive as to why power laws are so pervasive is the stability of sums of identical and independently distributed random variables. If $X_k$ are identical and independently distributed random variables, then their sum, $S$, will converge to a stable distribution:

$$ S_n = \sum_{k=0}^{n-1} X_k $$

Under suitable conditions, the random variable has power law tails, i.e.:

$$ \Pr\{ S > x \} \propto x^{-\alpha}, \ \ x \to \infty $$

Alternatively, we can talk about its probability density function (abusing notation a bit):

$$ \Pr\{ S = x \} \propto x^{-(\alpha+1) }, \ \ x \to \infty $$

for some $\alpha \in (0, 2]$. The only exception is when the random variables $X_k$ have finite second moment (or finite variance if you prefer), in which case $S$ converges to a Normal distribution (which is, incidentally, also a stable distribution).

The class of distributions that are stable are called Levy $\alpha$-stable. See Nolan's first chapter on his upcoming book for an introduction.

Incidentally, this is also why we often see power laws with exponent in the range of $(1,3]$. As $\alpha \to 1$, the distribution is no longer renormalizable (i.e. $\int_{-\infty}^{\infty} x^{-\alpha} dx \to \infty$ as $\alpha \to 1$)) whereas when $\alpha \ge 3$ the distribution has finite variance and the conditions for it being a normal distribution apply.

The stability is why I believe power laws are so prevalent in nature: They are the limiting distributions of random processes, irrespective of the fine-grained details, when random processes are combined. Be it terrorist attacks, word distribution, galaxy cluster size, neuron connectivity or friend networks, under suitably weak conditions, if these are the result of the addition of some underlying processes, they will converge to a power law distribution as it is the limiting, stable, distribution.

Just to expand on Ross Millikan's answer. Power-laws are important and pop up all the time because they obey scale-invariance. Scale-invariance is the property that a system behaves the same when all length-scales are multiplied with a common factor. System show scale invariance if there is no characteristic length scale associated which, e.g., happens at phase transitions.

The power law just is the equation $f(x)=ax^k$ (the Wikipedia article allows some deviation.) If $k \lt 0$ it says that big events get less likely than small events and how fast. It shows up many places in science, and finding the exponent is often a clue to the underlying laws. It also shows up in the real world. This paper discusses finding it in statistics on the internet. Of course, real world data doesn't fit a power law exactly, and how close is "close enough" may be in the eye of the beholder. It is also known as "Zipf's Law" if you want to search around for places it is found.