What does $\binom{-n}{k}$ mean?

It is the binomial coefficient for a negative exponent: $$ \begin{align} (1+x)^{-n} &=\sum_{k=0}^\infty\binom{-n}{k}x^k\\ &=\sum_{k=0}^\infty(-1)^k\binom{k+n-1}{k}x^k \end{align} $$ Note that this follows from the following formulation of the standard binomial coefficient: $$ \begin{align} \binom{-n}{k} &=\frac{\overbrace{-n(-n-1)(-n-2)\dots(-n-k+1)}^{k\text{ factors}}}{k!}\\ &=(-1)^k\frac{(n+k-1)(n+k-2)(n+k-3)\dots n}{k!}\\ &=(-1)^k\binom{n+k-1}{k} \end{align} $$


For integers $n,k\ge 0$,

$$\binom{-n}k=\frac{(-n)(-n-1)(-n-2)\dots(-n-k+1)}{k!}\;.$$

Thus, $$\binom{-3}4=\frac{(-3)(-4)(-5)(-6)}{4!}=15\;.$$

In fact $\dbinom{x}k$ is defined for all real $x$ and integers $k\ge 0$ by

$$\binom{x}k=\frac{x^{\underline k}}{k!}\;,$$

where $$x^{\underline k}=x(x-1)(x-2)\dots(x-k+1)$$ is a so-called falling factorial or falling $k$-th power.