What is the meaning of the third derivative of a function at a point

I've found the time, so I deleted one of my original comments in the OP and decided to expand it into a full answer.

One way to geometrically interpret the third derivative is in the notion of the osculating parabola. In much the same way that the first derivative enters into the defining equation for the tangent line (the line that best approximates your curve in the vicinity of a given point), and that the second derivative is involved in the expression for the osculating circle (the circle that best approximates your curve in the vicinity of a given point), the third derivative is required for expressing the osculating parabola, which is the parabola that best approximates... oh, you catch on quick. ;)

More specifically, if you remember the fact that four points uniquely determine a parabola, you can think of the osculating parabola as the limiting case of the parabola through four neighboring points of a given curve when those four points coalesce, or come together. The so-called aberrancy (a translation of the French "déviation") is the tangent of the angle the axis of the osculating parabola makes with the normal line, and is given by the formula

$$\tan\,\delta=\frac1{3\varrho}\frac{\mathrm d\varrho}{\mathrm d\phi}=\frac{\mathrm d y}{\mathrm d x}-\frac{1+\left(\frac{\mathrm d y}{\mathrm d x}\right)^2}{3\left(\frac{\mathrm d^2 y}{\mathrm d x^2}\right)^2}\frac{\mathrm d^3 y}{\mathrm d x^3}$$

where $\varrho$ is the radius of curvature and $\phi$ is the tangential angle.

From these considerations, one could derive an expression for the osculating parabola: given a curve represented parametrically as $(f(t)\quad g(t))^T$, the parametric equations for the osculating parabola of the curve at $t=t_0$ are

$$\begin{pmatrix}f(t_0)\\g(t_0)\end{pmatrix}+\frac{\varrho\;\cos^4\delta}{2}\begin{pmatrix}\cos\,\phi&-\sin\,\phi\\\sin\,\phi&\cos\,\phi\end{pmatrix}\cdot\begin{pmatrix}(u^2-2)\tan\,\delta-2u-\tan^3\,\delta\\(u+\tan\,\delta)^2\end{pmatrix}$$

Here for instance is the cardioid $(2\cos\,t+\cos\,2t\quad 2\sin\,t+\sin\,2t)^T$ and its osculating parabola at $t=2\pi/3$:

and an animation of the various osculating parabolas for the curve $(3\cos\,t-2\cos\,3t\quad 3\sin\,t-2\sin\,3t)^T$:

Further, one could also give a geometric interpretation for the fourth derivative; what one now considers is the osculating conic (the limiting conic through five neighboring points of a curve when those five points coalesce), and one could classify points of a plane curve as elliptic, parabolic or hyperbolic depending on the nature of the osculating conic. In this respect, the discriminant of the osculating conic depends on the first four derivatives.

A lot more information is in these two articles by Steven Schot (who also wrote a nice article on the "jerk"), and the references therein.

The rate of change of acceleration is studied in various situations in physics, mechanics and engineering design.

From wikipedia:

In physics, jerk, also known as jolt (especially in British English), surge and lurch, is the rate of change of acceleration; that is, the derivative of acceleration with respect to time, the second derivative of velocity, or the third derivative of position. Jerk is defined by any of the following equivalent expressions:

$$ \vec j=\frac {\mathrm{d} \vec a} {\mathrm{d}t}=\frac {\mathrm{d}^2 \vec v} {\mathrm{d}t^2}=\frac {\mathrm{d}^3 \vec s} {\mathrm{d}t^3}$$ where

$\vec a$ is acceleration, $\vec v$ is velocity, $\vec s$ is position and $\mathit{t}$ is time.

Jerk is a vector, and there is no generally used term to describe its scalar magnitude (e.g. "speed" as the scalar magnitude for velocity).

Think of roller coaster designs.

Indeed, in mechanics, the fourth derivative is also studied. It is called Jounce or Snap. From wikipedia:

In physics, jounce or snap is the fourth derivative of the position vector with respect to time, with the first, second, and third derivatives being velocity, acceleration, and jerk, respectively; in other words, the jounce is the rate of change of the jerk with respect to time.

$$\vec s =\frac {d \vec j} {dt}=\frac {d^2 \vec a} {dt^2}=\frac {d^3 \vec v} {dt^3}=\frac {d^4 \vec r} {dt^4}$$

Sometimes even the higher derivatives are nontrivial and come into play. Think of a sudden impact, an earthquake, a shock, or a lightning effect in electrical systems. Constructs like Dirac Delta function are very convenient for dealing with such situations. For instance, for a lightning, there is approximately a very high surge of current for a very brief instant and for any smooth approximation all the higher derivatives are nonzero. So you take the limit and manipulate it as if everything were concentrated at a single point.

For the position function $p=f(t)$, you probably know that the first derivative $f'(t)$ gives the instantaneous velocity, and that the second derivative $f''(t)$ gives the instantaneous acceleration.

The rate of change of the acceleration is called the "jerk" (also known as the "surge", the "jolt", or the "lurch"). So the third derivative $f'''(t)$ would give the instantaneous jerk.

It is a common theme in applied math that you can easily interpret first and second derivative or moment (in case of probability theory), but after that, trouble begins.

That being said, the third derivative is used in calculating the torsion of a curve.

Let's review an example (rather poor one, I admit). Let's work in $\mathbb{R}^3$ with a Cartesian coordinate system $x$, $y$, $z$ and the associated basis $\mathbf{e}_1$, $\mathbf{e}_2$, $\mathbf{e}_3$. Then let $\gamma$ be a unit circle given by $x = \cos \varphi$, $y = \sin \varphi$, $z = z_0$, parametrized naturally (by $\varphi$). Then $\dot{\gamma} \times \ddot{\gamma} = \mathbf{e}_3$, and then $\tau = {\dddot{\gamma}}^3 = 0$. I guess this particular curve is not very instructive, but I can't think of a better one off the top of my head :)