Does the phrase "instantaneous frequency" make sense?

Yes, you can make sense of the concept of instantaneous frequency. Imagine a time-dependent phase $\mathrm e^{i\phi(t)}$. For $\phi=\omega t$, we would call $\omega$ the angular frequency, and this is $\mathrm d\phi/\mathrm dt$. Thus we can regard $\mathrm d\phi/\mathrm dt$, the rate of change of the phase angle with time, as an instantaneous frequency. We can still do this if the amplitude is also time-dependent, by ignoring the amplitude and focussing on the rate of change of the phase.

If we only have a real signal, we would in general have to make an arbitrary decision what part of its change is due to a change in "amplitude" and what part is due to a change in "phase", since taking the real part of different complex signals can lead to the same real signal. However, there may be situations in which it makes sense to regard the signal as having a constant amplitude, and to regard all changes as changes in phase. The example you give is such a case, since the signal oscillates between $+1$ and $-1$. In such a case, we can consider the signal as the real part of a complex signal of constant amplitude.

That is, given a real signal $f(t)$ and a constant amplitude $A$, we can consider the complex signal $f(t)\pm\mathrm i\sqrt{A^2-f(t)^2}$, where the sign changes whenever the signal reaches an extremum with $f(t)=\pm A$. Then we can calculate the phase as $\phi(t)=\arccos(f(t)/A)$, where we let the phase wrap around and increase beyond the usual range $[0,2\pi]$ to avoid discontinuities. Thus we can define an instantaneous angular frequency as $\omega(t)=\mathrm d\phi(t)/\mathrm dt=\mathrm d/\mathrm dt(\arccos(f(t)/A)$. In your case, the result would be $\omega(t)=\mathrm d/\mathrm dt(t^2)=2t$.

To add a slightly different viewpoint to @Joriki's excellent answer, communications engineers often restrict the notion of instantaneous frequency to (real-valued) signals that can be expressed as $\text{Re}\{a(t)\exp(j\phi(t))\}$ where $a(t)$ is a (possibly complex-valued) continuous signal that is varying much more slowly with $t$ than $\phi(t)$ is, and $\phi(t)$ is a continuous function of $t$ that is differentiable for all $t$ except possibly for a discrete set of time instants. Then the instantaneous (radian or angular) frequency is the derivative of the phase. For the OP's problem, we have $a(t) = -j, \phi(t) = t^2$ and everyone is in agreement that the instantaneous frequency of $\sin(t^2)$ is $2t$.

To add to a previous reply: when a signal is not expressible as a sine/cosine or complex exponential of a function of time, it may still be decomposable into a superposition of such signals. In that case, the instantaneous frequency concept would apply to each component separately, thus resulting in a spectrum concentrated on a region shaped like a web, whose lines, in fact, are called "partials".

To obtain the spectrum, you'll first have to segregate the components. The simplest way to do that is to rely on the assumption that the different components are not crowding the same area of frequency space too much, to apply a time-frequency transform (e.g. windowed fourier transform, or wavelet or S-transform) and then apply the instantaneous frequency to each frequency channel of the resulting spectrum. If the components are well-enough spaced apart, then each one will register over a given frequency range in the spectrum -- namely the range that is "tuned into" that signal component.

I illustrate this here http://www.youtube.com/watch?v=iVO9-Y_Dgxw in a side-by-side contrast with a readout from a usual fourier-based spectrogram. The 3 spectra on the right in the view are of the same signal at three different resolutions (showing that the particular method can be applied at multiple resolutions, leading to equally sharp results) and the phase is color-coded in each one. This YouTube demo shows what partials looks like at a resolution of 5280 pixels/second (again the phase is color coded) http://www.youtube.com/watch?v=gDMixb7-_LM