For a probability distribution, what is the most fundamental object : the probability density function or the cumulative distribution function?

This is mostly to exampnd on the great comments, which address this pretty well. More generally, if you get into a little measure theory, the CDF can be used to define the underlying probability*measure* on the range of the random variable. Specifically -- the CDF assigns numbers to intervals in a consistent way whereas the density (if it exists) does not (you need to integrate it).

Therefore, the CDF is an example of a measure $\mu$ on half-open intervals $(-\infty,x]$ (in the one dimensional case).

In contrast, the density requires that the measure is differentiable. A more general version of the usual calculus derivative is the Radon-Nikodym derivative, which handles cases where you are not restricted to the reals.