Can true randomness come out of mathematical rules?

Mathematics doesn't purport to generate randomness from scratch.

Probability theory is a branch of mathematics dedicated to describing randomness, in the sense that if you ever find a source of real randomness somewhere, probability can help you analyze and predict its aggregate behavior.

Beware that "real randomness" in this context is essentially just code for "randomness that behaves in the way the theory assumes that real randomness will behave". But there seems to be plenty of real-world sources of randomness that agree with this assumption, at least well enough to allow casinos and insurance companies to make a profit.

You might argue that the apparent randomness of a coin flip is really just a result of the fine details of the starting conditions being unknown and uncontrollable, which is (or could be argued to be) different from randomness in principle. On the other hand, physicists claim to have solid experimental proof that the fundamental laws of the universe at a quantum level do produce mathematically "real" randomness.

Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin. - John von Neumann

From: Various techniques used in connection with random digits", Applied Mathematics Series, no. 12, 36–38 (1951).

One kind of randomness that cannot come from "rules" is Kolmogorov–Chaitin randomness. That follows immediately from its definition.

I'm not sure what kind of answer you are expecting, but randomness is a thing in mathematics. For example http://en.wikipedia.org/wiki/Random_variable and http://en.wikipedia.org/wiki/Kolmogorov_complexity

The binary digits of $\pi$ may look random (and it is conjectured that a property called being a "normal" number, which is related to this indeed holds). However, if you knew that the successive results of a coin flip were based on walking through the binary expansion of $\pi$, you could make a fortune by calculating ahead and betting accordingly. Determinately making a fortune is not possible with betting on a truly fair random coin.