St. Basil's cathedral, Moscow steeple shape
Short answer: differential geometry did not exist at the times of Ivan IV, and was not needed for the job. The reason is two-fold:
- The shape is not as regular as it might appear. Below is the panoramic view of Saint Basil's Cathedral. I drew the contours of the domes in the front, and below you can see them superimposed. As you can see, they are similar (and appear to come in pairs), but not quite the same, and not only in size, but in shape as well.
- The domes were usually made of wood and were subject to remaking after the fires, which at the time happened quite often. In the reconstruction process no masonry was required, so most likely the local carpenters were given the job. It also seems that the main engineering problem was setting the cross, which tended to become bigger and heavier with time (the tradition in Russia drifted from that of Byzantium, and also lead to the use of chains for holding it as you can see in the pictures). Below is the picture of the insides of an onion dome (this one is from Munich). So it appears that only the general appearance was set by the architector, and not the precise shape (one would also not expect the architector to be around at the time of dome remaking).
Onion domes were widespread in Russia at least as early as in the XIII century, as indicated by their appearance in the drawings of the time. Research (the paper by S. Zagraevsky in Russian) shows that they were very inlikely inspired by any foreign building and should have depicted actual churches in Russia. While for the end of the XVI century I can imagine a discussion whether there was something similar to differential geometry already, there certainly was none in the XIII century.
As for the use outside Russia, onion domes start appearing in the XVI century. The earliest example in Europe appear to be Convent of the Franciscan Sisters of Maria Stern, picture below.
Similar dome shapes also appear at the time in Mughal architecture, though they look more like a sphere with a cone on top of it (because they are built of stone, apparently).
Other examples and an overview can be found in Wikipedia, although the sections on the history should be taken with a grain of salt.