What is the definition of a mathematical point?

A point is defined in the elements as that which have no parts, but what does this mean? There is nothing in the physical world that doesn't have an extension. You can't say a point is primitive notion because primitive notions are for objects which fall under our senses but mathematical points don't. So how to define a mathematical point?

In Euclid's geometry a point is taken as a given in much the way you describe. It is essentially up to the geometer to gain an appreciation of the idea. Riemann (n.d.) refers to similar aspects, stating: "It is known that geometry assumes, as things given, both the notion of space and the first principles of constructions in space. She gives definitions of them which are merely nominal, while the true determinations appear in the form of axioms. The relation of these assumptions remains consequently in darkness; we neither perceive whether and how far their connection is necessary, nor a priori, whether it is possible. From Euclid to Legendre (to name the most famous of modern reforming geometers) this darkness was cleared up neither by mathematicians nor by such philosophers as concerned themselves with it." He later finds (p.149) "[I]t is therefore quite likely that the metric relations of space [(meaning forms of measurement, loosely speaking)] in the infinitely small do not agree with the assumptions of geometry, and in fact one would have to accept [an alternative theorem] as soon as the phenomena can thereby be explained in a simpler way." No further progress in this matter has been achieved in more recent times. In Hilbert’s ‘Foundations of Geometry’ (1971) the concepts of point, line, plane and the relation of betweenness remain simple (Goheen in Hilbert 1971). The bother in your concern is that is assumes an anthropocentric idea of space, which according to Kant is not one that allows us to get as concepts properly because we see the world through the 'lens of the mind'. Von Neumann (1996) identifies that contemporary mathematics finds its roots in our observations of the world (no matter the level of abstraction that results). In that context you question is problematic. To answer further would require swimming to the bottom of the pool of philosophy, a very dark place indeed.