Definition of set.

In formal set theory, the closest thing to a definition of a "set" we get is "something which every objects either belongs to or doesn't belong to" -- in other words, if you have any object, you can ask the set whether the thing you have in your hand is one of its member or not and it will answer either yes or no. And it will give you the same answer each time you ask it about the same object.

Not that this description does not openly speak about "distinct" objects -- there just isn't any way for a set to claim to contain some object "more than once" or "only once". All we can do with it is ask whether something is in it or not, and get a yes/no answer.

If you find yourself in a situation where you need to reason about collections that may contain the same object more than once -- and sometimes we do find ourselves in such a situation -- you're free to do that. Such collections are usually known as multisets, and they need a somewhat different formalization than sets, but there's nothing bad about them.

It all depends on what you need, and the words just serve to communicate which of the concepts you're working with at the time. If you're talking about things that give a yes/no answer to "do you contain this?", you say "set". If you're talking about things that give a numeric answer to "how many of this do you contain?", you say "multiset".

The naming reflects that in practice sets turn out to be what you need rather more often than multisets. But don't let that stop you from using multisets when they are what you need.

In mathematics definitions (and axioms) are the attempts to formalize some informal notions.

Sets come to formalize the notion of a "collection", so we can talk mathematically about collections of objects. The collection makes a distinction between two things which are not equal, but that's it. So if I open my wallet, and look at my coins, while I might have two coins of the same value, they are not the same coin.

Why this notion and not the notion of a multiset, where we also care about repetition? Because we want something bare, with as least structure as possible. You can always add structure to things which don't have any, but you can't remove structure from your atomic notion. (For example, a field is a ring, is an abelian group, is a group, is a set. But if the most basic objects in your world is a field, you can't strip it from structure anymore.)

In modern terms, sets are objects of a universe of set theory. It may sound circular, but only at the level of natural language where I used the term "set theory" to define "set". Where set theory is an informal, but rather well-understood term for theories whose concern is formalizing the notion of set into a mathematical object.

And why do we want them to be with the least structure possible? Because using the axioms of set theory we can prove that we can endow them with pretty much any structure we want (well, up to a certain limitation, but certainly we can endow them with the structure of a multiset).