How to go about defining a new category in category theory?

Why did they choose "group homomorphisms" for the morphism?

The concept of category is supposed to catch three things:

  1. Objects, whatever that means;
  2. Morphisms between those object, i.e. how we move between objects, whatever that means;
  3. Composition of morphisms, i.e. how to combine morphisms into new morphism;

We also want the composition to be associative, because that property seems to be almost everywhere. And we also want each object to have a special "identity" morphism, which seems to be almost everywhere as well.

So given a group as an object, there's not much choice for a morphism. It could be simply a function but then we just treat group category as set category. It could be just an abstract "arrow", but then we have to manually define each one and how to compose them, problematic. Plus what would be the point? So what other choice do we have? Group homomorphisms come naturally.

Note the important thing: the concept of group and group homomorphism was invented first, and only then someone realized it fits the category theory language. I'm pretty sure the same can be said about almost every category out there.

Why are groups themselves the objects and not the elements of the group like is your common reference point in the set-theoretic perspective?

You can define objects however you want. But given a category such that its objects are elements of some group, what would morphisms between those elements be? There doesn't seem to be a natural way to define them. Which doesn't mean it is impossible. The most important thing though is that some categories are more popular than other for a simple reason: usefulness.

You will here a lot about the category of topological spaces, groups, modules, rings, and so on, simply because these concepts are useful and share lots of properties that can be described in the language of category theory.

Why did they come up with all these labels like "the monomorphisms in Grp", and how did they come up with all the other statements? Etc.

I'm not exactly sure where names come from, its a linguistic question. But for ideas, this is often the following process. Consider the monomorphism example. In the group theory a monomorphism is defined as an injective group homomorphism. It is similar for set, modules, etc. Now how can we express this property in the language of category theory? Because injective functions/homomorphisms seem to be useful. And they are defined similarly. The problem is that those definitions are based on elements of set, group, module, etc. But in categories we don't have elements. We only have objects, morphisms and composition. So is it futile? Not necessarily, it requires some skill to express such property in those terms, and sometimes it can be done, sometimes not. That's not an automatic process, and requires some imagination.

Note that the categorical definition of "monomorphism" is not necessarily the same as "injective mapping". There are few categories where these concepts diverge, e.g. in the category of all divisible groups and group homomorphisms there are categorical monomorphisms that are not injective.

Do you just draw the categorical "objects" out of a hat, and then pick an arbitrary function (homomorphism I guess) as the morphism?

No. Typically those objects and morphisms occure together naturally. Topological spaces come naturally with continous maps. Groups come naturally with group homomorphisms. Vector spaces come naturally with linear maps. So typically you take some mathematical structures and functions between those structure that preserve that structure.

However note that there are arguably more complicated (and abstract) categories that are still useful. The notable example is treating a partially ordered set $(X,\leq)$ as a category whose objects are elements of the set and there's a unique abstract morphism (an arrow) $x\to y$ if and only if $x\leq y$ in $X$. This idea can then be used for example in the so called sheaf theory.