Term for group of guys attempting to show who is alpha?

Solution 1:

posturing

1. To assume a certain, often exaggerated body position; pose.
2. To assume a certain attitude or behave in a certain way, especially to make an impression or gain an advantage: "They postured as Southern Loyalists to win the support of ex-Confederates" (James M. Smallwood).
3. Zoology To assume a certain position of the body or of body parts, often as part of a display.

Solution 2:

An extremely crude term for this is a dick-measuring contest:

(vulgar, figuratively) A situation in which people (usually men) compete, often over superficial characteristics, to demonstrate their worthiness, power, etc.

This term would often be seen as insulting to the participants, and would not be used in formal speech.