Rise of Nations modding units

Trying to address your specific questions, without writing a full modding guide.

Is a 'rock paper scissors' system implemented somewhere in the game code outside the .xml file, or can you change unit behavior in the .xml file alone?

Additional combat effects are in balance.xml, in the same folder as unitrules.xml. The balance.xml file is rather large and lists a damage muliplier percentage for all possible match-ups of units. This includes specific unit types, referred to as Object Masks. For example "Flag_M_OBJMASK_MOUNTED". The object masks are explained at the top of unitrules.xml, and each unit's set of object masks is defined in the same file.

As an example the Heavy Machine Gun has OBJ_MASK of FTG, corresponding to F=Foot, T=Tank/War Machine, G=Gun

Where can you alter the text which shows in the in-game popup windows that tell the player that unit x is strong vs. unit y but weak against unit z?

This could depend on what you want to do with tooltips. But for existing units look in translated_strings.xml, at lines 2150,2151,3167,3169 and 2203-2261. They aren't really id'd but you should be able to figure it out.

For example, changing the entry on line 2240 to "Foot Archers, Stack Exchange" changes the tooltip as shown in the screenshot below

Is unit strength controlled only by the hit-point and attack strength in the .xml file or are there hidden combat effects of unit specific code elsewhere in the game?

In addition to damage bonuses/penalties balance.xml, there are other factors in unitrules.xml. The other important combat effect you are forgetting is Armor, which mitigates damage. Low damage units are ineffective against units with high armor. Other indirect factors you may be forgetting are Range and Speed, which for example, contribute to why Ranged Cavalry are effective against Heavy Infantry.

What are the specific effects of any 'rock paper scissors' settings? Does a unit strong vs. another unit inflict 2.5x normal damage or some other value?

A unit being strong or weak vs another does not indicate a specific multiplier. The data in balance.xml sets the damage multiplier to a specific value. It can be read as a percentage. Some examples below, with excerpts from balance.xml

War Elephants EMW are listed as "Strong vs. Heavy Infantry". In practice they do double damage (200%) to Heavy Infantry

ENTRY name="War_Elephant"
...
Flag_H_OBJMASK_HEAVY_INF="200"

Destroyers get varying bonuses against Submarines and Aircraft.

ENTRY name="Destroyer"
...
Submarine="700" Attack_Submarine="700"
...
Flag_3_OBJMASK_AIR="300"

While Light Tanks are listed as "Strong vs. Modern Infantry", but neither the tank or infantry have bonus or penalty against each other. This includes when checking their applicable object masks.

ENTRY name="Light_Tank" ... Flag_I_OBJMASK_MODERN_INF="100"

Flamethrowers are not listed as being weak vs. tanks, however Flamethrowers do less damage (66% normal) to various Tanks

ENTRY name="Flamethrower"
...
Light_Tank="66" Tank="66" Main_Battle_Tank="66"