Are enchantments beyond Protection 10 effective?
Solution 1:
No, Protection type enchantments are capped, and the cap is fairly low.
Damage reduction based on Armor and Armor Enchantments has some peculiar mechanics which are detailed on the Minecraft Wiki page for Armor1 (some emphasis mine).
Each protection enchantment protects against specific types of damage. The amount of damage reduction depends on the Enchantment Protection Factor (EPF) provided by that enchantment.
[...]
The EPFs for each enchantment and level are the result of the following formula:
floor ( (6 + level^2) * TypeModifier / 3 )When a player or mob wearing armor is subjected to damage, the EPFs of all applicable enchantments are added together, capped at 25, multiplied by a random value between 50% and 100%, rounded up, and capped again at 20. The damage is then reduced by 4% per point of total effective EPF (for example, a total effective EPF of 20 reduces damage by 80%).
The "TypeModifier" in the equation depends on the enchantment itself, and is 0.75 for Protection, 1.25 for Fire Protection, 1.5 for Blast and Projectile Protection and 2.5 for Feather Falling.
Note how the EPF is capped twice. We can't do anything about the random number multiplication and the subsequent second cap, but we can make sure to have an EPF of at least 25, to ensure maximum Protection. Using the above formula, we can calculate the EPF for a single armor piece with a certain Protection enchantment level.
level | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | ...
EPF | 1 | 2 | 3 | 5 | 7 | 10 | 13 | 17 | 21 | 26 | 31 | ...
From this table, it is immediately evident that you can reach the EPF cap with a single Protection X enchantment. Anything beyond Protection X has no effect on the game mechanics.
Using a full set of armor, and optimizing for lower enchantment levels, you can reach cap with 3 Protection V and 1 Protection IV enchantment (3*7+5=26).
1 This part of the page has been roughly the same in March 2013, when 1.5 was released.