Why doesn't this code simply print letters A to Z?

From the docs:

PHP follows Perl's convention when dealing with arithmetic operations on character variables and not C's.

For example, in Perl 'Z'+1 turns into 'AA', while in C 'Z'+1 turns into '[' ( ord('Z') == 90, ord('[') == 91 ).

Note that character variables can be incremented but not decremented and even so only plain ASCII characters (a-z and A-Z) are supported.

From Comments:-
It should also be noted that <= is a lexicographical comparison, so 'z'+1 ≤ 'z'. (Since 'z'+1 = 'aa' ≤ 'z'. But 'za' ≤ 'z' is the first time the comparison is false.) Breaking when $i == 'z' would work, for instance.

Example here.

Because once 'z' is reached (and this is a valid result within your range, the $i++ increments it to the next value in sequence), the next value will be 'aa'; and alphabetically, 'aa' is < 'z', so the comparison is never met

for ($i = 'a'; $i != 'aa'; $i++) 
    echo "$i\n";