What is Pseudo-polynomial complexity?

You have linked to other questions where this is explained fairly well for someone who understands the concept, so here comes a very brief version.

for i from 2 to n - 1:

can be rewritten as

i = 2
while(i < n - 1):
    if (n mod i) == 0:
        return false
    i = i + 1

Very often, we assume that the operations i < n - 1, i = i + 1 and n mod i are O(1). But this is not necessarily true. It is usually true for small values. And on a 32 bit machine, a "small value" is in the order of a billion.

Number that requires more than 32 bits to be represented will take more time to perform operations on than a number that fits in 32 bit. And it will take even more if it required more than 64 bit.

In practice, this rarely matters.

A very simple way to visualize this is to imagine that you get the task to implement the common mathematical operations where the operands are represented as strings. Here is a simple python function that takes two strings representing binary numbers and returns the sum as a string. It was quickly hacked together and assumes both strings has the same length. It may contain bugs and can most likely be refined. But it demonstrate the point. This function adds two numbers, but it will take longer time for longer numbers.

def binadd(a, b):
    carry = '0'
    result = list('0'*(len(a)+1))
    
    for i in range(len(a)-1,-1, -1):
        xor = '1' if (a[i] == '1') != (b[i] == '1') else '0'
        val = '1' if (xor == '1')  != (carry == '1') else '0'
        carry = '1' if (carry == '1' and xor == '1') or (a[i] == '1' and b[i] == '1') else '0'
        result[i] = val
    result[0]=carry
        
    return ''.join(result)

What makes this program so special to be measured in the amount of bits required to write out n?

There's nothing special about this particular program. At least not theoretical. In practice it is special in the sense that determining if a VERY big number is a prime is a common problem. Or to be more accurate, it would have been a much more common problem if there existed a very fast algorithm to do it. If it did, it would basically break encryption as we know it today.