Convert binary string to bytearray in Python 3

Despite the many related questions, I can't find any that match my problem. I'd like to change a binary string (for example, "0110100001101001") into a byte array (same example, b"hi").

I tried this:

bytes([int(i) for i in "0110100001101001"])

but I got:

b'\x00\x01\x01\x00\x01' #... and so on

What's the correct way to do this in Python 3?

Solution 1:

Here's an example of doing it the first way that Patrick mentioned: convert the bitstring to an int and take 8 bits at a time. The natural way to do that generates the bytes in reverse order. To get the bytes back into the proper order I use extended slice notation on the bytearray with a step of -1: b[::-1].

def bitstring_to_bytes(s):
    v = int(s, 2)
    b = bytearray()
    while v:
        b.append(v & 0xff)
        v >>= 8
    return bytes(b[::-1])

s = "0110100001101001"
print(bitstring_to_bytes(s))

Clearly, Patrick's second way is more compact. :)

However, there's a better way to do this in Python 3: use the int.to_bytes method:

def bitstring_to_bytes(s):
    return int(s, 2).to_bytes((len(s) + 7) // 8, byteorder='big')

If len(s) is guaranteed to be a multiple of 8, then the first arg of .to_bytes can be simplified:

return int(s, 2).to_bytes(len(s) // 8, byteorder='big')

This will raise OverflowError if len(s) is not a multiple of 8, which may be desirable in some circumstances.

Another option is to use double negation to perform ceiling division. For integers a & b, floor division using //

n = a // b

gives the integer n such that
n <= a/b < n + 1
Eg,
47 // 10 gives 4, and

-47 // 10 gives -5. So

-(-47 // 10) gives 5, effectively performing ceiling division.

Thus in bitstring_to_bytes we could do:

return int(s, 2).to_bytes(-(-len(s) // 8), byteorder='big')

However, not many people are familiar with this efficient & compact idiom, so it's generally considered to be less readable than

return (s, 2).to_bytes((len(s) + 7) // 8, byteorder='big')

Solution 2:

You have to either convert it to an int and take 8 bits at a time, or chop it into 8 byte long strings and then convert each of them into ints. In Python 3, as PM 2Ring and J.F Sebastian's answers show, the to_bytes() method of int allows you to do the first method very efficiently. This is not available in Python 2, so for people stuck with that, the second method may be more efficient. Here is an example:

>>> s = "0110100001101001"
>>> bytes(int(s[i : i + 8], 2) for i in range(0, len(s), 8))
b'hi'

To break this down, the range statement starts at index 0, and gives us indices into the source string, but advances 8 indices at a time. Since s is 16 characters long, it will give us two indices:

>>> list(range(0, 50, 8))
[0, 8, 16, 24, 32, 40, 48]
>>> list(range(0, len(s), 8))
[0, 8]

(We use list() here to show the values that will be retrieved from the range iterator in Python 3.)

We can then build on this to break the string apart by taking slices of it that are 8 characters long:

>>> [s[i : i + 8] for i in range(0, len(s), 8)]
['01101000', '01101001']

Then we can convert each of those into integers, base 2:

>>> list(int(s[i : i + 8], 2) for i in range(0, len(s), 8))
[104, 105]

And finally, we wrap the whole thing in bytes() to get the answer:

>>> bytes(int(s[i : i + 8], 2) for i in range(0, len(s), 8))
b'hi'