how to get sha1sum to output binary?
Try converting hex to base64. This answer is one option. There are a number of other implementations.
How can I convert from hex to base64?
Maybe something like:
echo mydata | sha1sum | xxd -r -p | base64
... would solve your problem.
See https://unix.stackexchange.com/questions/82561/convert-a-hex-string-to-binary-and-send-with-netcat for similar question.