"/usr/bin/ld: cannot find -lX11" error when installing htk
The following will also enable this to compile properly by making sure the 64bit and 32bit libraries are available.
sudo apt-get install libx11-dev:i386 libx11-dev
I found that you can avoid this error easily if you can afford not to build hslab
./configure --disable-hslab
Which is (I think) the only part that actually relies on using X.