Is it possible to build a torrent client using only HTML(5) and JavaScript?
No. It's not.
This is because the WebSocket specification falls outside of HTML5 and JavaScript ;-) That being said, opening up the question to "using features supported natively in [progressive/upcoming] browsers" then...
...still no :-)
This is because WebSocket requires a special handshake to setup with restrictions. It's not a free-for-all open-TCP-fest. The approach would require either
clients to be modified to accept WebSocket requests (as well as dealing with any cross-site access issues)
or, more realistically, a server to bounce through
There's a recent implementation based on WebRTC that works in node and the browser: https://github.com/feross/webtorrent
This is possible using Chrome Apps APIs: chrome.socket and chrome.fileSystem.
There is at least one pure JavaScript implementation for Google Chrome: JSTorrent.
There's no good reason why this can't be done today. BitTorrent/uTorrent both have code to support websocket connections with binary frames. However, they are currently compiled without support (due to political/product reasons I think). I used to work at BitTorrent and another engineer (Arty) wrote the support. For a while it was really cool being able to download torrents onto iPads from mobile Safari. (saving directly to Google Drive funnily enough)
HTTP trackers could be configured to send Access-Control-Allow-Origin headers.
Also the tracker announce protocol could use a bit in the "key" field to indicate that the client accepts websocket connections. Then a special argument like "typewant" could indicate that the announce response should only return clients who have sent that bit.
The tracker protocol could also be extended similarly to support negotiation of WebRTC P2P DataChannel connections, so that connections could be made directly browser<->browser.
For now, we have the chrome.socket platform API, and jstorrent, a chrome packaged app (designed for ChromeOS mainly)