How to know if someone is downloading torrents in my wifi network

Here's one comprehensive article. It suggests four approaches:

• Port-based analysis
• Protocol-based analysis
• Client-based analysis
• Behavioral analysis

Don't worry about who's doing it and stop them instead - if you log into the router, disable upnp, set the default to 'deny' for all ports and then only enable the ones you use (the services are normally listed in most domestic/soho routers) and see if anyone complains.

Ryan's answer is great, though I would start by maintaining a very strict access policy on your network. Use MAC address filtering and use a very strong WPA2-PSK key. Essentially, preventing unwanted traffic starts with preventing unwanted users.

QoS can be difficult to configure if you are totally unfamiliar. Can I have your router model?