What does the encryption setting of uTorrent do?

Solution 1:

What does encryption Do?

The RC4 encryption obfuscates not only the header but the entire stream. This means that it’s very hard for your ISP to detect that the traffic you are generating comes from BitTorrent.

Answer
  1. The goal of BitTorrent encryption is to obfuscate BitTorrent traffic and thereby make it harder for ISPs to detect. The encrypted network connections are harder to identify as BitTorrent streams, and therefore are harder to block.

  2. The whole point of BitTorrent is its ability to share files with complete strangers. So, encryption does not hide the actual data people are sharing, everyone can still connect to a BitTorrent swarm, record your IP-address, and send you an infringement notice.

(src)

Solution 2:

In the end I got my answer from this article, from Bram Cohen, the inventor of BitTorrent:

The so-called ‘encryption’ of BitTorrent traffic isn’t really encryption, it’s obfuscation. It provides no anonymity whatsoever, and only temporarily evades traffic shaping.

So it's called encryption but it's really just obfuscation, anyone checking the traffic can still find out what is being downloaded/uploaded.

Solution 3:

Some ISPs (Internet Service Providers) actively interfere with P2P activities in order to reduce their bandwidth requirements. This causes µTorrent and other file sharing download speeds to become slow. To avoid this, µTorrent and other clients have introduced an encryption protocol to prevent ISPs from identifying BitTorrent traffic.

-uTorrent: Enabling Protocol Encryption

A security protocol (cryptographic protocol or encryption protocol) is an abstract or concrete protocol that performs a security-related function and applies cryptographic methods. A protocol describes how the algorithms should be used. A sufficiently detailed protocol includes details about data structures and representations, at which point it can be used to implement multiple, interoperable versions of a program.

-Wikipedia: Cryptographic protocol