Some ISPs don't charge for bandwidth used on Youtube/Netflix/Facebook. How do they monitor that? [closed]

Solution 1:

Firstly they know the YouTube IP address.

ISP's have an IP database. For example YouTube's ASN is AS15169. On the server side they would make a grouping for each service. One of them is the default grouping and this is the billing group. When you make use of default group, that usage is recorded in the system.

For example a few YouTube addresses are listed below.

root@server ~>whois -h whois.radb.net -- '-i origin AS15169' | grep ^route
route:      192.179.147.0/24
route:      192.179.148.0/23
route:      192.179.148.0/24
route:      192.179.149.0/24
route:      192.179.150.0/23
route:      192.179.150.0/
...
route6:     2607:f8b0:4016::/48
route6:     2604:31C0::/32
route6:     2620:33:c000::/48
route6:     2607:f8b0:4000::/48
route6:     2404:f340::/32

When you are trying to reach YouTube or other YouTube services (Google video storage) your phone will try to reach these IP addresses.

The ISP checks the IP address, and if it is inside the YouTube group, they don't apply charge at this group.

Another option is checking the SNI header at the initial HTTP connection. When you make a connection with HTTPS sites not all the data is encrypted.

For example, when you make a search on Google, you can see the URL in your browser like this: https://www.google.com/search?q=hello+world.

Encrypted data is /search?q=hello+world and all page content. Now you are reaching a site like www.google.com, but they don't know which page or the content inside of that page.

Some ISPs use SNI for this. For example in Turkey this method is used for making specific internet packages like 5GB internet+4GB Spotify or 7GB internet with unlimited WhatsApp. Also they use SNI for banning websites. Some websites use the same IP addresses like wikimedia.com or wikipedia.org. If they try to block Wikipedia with an IP addresses they block all Wikimedia services.

Solution 2:

HTTPS obscures the content of the traffic, but not the endpoints. So, for instance, my ISP does not know that I'm responding to this particular question, because I'm using HTTPS, but they do still know that I'm accessing content on serverfault.com port 443.

In specific cases such as the ISP/Neflix partnerships you describe, it's also common for Netflix to co-locate one of their endpoints in the ISP's data center, which then operates similarly to a CDN - when you connect to Netflix, you get the Netflix server on the ISP's own network, which makes it even easier for them to track it for purposes of the deal, since your traffic never leaves their own network. (The co-located server still needs to get the video streams from another Netflix server outside of the ISP's network, of course, but funneling everything through the co-located server allows them to cache data, use dedicated connections, aggregate streams, etc. to reduce their costs and/or pass some of those costs back to Netflix as part of the partnership deal.)

Solution 3:

Even more simple - THEIR DNS will answer with specific IP addresses of local proxies. This may not even break HTTPS - they can port forward from there. But this allows you to have the free traffic on specific IP addresses and remove those from the accounting.

Solution 4:

In addition to knowing IP addresses as explained in other answers, some CDNs offer their nodes to host at ISPs. For example: Edge nodes / Google Global Cache. Data served from those nodes never leaves ISP perimeter and never hits "outbound" traffic counter. This, incidentally, is also exactly why ISP are able to offer this traffic for free or at significantly cheaper price.

Solution 5:

It's a mix of having a look on IP endpoints i.e ip addresses and DNS. This is usually done at the so called PGW which stands for Packe Data Gateway. The PGW is part of every mobile network.