How can a MBR formatted hard drive exceed 1.81 TiB capacity?

Solution 1:

Your USB drive presents itself to the host computer with a logical sector size of 4096 bytes, and that's what enables it to be partitioned with an MBR partition table that can use more than 2 TB of the drive's space, even in Windows XP. MBR partition table entries are 32 bits and Windows XP uses 32-bit math to calculate sector addresses from the partition table, allowing only the first 2^32 sectors of a drive to be used. 2^32 sectors x 4096 bytes/sector = 16 TB. With 512 bytes/sector, only the first 2 TB of a drive could be used by an MBR partition table. (2^32 sectors x 512 bytes/sector = 2 TB)

However, your drive is not 4k native. There are two layers of translation:

1. The USB enclosure contains a USB-to-SATA bridge that translates from 4096 bytes per logical sector at the USB interface with the host computer to 512 bytes per logical sector at the SATA interface with the drive inside.
2. The drive inside internally translates from 512 bytes per logical sector on its SATA interface to 4096 bytes per physical sector. (That's 512 byte/sector emulation or "512e".)

A SMART query passed though the USB bridge to the SATA drive inside reveals the truth (run here on my own Western Digital My Book 3 TB external USB drive in Windows XP):

C:\Program Files\GSmartControl> smartctl -a -d sat pd11
smartctl 6.5 2016-05-07 r4318 [i686-w64-mingw32-xp-sp3] (sf-6.5-1)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ12xxxxx
LU WWN Device Id: 5 0014ee 2063xxxxx
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical

The internal SATA drive reports 512 bytes per logical sector and 4096 bytes per physical sector: that's 512 byte/sector emulation, or "512e".

However the drive as seen by the host computer connected by USB has a logical sector size of 4096 bytes:

C:\> wmic DISKDRIVE get bytespersector, caption
BytesPerSector  Caption
4096            WD My Book 1140 USB Device

and that's how Windows XP is able to use all 3 TB of the drive's space.

If you need more proof than that, you could remove the drive from its enclosure and connect it directly to a motherboard's SATA connector: you'd then find that the drive has a logical sector size of 512 bytes, and the drive's partitions would all appear as 1/8 their previous size and be unreadable due to incorrect partition table entries caused by the sector size mismatch. Plenty of people have done this and then had to fix the problem by rewriting the partition table.

I've checked several different external USB drives and they are all 512e internally despite being configured for Windows XP compatibility with 4096 bytes per logical sector at the USB interface: WD My Book 3 TB, WD Elements 3 TB, HGST Touro 4 TB, WD Easystore 8 TB, WD Easystore 12 TB, WD Easystore 14 TB.

Your drive came with an MBR partition table (or someone changed it to MBR at some point) to make it compatible with Windows XP. Newer drives come with GPT partition tables, which require Windows Vista or later.

The WD Quick Formatter tool can be used to change a Western Digital external USB drive between 512 bytes/sector with a GPT partition table (for Vista and later) and 4096 bytes/sector with an MBR partition table (for Windows XP compatibility).

Solution 2:

Your drive uses Advanced Format 4Kn, where the drive exposes its native 4K sector size instead of emulating 512-byte sectors (512e). This ensures compatibility with devices that do not support GPT by extending the MBR limit to 17.6 TB (16 TiB), thereby avoiding the need to use GPT.

• MBR permits a maximum of 2³² sectors. This means that with conventional 512-byte sectors, the maximum supported disk size is 2.2 TB (2 TiB). However, with 4K sectors, the maximum disk size becomes 17.6 TB (16 TiB).

• According to this article (via Google Translate, lightly edited), several external hard drives exceeding 2.2 TB in capacity use 4K native sectors with an MBR partition table for compatibility with Windows XP:

  Certain external Seagate (FreeAgent GoFlex) and Western Digital (My Book Essential) external 3-TB disks can also be used on XP at full capacity thanks to another technical twist. They do not [connect] to the system via their USB or FireWire interface [with the usual] 512-byte sectors, but rather with 4-KB sectors. This bypasses the 2TB limitation of the MBR. But even this trick has drawbacks: programs that access the disk low-level - such as formatting tools - may not work on the disks. [T]hey are [also] not suitable [as boot disks].

• The manufacturer is taking advantage of the fact that this is an external hard drive, with an assumption that it will only be used for storage and not booting. However, this creates another set of compatibility problems, where programs that rely on 512-byte sector sizes may not work correctly:

  • Partitioning tools that do not support 4Kn drives will not function correctly with this drive.

  • Some database programs use direct disk I/O for performance and data integrity. These programs will not function correctly if they are not designed to handle 4K sectors.

• The underlying internal hard drive may well be 512e—if this is the case, the enclosure electronics are exposing 4K sectors to the system for compatibility.

Solution 3:

I got in touch with a WD support technician and he told me that it was an anomaly and that I should convert the disk to GPT instead to make sure I don't encounter any accidental data loss in the future.

However, he was unsure about whether or not it would actually cause any data loss but recommended I convert the disk to GPT and he also let me know that mine was the first case of a MBR formatted high-capacity drive (>2TB) that he's encountered.

If you have encountered a similar situation before and know why this is or isn't an issue, please add your answer.

Disregard everything I wrote below. It may or may not be correct.

With this technology, WD has increased the block size of the HDD from 512 bytes to 4 kilobytes so Advanced Format drives can grow to 16 TB without hitting the 32-bit LBA size limit.

Stephen Foskett has explained this very well in his blog post.

Current hard disk addressing scheme tops out around 2.1 TB. There are two simple factors in this equation:

1. There are 4,294,967,296 addressable blocks in a 32-bit “namespace”, and this is all most PCs can handle
2. Each block is 512 bytes in size, and has been since dinosaurs sold the computers

If we multiply 4,294,967,296 times 512, we get 2,147,483,648 KB of capacity. In wacky base-10 storage industry speak, this is 2.15 TB of capacity.

The hard disk drive manufacturers have two roads forward, and they are not mutually-exclusive:

1. Western Digital was first out with an Advanced Format drive late in 2009. This addresses the (archaic) block size limit, increasing it from 512 bytes to 4 kilobytes. With each address now eight times larger, we need one-eighth as many addresses for a given drive size. Advanced Format drives can grow to 16 TB without hitting the 32-bit LBA size limit.

2. Seagate attacked the other variable, increasing the address space to the full 48 bits specified in the LBA standard. This gives an amazing 128 petabytes of capacity even with tiny 512 byte blocks.

Solution 4:

I'm inclined to go with the blog rather than the WD support technician's reply. I have a couple of Toshiba Stor.e Canvio 3TB drives. Both are factory formatted using 'advanced format' with cluster and sector sizes of 4k.

Drives are used on a Samsung smart TV which does not recognize GPT partition (most multi media devices will only recognize MBR).