Why do disks fragment?

As a part of regular Windows maintenance, I defragment my hard drive. But why does the hard drive fragment on NTFS and FAT* systems? Apparantly ext* does not, why is this? Should I also be defragmenting my USB drives?

Fragmentation is not the issue it was 30 years ago. Back then you had hard drives that were scarcely faster than floppies, and processor memory sizes that were minuscule. Now you have very fast drives and large processor memories, and sometimes substantial buffering on the hard drive or in the controller. Plus sector sizes have gotten larger (or files are allocated in larger blocks) so that more data is inherently contiguous.

Operating systems have gotten smarter as well. Whereas DOS 1.x would have fetched each sector from disk as it was referenced, a modern OS is able to see that you have a file open for sequential access and can reasonably predict that you'll be fetching additional sectors once you've consumed those you have now. Thus it can "pre-fetch" the next several (dozen) sectors.

And any more it's often better to not have a file contiguous. On a (large) system where the file system is spread across multiple drives a file can actually be accessed faster if it is "spread" as well, since multiple disks can be seeking the file simultaneously.

I defragment every 2-3 years, whether my box needs it or not.

[I'll add that the important thing is not so much whether the data on the disk gets defragmented as whether the free space does. FAT was terrible at this -- unless you defragged things kept getting worse and worse until there were no two contiguous blocks of free space. Most other schemes can coalesce free space and allocate pieces in a somewhat "smart" fashion so the fragmentation reaches a certain threshold and then stabilizes, rather than getting worse and worse.]

ALL file systems fragment. ext and other Linux file systems fragment less due to the way they're designed - to quote Wikipedia regarding the Linux Network Administrators' Guide:

Modern Linux filesystem(s) keep fragmentation at a minimum by keeping all blocks in a file close together, even if they can't be stored in consecutive sectors. Some filesystems, like ext3, effectively allocate the free block that is nearest to other blocks in a file. Therefore it is not necessary to worry about fragmentation in a Linux system.

I'd note though that ext4 has online defragmentation so eventually fragmentation IS an issue, even with Linux file systems.

Windows file systems have their clusters placed wherever there's space to put them, and defrag runs around and replaces them. With Linux, files are preferentially placed where there's enough space.

I'd note though, Windows 7 has scheduled defragmentation runs, so it isn't really necessary to run defrag manually.