Using UDF on a USB flash drive

Solution 1:

First, I zeroed completely the drive before creating the UDF filesystem with:

dd if=/dev/zero of=/dev/sdx bs=512

This is to avoid any leftover superblocks or other metadata which could confuse the operating systems' filesystem type detection (at least zeroing the first sector should be needed, to obliterate the partition table; the first few sectors are not used by UDF, and a leftover partition table could really confuse things). You could also use the count=1 switch on the dd command, in order to more-quickly zero just the first 512 bytes of the drive (where the MBR usually is located within), though this was not tested.

To create the file system, the command I used was:

mkudffs --media-type=hd --blocksize=512 /dev/sdx

mkudffs command will become available on Debian-based Linux distros (such as Ubuntu) after installing a udftools package:

sudo apt-get install udftools

The default blocksize for mkudffs is 2048, which is wrong for a USB flash drive (which uses 512-byte sectors). Since the block size is used to find the filesystem metadata, using a wrong block size can make it not be recognized as a UDF filesystem (since the anchor will not be where the filesystem driver is expecting). Note that the mkudffs man page is wrong; 512 is a valid value for the block size (and the code explicitly accepts it).

I also used the whole drive instead of a partition; this should be more compatible.

The result of my testing so far:

• Linux with the most recent kernel (2.6.31, from Ubuntu 9.10): works.
• Linux with an older kernel: needs the bs=512 option to mount, because it incorrectly used 2048 instead of the device sector size (fixed in commit 1197e4d).
• Windows Vista: works.
• A brand-new Mac: works.
• Windows XP: can read fine, but gives "access denied" when trying to write; also seems to think the disk is full.

While I have not so far tried to create a file larger than 4G in it, I see no reason why it would not work.

Given that it worked perfectly on all recent operating systems (only having to mount manually on Linux, which will not be needed anymore as soon as Ubuntu 9.10 and Fedora 12 are out), and worked read-only in Windows XP (which was a surprise to me; I was expecting it to not recognize the filesystem at all), using UDF instead of FAT32 or NTFS in big USB flash drives seems a good idea.

Solution 2:

CesarB did a great job getting to the crux of the issue. One thing that can't be underscored enough is how important it is to use the proper block size when formatting UDF.

Inspired by CesarB's post (and my other research/testing), I wrote a script to automate the process of formatting in UDF--using the properly detected sector size. See format-udf on GitHub. Notable features:

• Formats a block drive (hard drive or Flash drive) in Universal Disk Format (UDF)
  • UDF revision 2.01 used for maximal compatibility
  • First 4096 sectors are zeroed out to erase any existing MBR (necessary for proper UDF detection)
• Resulting file system can be read/written across multiple operating system families (Windows, OS X, and Linux)
• Runs on any OS having a Bash environment

Because of the last point, this script I wrote cannot be used on Windows. However, the script will run on OS X and Linux. After doing so, Windows should be able to magically detect the newly formatted UDF drive.

To directly answer the questions posted, format-udf will:

• choose the appropriate tool for formatting based on operating system and environment
• automatically detect and populate all parameters necessary for formatting
• maximize OS compatibility (see GitHub page for compatibility chart)
• yield the maximum feature set (and minimal limitations) that the asker is looking for

Solution 3:

I seem to recall having done that, the problem I found is that the linux version I had mounted it read only, as the driver had not been built for r/w. It did work in windows, and I think mac.

Yeah, a good solution is hard to find. For a while I had an external drive with a fat32 partition that had drivers for win and mac, a mac partition, and a big ext3 partition. It worked, but it meant installing drivers. Neat trick was it was also bootable on a mac (fw&usb), you have to leave space and take some notes, then you can add partitions via the command line and a mac partition table as well.

The world needs a free, usable by everything, file system. ZFS would be a nice choice. :-)