What does the "Bitmap Caching" option do in the Remote Desktop Client?

Caching bitmap means that images and other bitmap resources are locally stored on the client computer for reusing them later. This way, the remote server or PC doesn't send images twice reducing the amout of data sent and saving your bandwidth usage.

The option makes particularly sense for slow (low bandwidth) connections, less if you connect to a machine in the same local area network.

If you enable the option the Remote Desktop client caches bitmaps into a BMC file located on the client hard disk in (example for Windows XP)

C:\Documents and Settings\<username>\Local Settings\
     Application Data\Microsoft\Terminal Server Client\Cache 

folder.

Note (interesting if your computer is part of a domain): since the cache folder is stored in a "Local Settings" folder, it won't be replicated as part of a roaming user profile.

It caches bitmaps, like the desktop background, icons, etc. :)

Here's a Microsoft article for you.

From the document:

Persistent Bitmap Caching

Persistent bitmap caching was added in addition to the memory caching of bitmaps and glyphs that existed in TS 4.0. The bitmaps from the server are now saved to disk on the client machine, which allows cached bitmaps to be reused between client sessions and also provides a much larger cache size (10MB vs. 1.5MB). As seen in Figure 4 (in the linked document), the addition of persistent caching decreases the amount of data sent over the network connection, which in turn reduces the amount of time it takes to render bitmaps on the screen, proportional to speed of the network connection

It's likely that some of that is now out of date, since the doc was produced 9 years ago, but hopefully it still gets across the reasoning behind bitmap caching.