Does Time Machine backup iCloud Drive files that are not stored locally?

No, these files are not backed up by Time Machine.

iCloud has its own deleted file recovery, available from iCloud.com Settings → Restore Files, but this isn't a substitute for backups since modified files are not stored, only those deleted completely.

If you change a file it should show up on your local drive long enough for a backup. (Edit) I checked a file marked as only on iCloud and it is indeed on my Time Machine backup (Edit #2) An odd test scenario: I actually forced the file off of my local system "brctl evict testdocument.pages" Then I changed the file using iCloud.com. Within minutes it appears that iCloud on my Mac detected that the file changed and actually re-downloaded the physical file to my local drive. The next Time Machine backup did in fact place the iCloud changed file into the Backups.backupdb folder. I verified the file contents by actually examining the Time Machine file for change I made on iCloud.com.