Why is Time Machine over-stating the amount of data that needs to be backed up?

I shuffled my home directory around and moved it back to my spinning platter drive in my MacBook. Ever since, Time Machine has been running my machine hot (the fan comes on when Time Machine starts up) and the backup data size estimates shown by Time Machine in the menu bar are huge. Many gigabytes.

But when I inspect the completed backups with BackupLoupe the actual backup size is very, very small.

This wouldn't bother me so much if it weren't for the fact that Time Machine runs my machine very hot when it runs, occupies a fair bit of CPU and takes a good 20-30 minutes to complete the hourly backup run now.

My complete exclusion list for Time Machine is currently:

~/Library/Application Support
~/Documents/Virtual Machines.localized
~/Virtual Machines

Update 1:

I excluded ~/Library/Application Support on a hunch that it might be time stamp issues on cache files that end up having no change in in their bits. But TM is still saying it's backing up >2 GB of data.

Update 2:

Ultimately I'd like to trim the amount of time Time Machine spends running on my machine. Currently it's spending upwards of 30 minutes every hour spinning on these backups.

Why is Time Machine's reported effort in the tool bar so much higher than the actual backup contents as reported by BackupLoupe? Is one of them wrong? Which one?

How can I help guide Time Machine in it's estimate of what needs to be backed up so it doesn't have to work so hard?

Solution 1:

Time Machine uses FSEvents to determine what has changed, which is based on folders rather than files. Hence, I've always assumed^† that the larger number indicates the total size of the folders it has to process. Next, within each folder it will evaluate each file. (In 2008, some wrote "Time Machine considers two files to be identical if their path and dates and size are the same", so some changes might even go unnoticed. But that's another story.)

If many files in a large folder have not changed, then the total number to actually back up is much lower, but Time Machine does not know until it has processed those. Note that files might have the extended attribute com.apple.metadata:com_apple_backup_excludeItem set, or might be excluded by Apple's factory settings, implying they won't be put on the backup. See Does Apple's Time Machine app really copy everthing on Super User.

(So: I guess you need to find large source folders that are small on the backup. Tools like Disk Inventory X or GrandPerspective could indeed show large folders, like in /private/var. Might be easy to compare that to the few folders on the backup?)

^† Now that I think of it years later: seeing the size of my Downloads folder, I doubt that it's (always) showing the folder size in the amount of data it's going to back up. Downloading another small file does NOT make Time Machine claim it needs to back up the full size of that Downloads folder. (And yes, that folder is on my backups.)

Solution 2:

A couple of suggestions that may or may not help; firstly take a look at this app, TimeTracker, it shows what was backed up and, most importantly, its size, in TM. A bit like the app in the screenshot you provided, but for TM.

A guy on a forum post I found said something about an Office identity being the problem, and it was also suggested that Virtual Machine swap files may be an issue as well. I don't think either of these folders will be located in Application support, hence why the massive backups will still happen, if they are to blame.

Hopes this helps to shed some light on the mystery!