iCloud Photo Library - force upload

Solution 1:

This is not a complete answer, in that I am not sure if there is a method to force or otherwise unthrottle the uploads. However, it may be helpful to some to be able to better understand what is happening during the uploads. I believe there are some cases where the Photos system will change some parameters of how it is uploading, but can't say for certain the reasons or ways of affecting it (hopefully someone can edit this in the future if they garner more information!).

As of Sierra (macOS 10.12), the way to see the logs for the iCloud Photo Library system is as follows (you can likely achieve something similar in the Console app too):

log show --last 1h --style syslog --predicate '(subsystem beginswith[cd] "com.apple.photos.cpl")'
log stream --style syslog --predicate '(subsystem beginswith[cd] "com.apple.photos.cpl")'

The first command above will show the last 1hour of logs, the second will stream new logs from the time it is run. You may get output along the lines of the following (this is style=compact, syslog may be easier to read though)

2017-12-18 14:02:42.541 Df cloudphotosd[373:2205bf] [com.apple.photos.cpl:engine.scheduler] Client has put some changes in push repository
2017-12-18 14:02:42.541 Df cloudphotosd[373:2205bf] [com.apple.photos.cpl:engine.scheduler] We are asked to schedule a sync session from CPLEngineSyncManagerStatePush
2017-12-18 14:02:52.924 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] Notified that Server has changes to be pulled
2017-12-18 14:02:52.924 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] We are in the middle of CPLEngineSyncManagerStatePush and requested CPLEngineSyncManagerStatePull will happen later in the sync session.
2017-12-18 14:02:52.925 Df cloudphotosd[373:221837] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 72.56s (48 resources uploaded): <private>
2017-12-18 14:02:52.941 Df cloudphotosd[373:221a88] [com.apple.photos.cpl:engine.scheduler] Notified that Server has changes to be pulled
2017-12-18 14:02:52.941 Df cloudphotosd[373:221a88] [com.apple.photos.cpl:engine.scheduler] We are in the middle of CPLEngineSyncManagerStatePush and requested CPLEngineSyncManagerStatePull will happen later in the sync session.
2017-12-18 14:02:53.012 Df cloudphotosd[373:221837] [com.apple.photos.cpl:engine.scheduler] Informing transport of significant work ending
2017-12-18 14:02:53.012 Df cloudphotosd[373:221837] [com.apple.photos.cpl:engine.transport.cloudkit] Setting significant work not pending, had been set for 11.69s
2017-12-18 14:04:01.865 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] Notified that Server has changes to be pulled
2017-12-18 14:04:01.866 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] We are in the middle of CPLEngineSyncManagerStatePush and requested CPLEngineSyncManagerStatePull will happen later in the sync session.

These log commands gather all of the com.apple.photos.cpl (Cloud Photo Library) logs; you might want to further limit to a particular 'category', e.g. just the sync engine:

log show --last 1h --style compact --predicate '(subsystem beginswith[cd] "com.apple.photos.cpl") && (category beginswith[cd] "engine.sync")'

Which will show just the bulk transport messages:

2017-12-18 13:13:31.427 Df cloudphotosd[373:20ff1e] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 37.62s (40 resources uploaded): <private>
2017-12-18 13:15:12.093 Df cloudphotosd[373:21009e] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 100.20s (40 resources uploaded): <private>
2017-12-18 13:18:50.131 Df cloudphotosd[373:210ec1] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 152.81s (44 resources uploaded): <private>
2017-12-18 13:19:22.753 Df cloudphotosd[373:2126dc] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 32.26s (44 resources uploaded): <private>
2017-12-18 13:19:56.137 Df cloudphotosd[373:212866] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 33.00s (60 resources uploaded): <private>
2017-12-18 13:20:32.789 Df cloudphotosd[373:212a7c] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 36.22s (44 resources uploaded): <private>

These show how many resources (photos or videos?) are uploaded in each 'chunk', and you can see how much time has elapsed between them. I've yet to find a particular method behind how it chooses the size of the chunk, or the delay between them, but they seem to have varied for me at different times of the day, and whether I'm using the computer/Photos or not (though this could easily be due to other bandwidth usage). I suspect the number of resources is indeed photos/videos, because there are also many engine.sync.pushtotransport events with 0 resources that take less than a second, which I suspect are syncing faces, albums, etc (at least, since High Sierra, now that faces are synced to iCloud Photo Library).

They may be selected with a minimum chunk size, or based on size of recent successfully uploaded 'chunks'. My guess is they're selected to take approximately 30 seconds (based on recent chunk upload speeds), since most of mine are at least 30 seconds (and the distribution of times looks vaguely Poisson, though I've not plotted it to test). One particular thing I've noticed in the logs is messages about the Photos app itself being opened/closed (or foregrounded/backgrounded):

2017-12-18 14:33:13.367 Df cloudphotosd[373:22a51f] [com.apple.photos.cpl:engine.foreground] Client is in foreground
2017-12-18 14:33:13.367 Df cloudphotosd[373:22a51f] [com.apple.photos.cpl:engine.scheduler] Some client is in foreground, current required state is CPLEngineSyncManagerStatePush
2017-12-18 14:33:18.517 Df cloudphotosd[373:2299dd] [com.apple.photos.cpl:engine.foreground] Client is in background

The fact that this is logged suggests to me that the sync engine considers this in its algorithm for synchronising photos, e.g. it may reduce/throttle the sync activity when the Photos app is open or in the foreground (probably to ensure a better user experience). This is certainly the case for face/scene analysis (they are stopped/throttled while Photos is in the foreground/opened), so it might be a little faster to quit Photos. That said, the messages aren't particularly enlightening, and it still won't max out my upload bandwidth when I do this.

Sorry that this doesn't answer the question in a completely useful way, but it at least might help understand the situations in which uploads are completed faster or slower, even if Apple don't give commands or settings to 'force' or de-throttle the uploads.

More speculatively, I'm wondering if there's any undocumented sysctl settings like the one for Time Machine to enable/disable throttling of particular iCloud Library functions.