How to add metadata to log archive, so that `log show --last` works?
Whenever I run the log show
command from the command line, I get a warning message, log: warning: The log archive contains partial or missing metadata
.
This appears to prevent me from using the --last
option to limit output to the most recent time interval. The failed log
command returns an exit status of 65.
% log show --last 2s --style syslog
log: warning: The log archive contains partial or missing metadata
log: cannot use --last when archive metadata is missing
% echo $?
65
What I want to happen is something similar to what I get when I use the --start
option with a timestamp in the recent past. The successful command returns an exit status of 0.
% date
Fri 10 Jan 2020 18:43:56 PST
% log show --start "2020-01-10 18:44:00" --style syslog
log: warning: The log archive contains partial or missing metadata
Skipping info and debug messages, pass --info and/or --debug to include.
Timestamp (process)[PID]
2020-01-10 18:44:04.459968-0800 localhost xpcproxy[99272]: (libsystem_info.dylib) Created Activity ID: 0x1f9010, Description: Retrieve User by ID
2020-01-10 18:44:04.460147-0800 localhost xpcproxy[99273]: (libsystem_info.dylib) Created Activity ID: 0x1f9020, Description: Retrieve User by ID
…[snip, remaining output elided]…
% echo $?
0
How can I get the log archive to contain sufficient metadata so that log show --last 2s
works as I want it to?
This is observed on macOS High Sierra 10.13.6. My account has administrator access, which the log
command requires to return any content. The access is demonstrated by the log --start
command succeeding.
I have a very unsatisfying answer to this question: wait 10 days and hope the problem goes away by itself.
That approach just worked for me:
% date
Sun 19 Jan 2020 07:26:30 PST
% log show --last 2s --style syslog
Skipping info and debug messages, pass --info and/or --debug to include.
Timestamp (process)[PID]
% echo $?
0
Note that the message, log: warning: The log archive contains partial or missing metadata
no longer appears, and the --last
option now works.
While investigating, I came across the very useful essay, Inside the macOS log: logd and the files that it manages, by Howard Oakley. This pointed me to the directories /var/db/diagnostics/
and /var/db/uuidtext/
.
I observe that subdirectories /var/db/diagnostics/Persist/
and /var/db/diagnostics/Special/
contain opaque .tracev3
data files:
% cd /var/db/diagnostics
% ls -lF@ Persist
total 533688
-rw-r--r--@ 1 root admin 2730128 21 Dec 14:02 0000000000000061.tracev3
com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin 10477520 22 Dec 20:45 0000000000000062.tracev3
com.apple.logd.metadata 40
…[snip, omitted for brevity]…
-rw-r--r--@ 1 root admin 10479128 18 Jan 22:56 000000000000007b.tracev3
com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin 2431808 19 Jan 06:29 000000000000007c.tracev3
com.apple.logd.metadata 40
% ls -lF@ Special
total 35624
-rw-r--r-- 1 root admin 1544 10 Jan 06:35 0000000000000048.tracev3
-rw-r--r-- 1 root admin 1536 13 Jan 04:43 0000000000000049.tracev3
-rw-r--r-- 1 root admin 1416 18 Jan 04:56 000000000000004b.tracev3
-rw-r--r-- 1 root admin 258128 19 Jan 05:25 000000000000004c.tracev3
-rw-r--r--@ 1 root admin 2082432 8 Jan 02:58 000000000000004d.tracev3
com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin 2087952 10 Jan 21:52 000000000000004e.tracev3
com.apple.logd.metadata 40
…[snip, omitted for brevity]…
-rw-r--r--@ 1 root admin 2097344 18 Jan 23:11 0000000000000054.tracev3
com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin 984016 19 Jan 06:27 0000000000000055.tracev3
com.apple.logd.metadata 40
Oakley says that the macOS log system rotates these files: gradually prunes out old and unnecessary content, then deletes old files altogether. The date stamps back him up. Note that the oldest tracev3
file in Special/
is only 10 days old. Older files existed in this directory when I originally had the problem, but are now gone.
It is interesting that many (but not all) of these files contain an extended file attribute "com.apple.logd.metadata". Oakley, in xattr: com.apple.logd.metadata, log metadata, says "their content and function are unknown". I speculate that the "metadata" term in the error message might be a reference to the data in the "com.apple.logd.metadata" xattr.
So, a wild guess is that a file in /var/db/diagnostics/Special/
had a corrupted or missing xattr "com.apple.logd.metadata". In the course of a week, this file became old enough and was discarded. Once that file was discarded, its metadata was gone, and the so was the problem.