On macOS 10.13 High Sierra, one way to diagnose Time Machine slowdowns and other problems is to use the log shell command. Unfortunately, on my system I sometimes get an error:

% log show --last 2s --style syslog
log: warning: The log archive contains partial or missing metadata
log: cannot use --last when archive metadata is missing

This appears to prevent me from using the --last option to limit output to the most recent time interval. The failed log command returns an exit status of 65, which we can see using echo $? : 

% echo $?
65

For the benefit of web searchers, here is what I have found out about how to resolve this error message. Warning: it won’t be a fix, but I do have workaround.

If you are using the handy Consolation3 application from Howard Oakley’s Eclectic Light Company, this error manifests as an alert box reading, “log command returned error number 65”. (I suspect that the “65” in Consolation3‘s alert box is the exit status of the log command, propagated forward.)

Consolation3 dialogue: “log command returned error number 65”

What I want to happen is something similar to what I get when I use the --start option with a timestamp in the recent past. The successful command returns an exit status of 0.

% date 
Fri 10 Jan 2020 18:43:56 PST 
% log show --start "2020-01-10 18:44:00" --style syslog log: warning: The log archive contains partial or missing metadata
Skipping info and debug messages, pass --info and/or --debug to include.
Timestamp                      (process)[PID] 
2020-01-10 18:44:04.459968-0800 localhost xpcproxy[99272]: (libsystem_info.dylib) Created Activity ID: 0x1f9010, Description: Retrieve User by ID 
2020-01-10 18:44:04.460147-0800 localhost xpcproxy[99273]: (libsystem_info.dylib) Created Activity ID: 0x1f9020, Description: Retrieve User by ID 
…[snip, remaining output elided]… 
% echo $? 
0

Note: I observed this problem on macOS High Sierra 10.13.6. My account has administrator access, which the log command requires to return any content. The access is demonstrated by fact that the log --start command succeeds.

I have a workaround for the “archive metadata is missing” error 65: whenever log show --last fails, I calculate a date and time to use as a starting point, and use that with log show --start instead. The command will show log entries until the current moment (but you can also use --end to set an end time). The format to use is “yyyy-mm-dd hh:mm:ss“. See the example above.

I also have a very unsatisfying fix to this problem: wait 10 days and hope the problem goes away by itself.

That approach has worked for me. Here is the result of me running the log show --last command, 9 days after it failed in the example above. Note that the warning, “The log archive contains partial or missing metadata”, no longer appears, and the --last option now works.

% date 
Sun 19 Jan 2020 07:26:30 PST
% log show --last 2s --style syslog
Skipping info and debug messages, pass --info and/or --debug to include.
Timestamp                      (process)[PID]
% echo $?

But to what might this “partial or missing metadata” refer?

While investigating, I came across the very useful essay, Inside the macOS log: logd and the files that it manages, by same Howard Oakley of the Consolation3 app and the Eclectic Light Company website. This article pointed me to the directories /var/db/diagnostics/ and /var/db/uuidtext/.

% cd /var/db/diagnostics
% ls -lF@ Persist
total 533688
-rw-r--r--@ 1 root admin   2730128 21 Dec 14:02 0000000000000061.tracev3
    com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin  10477520 22 Dec 20:45 0000000000000062.tracev3
    com.apple.logd.metadata 40
…[snip, omitted for brevity]…
-rw-r--r--@ 1 root admin  10479128 18 Jan 22:56 000000000000007b.tracev3
    com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin   2431808 19 Jan 06:29 000000000000007c.tracev3
    com.apple.logd.metadata 40
% ls -lF@ Special
total 35624
-rw-r--r-- 1 root admin       1544 10 Jan 06:35 0000000000000048.tracev3
-rw-r--r-- 1 root admin       1536 13 Jan 04:43 0000000000000049.tracev3
-rw-r--r-- 1 root admin       1416 18 Jan 04:56 000000000000004b.tracev3
-rw-r--r-- 1 root admin     258128 19 Jan 05:25 000000000000004c.tracev3
-rw-r--r--@ 1 root admin   2082432  8 Jan 02:58 000000000000004d.tracev3
    com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin   2087952 10 Jan 21:52 000000000000004e.tracev3
    com.apple.logd.metadata 40
…[snip, omitted for brevity]…
-rw-r--r--@ 1 root admin   2097344 18 Jan 23:11 0000000000000054.tracev3
    com.apple.logd.metadata 40
-rw-r--r--@ 1 root admin    984016 19 Jan 06:27 0000000000000055.tracev3
    com.apple.logd.metadata 40

Oakley says that the macOS log system rotates these files: it gradually prunes out old and unnecessary content, then deletes old files altogether. The date stamps back him up. Note that the oldest tracev3 file in Special/ is only 10 days old. Older files existed in this directory when I originally had the problem, but are now gone.

It is interesting that many (but not all) of these files contain an extended file attribute com.apple.logd.metadata. Oakley, in xattr: com.apple.logd.metadata, log metadata, says “their content and function are unknown”. I speculate that the “metadata” term in the error message might be a reference to the data in the com.apple.logd.metadata xattr.

So, a wild guess is that a file in /var/db/diagnostics/Special/ had a corrupted or missing xattr “com.apple.logd.metadata”. In the course of a week, this file became old enough and was discarded. Once that file was discarded, its metadata was gone, and the so was the problem.

Note: I originally wrote this post in January 2020, as a question and then a self-answer on the Ask Different Q&A website, How to add metadata to log archive, so that log show --last works? I have polished it a little and included it here, for my own record. It might be worth checking that Ask Different question from time to time; someone might post a better answer there someday.

2 Comments »