Why is .bash_history periodically wiped?

This is the second time it's happened. I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean. I use iTerm2 and on MacOS 10.13.6. I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.

Solution 1:

There can be multiple reasons as to why this happens - I'll try to outline how it works below.

However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.

You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.

This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.

When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.

This is all handled by the /etc/bashrc_Apple_Terminal script.

Now that you know how it works, here are some possible causes for what you're seeing:

• You may accidentially have removed the history yourself (history -c)

• Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file

• You have software installed that periodically empties/removes .bash_history

These reasons are not that likely, I would say.

Now there's one last possible cause:

There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.

This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.

The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.