Inconsistent output of `du` command

Solution 1:

At first I thought maybe it's accounting for hidden files but that doesn't explain nearly 4GB of hidden files/folders.

How did you measure this? I think you do have 4 GB of hidden files/directories, or maybe even more (in a sense) if there are hardlinks involved.

This will tell you (I dropped -h to get more exact results; tail is to pass the "total" only; tested in bash):

du -sc ./{.[!.],..?}*  | tail -n 1   # hidden
du -sc ./*             | tail -n 1   # non-hidden
du -sc ./{.[!.],..?,}* | tail -n 1   # hidden and non-hidden by wildcards
du -sc ./              | tail -n 1   # directory as a whole

(syntax taken from this answer).

The first command is what you're missing. Get a sum of the first and the second result; it will be equal to the last two results, unless...

Unless there are hardlinked files between the hidden and non-hidden part. If so, the sum will be even greater because these files will be counted twice when you sum by hand, but only once when du sums them for you in its single pass. In this case there is even more space taken by hidden files/directories than you think (although some of this space is "shared" with non-hidden part). Note there is -l option of du that controls this behavior:

-l, --count-links
count sizes many times if hard linked