gpg2: no secret key

I've been using enigmail for over a year without issue, and today it doesn't work.

I found the following interesting fact:

gpg --decrypt something.gpg # this works
gpg2 --decrypt something.gpg # this fails

So something is broken with gpg version 2 on my machine.

This led me to see that:

gpg --list-secret-keys # reads from ~/.gnupg/secring.gpg
gpg2 --list-secret-keys # reads from ~/.gnupg/pubring.gpg (pubring?!) 

This seems to be the root of the problem... of course gpg2 can't find the secret key because it's looking in the wrong file.

How could my gpg2 fail when my gpg works fine? I don't see any options to specify where the secret keys are read from.

Anybody have any ideas?

Response to @grawity:

Thanks, I appreciate your help. I ran strace, and I see what you are talking about.

However, even after gpg2 --import ... I am seeing no difference in behavior. I can only get it to work if I reboot (without starting gpg-agent), run gpg2 --import ..., then run gpg2 --decrypt .... After that sequence, thunderbird + enigmail also behaves nicely. However, after 15 minutes or so (my guess is the password I entered to decrypt has expired), then gpg-agent is back to its old behavior. This sequence is repeatable.

So here is some output if it helps clear anything up:

output of gpg2 -K:

/home/<username>/.gnupg/pubring.gpg
---------------------------------
sec   rsa4096/AAAAAAAA <date> [SC]
uid         [ultimate] <description of me>
ssb   rsa4096/BBBBBBBB <date> [E]

output of gpg-connect-agent

> keyinfo --list
S KEYINFO <keygrip associated with AAAAAAAA> D - - - P - - -
S KEYINFO <keygrip associated with BBBBBBBB> D - - - P - - -
OK

output of gpg2 -v -r <my email> -e testfile

gpg: using PGP trust model
gpg: using subkey BBBBBBBB instead of primary key AAAAAAAA
gpg: This key belongs to us
gpg: reading from 'testfile'
gpg: writing to 'testfile.gpg'
gpg: RSA/AES256 encrypted for: "BBBBBBBB <description of me>"

output of gpg2 -v -d testfile.gpg

gpg: public key is BBBBBBBB
gpg: using subkey BBBBBBBB instead of primary key AAAAAAAA
gpg: using subkey BBBBBBBB instead of primary key AAAAAAAA
gpg: encrypted with 4096-bit RSA key, ID BBBBBBBB, created <date>
      "<description of me>"
gpg: public key decryption failed: Operation cancelled
gpg: decryption failed: No secret key

…of course gpg2 can't find the secret key because it's looking in the wrong file.

That's not the only file it's looking at.

In GnuPG 1.x (and 2.0), the "secring" used to have a duplicate copy of your keyblock's public data as well, so it was entirely self-contained (and the only difference between gpg -k and gpg -K was which file it'd read), but at the same time more difficult for the program to maintain.

In GnuPG 2.1, secret keys are now stored independently – they're maintained by gpg-agent, which keeps them in ~/.gnupg/private-keys-v1.d/. So both gpg -k and gpg -K now have to read the OpenPGP information from the pubring, but the latter additionally asks gpg-agent about which certificates have associated secret keys. If you're using strace, you should notice a connect() call right after reading the pubring.

If GnuPG didn't automatically migrate the keys, just import the whole secring directly:

gpg2 --import ~/.gnupg/secring.gpg

To check the agent's contents manually:

$ gpg-connect-agent 
> keyinfo --list
S KEYINFO 926145FFCA32B3E6E079A0CF73EA77C40733A349 D - - - P - - -
S KEYINFO BACFB81EAFC864F4AB2926E8B1F55AD579F78D1A D - - - P - - -
S KEYINFO FF3D1DD51B9C79E148CCCEA5F7F3E25EC96048B7 D - - - P - - -
S KEYINFO 4D29EF1460F164CDB11D0FC0247214660ACDD60F D - - - P - - -
S KEYINFO 06B13685B9AA429B9CABCE480930D74B991C8DF0 D - - - P - - -
S KEYINFO B28DB8D045654E8A6A40466A07FCD9E432935E29 D - - - P - - -
OK
> /bye
$ 

These are "keygrips" – compare them with GnuPG's secring:

$ gpg --list-secret-keys --with-keygrip
/home/fred/.gnupg/pubring.kbx
--------------------------------
sec   ed25519 2018-08-18 [SC]
      2357E133AD5D24F6CB2C1B0CEF4F7ED27E252632
      Keygrip = 4D29EF1460F164CDB11D0FC0247214660ACDD60F 
uid           [ultimate] Fred Foobar <[email protected]>

Eventually, I decided the issue was that I was using Debian Unstable and there was a version mismatch introduced by an apt-get dist-upgrade. I suppose that is why they call it "Unstable".