What does the "Use proxy to perform DNS queries (SOCKS v5 only)" option in Pale Moon/Firefox's proxy settings mean?

Does this really mean that it has been bypassing my specified proxy for every DNS lookup, instead making those directly with my main connection, just because I did not actively check (or even notice) that box?

Yes.

And who would ever want to use a proxy but not have DNS queries go through it?

People who use a proxy for reasons other than privacy, e.g. developers who want to access a remote server according to their local /etc/hosts names, or corporate users who want their locally configured "domain search suffix" to be honored.

In general, just as with VPNs, the original purpose of proxy servers had nothing to do with privacy or information hiding – they were used for authentication and filtering (and in the case of HTTP proxies, caching and more caching), e.g. using a corporate SOCKS gateway to access internal services while working from home, or using a local HTTP caching proxy so that the entire campus wouldn't need to fetch the exact same website assets 100 times through a slow radio link. (With HTTP proxies the domain name is resolved by the proxy but that's not the point I was making.)

Admittedly the Mozilla proxy settings page hasn't been keeping up with the times much, especially when SOCKS usage shifted towards it being primarily an interface to Tor.

(If I remember correctly, Mozilla even had the ability to send hostnames to the SOCKS v5 proxy for a long time before this checkbox got added, but it was only visible through about:config for a long while.)

Surely the SOCKS protocol(s) must be able to communicate which version it runs, though? So that it could auto-enable this crucial privacy "feature" if v5 is detected?

No, that's why the settings page asks you to choose between SOCKS v4 and v5 in the first place. The same server may support both, but it has no way of telling the server which versions are supported.

(There is actually the original SOCKS v4, which strictly dealt with IPv4 addresses, and SOCKS "v4a" which added the capability to use DNS names before version 5 got published. I think Mozilla only supports the former?)

This option was introduced in Pale Moon version 27.6.0 in 2017 (link).

Its purpose is to prevent DNS leaks:

A DNS leak refers to a security flaw that allows DNS requests to be revealed to ISP DNS servers, despite the use of a VPN service to attempt to conceal them.[1] Although primarily of concern to VPN users, it is also possible to prevent it for proxy and direct internet users.

This would only be applicable to a SOCKS v5 proxy. With this option, the browser asks the proxy to connect to the host using the host-name of the host instead of its IP address. It's up to the SOCKS5 proxy then to do the lookup.