The answer is historical. In Old English, voiced and voiceless fricatives were phonologically equivalent. (The current English graphemes <v> and <z> did not exist in Old English.) Between voiced segments, voiceless fricatives became voiced--and this is called sibilant softening. Then, in Middle English, these fricatives became phonemic. That's why you see these discrepancies. Words with <s> between voiced segments after the Middle Ages don't fit in this pattern. That's why lousy (from the 14th cy) got /z/ (softened /s/), whereas mousy (from 1853) got /s/.


The clue may well be in your observation that "lousy" is no longer perceived as being to do with lice, so there's less of a pressure to avoid the sound change and keep the pronunciation similar to the base form.

Another possibility to consider is that the voicing change accompanying the -y has "gone out of fashion" (possible pair to compare: 'rout' > 'rowdy', where there also appears to have been a voicing change). I wonder if people can come up with other pairs, especially that are earlier derivations, for comparison?

When derivation involves a sound change, it does happen elsewhere that a few words for whatever reason "escape" the change, e.g. because trends in where sound changes occur have changed while the use of the derivation hasn't. For example, compare the vowel change in pairs such as 'pl*ea*se'~'pl*ea*sure', 'comp*e*te'~'comp*e*titive' with the failure of this change to occur in 'obese'~'obesity', and the optionality of the change in 'beast'~'beastial'.