What's the rule for pronouncing “’s” as /z/ or /s/?

Is there a hard rule for what sound the 's makes? In words like John's, Dave's, man's, lord's, etc. it makes a /z/ sound, but in words like that's, it's, ship's, poet's, etc., it makes an /s/ sound.

If the final sound in the base of the word is voiced, we use the voiced alveolar sibilant /z/.

If the last sound in the base is an unvoiced consonant, we use /s/.

However, if the last sound in the base form is another sibilant of any description—/s, z, ʃ, ʒ, tʃ, dʒ/—we need to insert a vowel /ɪ/ to make the ending audible. Because this vowel is voiced the very last sound will be /z/. In other words, if the last sound is a sibilant we add /ɪz/:

• /bʌs/ ---> /bʌsɪz/ (buses)
• /bʌz/ ---> /bʌzɪz/ (buzzes)
• /bʊʃ/ ---> /bʊʃɪz/ (bushes)
• /ru:ʒ/ ---> /ru:ʒɪz/ (rouges)
• /hʌtʃ/ ---> /hʌtʃɪz/ (hutches)
• /bʌdʒ/ ---> /bʌdʒɪz/ (budges)

The word ending spelled apostrophe "s" is a phonemic /z/ in all the instances I can think of. (But English spelling is not very regular, so there could be exceptions.) However, English has a morphophonemic rule that converts a voiced obstruent (e.g. /z/) to the corresponding voiceless phoneme (for /z/ that would be /s/) when the /z/ is immediately preceded by a voiceless obstruent phoneme.

The term "obstruent" means that a sound is naturally voiceless, as compared with a "sonorant", which is naturally voiced. English vowels, nasal consonants, liquids, and glides are sonorants. Other English phonemes are obstruents.

The obstruents have a constriction in the mouth that retards the airflow sufficiently to prevent the vocal cords from vibrating without making some special articulatory adjustment (such as laxing the cheeks or lips to let them puff out, or dropping the jaw, both of which let the mouth cavity hold more air so air flow past the vocal cords can speed up). Many languages of the world do not have voiced obstruent phonemes, but English does have some, in addition to voiceless obstruent phonemes.

For determining whether a /z/ is preceded by a voiceless obstruent, what counts is the phonemic form of a word. Another morphophonemic rule of English inserts a vowel between two coronal obstruent fricatives or affricates, where "coronal" means that a sound is articulated with the tip or blade of the tongue. In "pitch's", e.g., a vowel is inserted between "pitch" and the ending "'s" (which is phonemic /z/), and so the /z/ is not changed to /s/, because /z/ is no longer immediately preceded by a voiceless obstruent.

This covers all the examples you mentioned, since none of /n,v,d/ is a voiceless obstruent, and all of /t,p/ are voiceless obstruents. I have used slashes for all the examples, rather than brackets indicating actual pronunciations, because I agree with you that this is a phonemic phenomenon and not a phonetic one.