Why are Leicester & co pronounced as they are?

Solution 1:

These are all quite old names with many syllables—an ideal place for ellipsis, elision, and contraction.

The -cester bit is from the Old English word ceaster, which in itself is borrowed from Latin castrum. It is thus cognate to castle. (The palatalisation of the initial /k/ to /tʃ/ is common in Old English and was more widespread in some dialects than in others, so at some point, there were probably two variants around: caster /kastər/ and ceaster /tʃastər/.)

In Old English, ceaster referred to a Roman town or settlement (in England). As tchrist notes in his comment, some other places that have this element in their names include Lancaster, Doncaster, and Manchester.

The first part of names that have -cester tend to be inherited place names of either Anglo-Saxon or (even earlier) British Celtic stock, often names of rivers or other local toponyms.

Going from Old English down through the centuries, the cities the first element of whose names ended in a consonant (like Lan-, Don-, Man-) usually kept a fuller version of the word ceaster, retaining the initial consonant. The ones whose first element ended in a vowel, on the other hand, ended up having the initial /tʃ/ (or /k/, as in Lancaster and Doncaster) in an intravocalic position, where it was quite likely to be weakened—something that happens often in place names, especially longer ones.

Thus, Leicester and Gloucester were once pronounced as they’re written, with a /tʃ/ (or perhaps /k/); but that consonant was weakened over time and eventually disappeared altogether, leaving the vowels free to contract into single monophthongs, too.

Solution 2:

The cester/chester ending can be traced back to the Roman occupation: castra is Latin for camp, so these placenames indicate Roman army encampments (or more likely permanent garrisons, given the names' persistence).

The Grammarphobia blog gives a good run down of the evidence which traces these pronunciations through history; the scansion in Shakespeare, for example, suggests that the elision was in common use in his time. The article also speculates on why the pronunciation and spelling are so divergent: perhaps because the people writing down the placenames (the literate scribes of Domesday Book; the monarch's tax-collectors &c) weren't the ones living there, coupled with the usual spelling free-for-all that only started to be regularised with mass-printing.

Sometimes pronunciation can swing back into line with spelling; while Cirencester is now generally pronounced as spelled, it had been elided to cicester.

Similar elisions are common in other patterns of placenames (Kirkby:kirby; Kircudbright:Kurkoobree; Marylebone:marleybone/mairbun/m'bn) and so probably represent a universal trend towards elision in familiar and locally-shared placenames.

The relative lack of shortening in such names as Manchester, Lancaster, Chichester, Colchester looks to be caused by the middle consonant 'catching' the speaker and preventing further elision. The examples in the question, by contrast, don't have a strong middle consonant.