In old books, why is the first letter of the word after the exclamation mark not capitalised?

In these cases the exclamation mark is not being used as terminal punctuation and does not mark the end of the sentence, so there is no need to capitalize the following word. This is an archaic usage. An example from Poe's The Tale-Tell Heart (1843) is given in the Wikipedia article (Frankenstein was written in 1818).

"On the walk, oh! there was a frightful noise."

I'd be interested to hear of it being used this way in modern texts.