Capitalization of "Theorem 1" in mathematics papers

I think this is a matter of style. There's no reason internal references to theorems shouldn't be treated the same way as chapters. The grammatical reason for capitalizing them would be that they are specific designators, but it's somewhat of a judgment call as to whether they are. Here, Google Ngrams come in very handy. If you look at whether people capitalize Chapter 3, they usually do:

And they capitalize Theorem 3 a slightly higher percentage of the time.

Google Ngrams also shows that very few people would capitalize page 3.

Theorem 4.5 is the title of the item in your book. You would refer to it in the same way that you would refer to a chapter or section of the book, or the name of another book, by reproducing the same capitalization as the title's original appearance.

It looks unusual because we are used to making other references within the same book by chapter number (or page number, or whatever), rather than by title. This is purely a matter of convenience; you have a general idea of where "chapter two" is where you'd probably have to look up "Elliptical Integrals". It just happens that in this case the title contains a number.