Meaning and etymology of the “ite” and “ine/ene” endings of mineral names
The reason is in its etymology. I got this from a mineralogy site:
The suffix "ite" is derived from the Greek word lithos (from its adjectival form -ites), meaning rock or stone.
"ine" means :
suffix of adjectives of Greek or Latin origin, meaning “of or pertaining to,” “of the nature of,” “made of,”
So, minerals named that depends on the earlier part of the name. Olivine would be a mineral that pertains to olive, and so possibly has an olive colour.
Though the derivation of -ite from Greek λίθος lithos ("stone") is often asserted, it seems to be derived from the Greek suffix -ίτης -ites via Latin and French. This is the pattern in derivation of non-mineralogical terms.
I don't think that one can prove the -ites derivation, but the lithos derivation is a very implausible one to linguists. Furthermore, before the 20th century, well-educated scientists, even mineralogists, knew some Latin and Greek and would have been aware of both suffixes as possibilities in coining new terms. It would go against the grain for a classically educated scientist to add -ite rather than -lite when naming a stony mineral.
Greek λίθος lithos ("stone") is the source of the suffixes -lite and -lith.