Is "must" ever grammatical as a past tense verb?

I have seen uses of must that appear to be in the simple past tense. Sometimes these seem grammatical, but sometimes not. Examples that help illustrate my confusion:

He knew he must go to New York - sounds fine to me.

He went to New York because he must - sounds questionable.

Because he must go to New York, he bought plane tickets - sounds completely ungrammatical.

He must go to New York - grammatical, but with no other verbs to provide context, can't interpret this as being past-tense.

My question: is this past-tense use of must ever acceptable? Is it only acceptable in certain contexts - if so, what are those contexts and why?

(Note - I'm not looking for had to. Have is a different verb, so had to expresses the same meaning as the past tense of must, but it itself is not the past tense of must.)

Rarely, must is used as a past tense. Belshazzar, by H. Rider Haggard, has we went because we must, in a prose style which is perhaps deliberately archaic to reflect the ancient Egyptian context.

In this odd snippet, If Thoreau went because he would, Hawthorne went because he must, one might say the author is "playing with language".

But here's Ralph Waldo Emerson with What he did, he did because he must. I would not wish to say Emerson doesn't know his own language.

From comments under @Henry's answer, it seems something quite odd has been going on. Many people will know the archaic present tense mote because Freemasons & such still say So mote it be in a "ritual" context. Bizarrely, the past tense "must" eclipsed "mote" for present tense usage. But in so doing, "must" somehow almost completely lost its ability to still be used as a past tense.

In spite of all the above, ordinary mortals in ordinary contexts today should stick with the standard position put forward by other answers. Use had to for the past tense.

The reason that must does not seem to have a past tense in English is that etymologically it already was a past (or preterite) tense of motan.

Compare with the modern Dutch where ik moet means I have to while ik moest means I had to.

If you will not use had to, you can often use needed to, which fits your examples, or something similar.

The Chicago Manual of Style devotes Section 147 of its "Grammar and Usage" chapter to must:

Must denotes a necessity that arises from someone’s will [we must obey the rules] or from circumstances [you must ask what the next step is] [he went to New York because he must].

Must also connotes a logical conclusion [that must be the right answer] [that must be the house we’re looking for] [it must have been Donna who phoned].

This verb does not vary its form in either the present or past indicative. It does not have an infinitive form (to have to is substituted) or a present or past participle.

Denoting obligation, necessity, or inference, must is always used with an express or implied infinitive [we must finish this design] [everyone must eat] [the movie must be over by now].

But, to answer the question, I've found this guideline useful (it's not where I originally heard it, but its formulation is the same):

Use had to in past tense independent clauses, and use must in dependent clauses (usually noun or adjective clauses) when the sense is that of strong obligation or strong probability.

All modal verbs, of which must is one, are invariable. That is to say, they do not inflect for person, number or tense. There is no *musted. If we wish to express the past sense of a modal verb, we have to do so by other means. In the case of must we use, as you have said, had to.