What do you do when a sentence ends with a factorial?

The manual of style of the AMS advises:

Each mathematics equation reads as a clause or sentence and is punctuated accordingly. Authors, however, sometimes leave displayed math unpunctuated. Insert commas and periods as needed so that the equation(s), the preceding text, and the following text together read grammatically. Follow CMS, sections 12.18–12.20, for punctuating mathematical expressions and elided lists, operators, and relations.

This implies that you can choose.

I never punctuate displayed math. The capitalization, or lack of it, of the next line, the vertical spacing, and the indentation are punctuation enough. I would use a period after an inline equation, specially after a factorial, otherwise it is ambiguous if the sentence is an exclamation or an equation involving factorial.

If every sentence is required to end with a punctuation mark (which is reasonable), then IMO it's better to punctuate to avoid ambiguity.

With this convention, the sentence

The answer is 5!

is exclamatory.

Note that the above convention is especially important when posing a question involving a factorial expression:

Is the answer 5!?

The only potential headache is something like:

The answer is 5!!

But, with the above convention in mind, the meaning is clear: the writer is excited that the answer is 5!.

The accepted answer is wrong in that "This implies that you can choose". That is only true if you have an editor that corrects your choice of not punctuating your displayed equations. This is what the AMS Style Guide means with its formulation. It states in Section 12.1 that "Many authors do not punctuate series of equations even if their text punctuation is acceptable. We edit to include this punctuation."

Mathematical expressions, including displayed equations, are treated as phrases or sentences and are punctuated accordingly. (See for example Section 12.7 of the AMS Style Guide.)