What is a correct punctuation for a sentence starting with "One more thing"?

The simplest and the one I would prefer is to treat it as two sentences:

One more thing. Don't tell anyone about our conversation (on the subject of this and that ...)

This also gives you the facility to maneuver the second part.

Where the second part is short, and more important, use a colon to direct attention:

One more thing: don't tell her.

Where the whole thing is rather simple and short, just include a comma:

One more thing, it's better you don't tell her.

[EDIT] additional info, the em dash.
An em dash (—) may also be used: it serves the same purpose as the comma, with a supposedly stronger effect.

One more thing — don't tell anyone about our conversation.

A sentence like that would not be appropriate in formal writing at all. If you are reporting it as direct speech, a comma after thing would represent more or less accurately the way it might have been delivered.