Which is the more accepted spelling between “doughnut” or “donut” in formal academic prose?

When asking about contemporary usage in academia it is almost always better to simply check usage in the specific domain. For example, in mathematics and engineering the word "doughnut" / "donut" may be used to refer to a torus.

"Doughnut" currently seems preferred in both the humanities and sciences -- although both are used -- based on the following quick inspections.

For the humanities, JSTOR ("a digital library of academic journals, books, and primary sources") lists:

• 'doughnut' 4699 / 'donut' 2314

Google Scholar returns more results and with higher citation counts for 'doughnut'. Top results are almost all metaphorical uses in the sciences, like the "banana-doughnut paradox" in geophysics. There are fewer listings for 'donut', and the highest cited is actually an article with JR Donut as a named author, not even a usage as a common noun! (Ironically, even this was an OCR error while scanning the name of JR Donat).

Note that searching JSTOR and Google Scholar with a recent time-window may indicate a narrowing of the gap in recent years. While even recent top-cited articles predominantly use 'doughnut', the total proportion of 'donut' titles has grown in the past decade. However different groups of academics are often writing for very specific audiences. If you are citing previous research that uses a particular spelling then you should probably follow that usage. The editors of a specific journal may also have opinions based on their chosen style guides or references, e.g. AP Style prefers "doughnut." Your field or publisher may vary!