A formal word for 'disemvowel'
Solution 1:
There isn’t one.
If you are going to write about this in a formal context (for example an academic article on how communication works in various forms of ‘limited’ media, such as texting), you would do one of two things:
Use an explanatory phrasing throughout, like “abbreviate words by removing vowels”, vel sim.
Use the term disemvowel with an explanatory note and perhaps ‘scare quotes’ the first time you use the word, and treat it as a context-specific technical term thereafter, thus removing its air of informality.
There are other verbs that have meanings similar to disemvowel, such as aphaeretise, syncopate, apocopate (for removing sounds from the beginning, middle, and end of words, respectively), synaeretise (for diphthongs turning into monophthongs), contract (a subset of syncopating only applied in multiverbal contexts, such as not being able to undergo syncope to n’t, but only when it appears after an inflected verb), etc.
These words are all more general, however, than disemvowel—they refer to the loss of any sound(s), whether consonant(s) or vowel(s)—and they are all based on the actual, spoken language, not the writing that secondarily represents spoken language. Disemvowelment, on the other hand, is an artificial by-product of specific (and very recent) types of written language: it is the removal of only one type of orthographical glyphs (vowels), and it is a removal that has no counterpart in the spoken language.
When vowels (and/or consonants) are syncopated in writing, it is to represent that they have been dropped in speech; when words are disemvowelled in writing, their pronunciation does not change: they are pronounced as though the removed vowels are still there.
Solution 2:
What makes disemvowel informal, it seems, is the coiner's apparently conscious choice (I can't believe it was by accident) to make it sound like a totally unrelated word corrupt a word (disembowel) that might have even worked on its own, if metaphorically. So while as far as I know Janus is right, there is no formal word, I don't see any reason why we can't make one up that doesn't make any kind of hyperbolic analogy. To that end, I'll put forth abjadify, after the word for alphabets like those of Arabic and Hebrew that often leave out vowels.
Actually, while I did just make that word up, I thought it might be a good idea to google it. Turns out someone else has already been there.