To prevent SQL-injection in user-defined formulae, is character whitelisting enough?

No, using a regular expression like the one you show would not protect against unauthorized expressions.

The example of allowing a user to write an expression and executing a query including that expression is SQL injection, by definition.

The way to prevent SQL injection is never to copy the user's input into SQL syntax at all. But that means users cannot execute their custom expressions verbatim.

Here's how one might provide custom functions safely: allow a user to submit an expression for your review. You would vet it, and if you determine it is safe, you can store that expression as an option they can choose in the future. For example, your UI would present a list of approved expressions, and they would choose one by its id. Then on the server-side, your could would look up the expression they specified in their request, and use that in an SQL query. Thus the dynamic part of the query could only be content that you had previously reviewed, not their literal input.