What exactly are the "higher order terms" (H.O.T.) in Taylor series expansion in multivariable case?

The formula is almost identical to the single variable case. You should be able to find this in any good book covering calculus on Banach spaces.

Theorem. Let $E,F$ be Banach spaces, let $A\subseteq E$ be an open set and let $f:A\to F$ be a class $C^p$ map. Let $x\in A$ and let $v\in E$. Assume that the line segment $x+tv$ with $0\le t\le1$ is contained in $A$. Write $v^{(k)}$ for the $k$-tuple $(v,\dots,v)$. Then $$ f(x+v)=\sum_{k=0}^{p-1}\frac{f^{(k)}(x)v^{(k)}}{k!} + R_p, $$ where $$ R_p = \int_0^1 \frac{(1-t)^{p-1}}{(p-1)!} f^{(p)}(x+tv)v^{(p)}\,dt. $$

Proof. Integration by parts.

Here $f^{(k)}(x)\in L(E,L(E,\dots,L(E,F))\dots)$. (You should already know that the first derivative $f'(x)$ is just a linear map from $E$ to $F$, so we can differentiate the map $f':A\to L(E,F)$ to get $f'':A\to L(E,L(E,F))$ and so on.)

Check out Mathematical Analysis II by Zorich & Cooke or Real and Functional Analysis by Lang.

The terms are no different from those you'd expect based on the univariate case.

The general sum can be stated as

$$f(x) = \sum_{n=0}^\infty \frac{1}{n!}[(x-x_0) \cdot \nabla]^n f|_{x_0}$$

This follows from the single variable case by writing $x-x_0 = t \hat v$ for some unit vector $v$. Each component can then be expressed as a function of $t$ and expanded in a 1D Taylor series.

They are just higher order tensors. Think of them multidimensional matrices. So (assuming the output of $f$ is just a real scalar), the first term in the Taylor expansion is just a scalar. The second term is a vector, all the first partials (AKA the gradient vector). The third term is a square matrix, all of the possible second partials (AKA the Hessian matrix). The fourth term is a 3D matrix, all of the possible third partials, and so on.

In addition to below answers the explicit form for three variables are $$f(x,y,z)=f(x_0,y_0,z_0)$$ $$+\frac{\partial f_0}{\partial x}(x-x_0)+\frac{\partial f_0}{\partial y}(y-y_0)+\frac{\partial f_0}{\partial z}(z-z_0)\quad \Rightarrow Order 1$$ $$+\frac{1}{2} \bigg(\frac{\partial^2 f_0}{\partial x^2}(x-x_0)^2+\frac{\partial^2 f_0}{\partial y^2}(y-y_0)^2+\frac{\partial^2 f_0}{\partial z^2}(z-z_0)^2+2\frac{\partial^2 f_0}{\partial x\partial y}(x-x_0)(y-y_0) $$ $$+2\frac{\partial^2 f_0}{\partial x\partial z}(x-x_0)(z-z_0)+2\frac{\partial^2 f_0}{\partial z\partial y}(z-z_0)(y-y_0)\bigg)\quad \Rightarrow Order 2$$ And it goes like this to higher orders