Squaring a Vector?

$$ {\bf r} \cdot {\bf r} = \left|{\bf r}\right|^2 = \sum_{i,j} r_i r_j {\bf e}_i \cdot {\bf e}_j = \sum_{i,j} r_i r_j \delta_{ij} = \sum_i r_i r_i $$ is often written as ${\bf r}^2$, when it's clear that you don't mean $$ {\bf r}{\bf r} = \sum_{i,j} r_i r_j {\bf e}_i {\bf e}_j $$ which is a perfectly valid quantity [1].

As for ${\bf r} = {\bf x} + {\bf y} + {\bf z}$, $$ \begin{eqnarray} {\bf r} \cdot {\bf r} &=& {\bf x} \cdot {\bf x} + {\bf x} \cdot {\bf y} + {\bf x} \cdot {\bf z} + {\bf y} \cdot {\bf x} + {\bf y} \cdot {\bf y} + {\bf y} \cdot {\bf z} + {\bf z} \cdot {\bf x} + {\bf z} \cdot {\bf y} + {\bf z} \cdot {\bf z} \\ &=& {\bf x} \cdot {\bf x} + {\bf y} \cdot {\bf y} + {\bf z} \cdot {\bf z} + 2\left( {\bf x} \cdot {\bf y} + {\bf x} \cdot {\bf z} + {\bf y} \cdot {\bf z}\right) \end{eqnarray} $$ equals ${\bf x} \cdot {\bf x} + {\bf y} \cdot {\bf y} + {\bf z} \cdot {\bf z}$ for all $\bf x$, $\bf y$, $\bf z$ iff ${\bf x} \cdot {\bf y} + {\bf x} \cdot {\bf z} + {\bf y} \cdot {\bf z} = 0$ -- a special case is when ${\bf x} \cdot {\bf y} = {\bf x} \cdot {\bf z} = {\bf y} \cdot {\bf z} = 0$.

1. For example, both the dot product and regular product are used in the expression for the inertia tensor: $$ \int d^3{\bf r} \ \rho\left({\bf r}\right) \left[\left({\bf r} \cdot {\bf r}\right) {\bf I} - {\bf r}{\bf r}\right] $$ where $$ {\bf I} = \sum_{i,j} \delta_{ij} {\bf e}_i {\bf e}_j $$

Why don't you work out the expansion yourself.

$$ \vec{r}\cdot \vec{r} = \left( \vec{x}+\vec{y}+\vec{z} \right) \cdot \left( \vec{x}+\vec{y}+\vec{z} \right) =\\= (\vec{x} \cdot \vec{x})+(\vec{y}\cdot\vec{y}) + (\vec{z}\cdot\vec{z}) + 2(\vec{x} \cdot\vec{z}) +2(\vec{y}\cdot\vec{z}) + 2(\vec{x} \cdot\vec{y}) = \\ = \|\vec{x}\|^2 + \|\vec{y}\|^2+\|\vec{z}\|^2 + 2 \left((\vec{x} \cdot\vec{z}) +(\vec{y}\cdot\vec{z}) + (\vec{x} \cdot\vec{y}) \right)$$

IF the basis vectors are orthogonal then all the inner products are zero between vectors and you have $$|\vec{r}|^2 = \|\vec{x}\|^2 + \|\vec{y}\|^2+\|\vec{z}\|^2$$

otherwise no.

First if nothing is said you can assume that $\vec{x}\vec{y}$ is a dot product.

What you said ${(\vec{x}+\vec{y}+\vec{z})}^{2}=x^2+y^2+z^2$ is only true if the vectors $\vec{x},\vec{y},\vec{z}$ are perpendicular to each other but in the general case:

$$ {\vec{r}}^{2}={(\vec{x}+\vec{y}+\vec{z})}^{2}\\ =x^2+y^2+z^2+2(xy\cos{\alpha}+yz\cos{\beta}+xz\cos{\gamma}) $$

Where : $$ x=||\vec{x}|| $$

There are two basic ways you can multiply a vector, the dot product, as demonstrated in the link Dot Product, which gives you a scalar, no matter if you are multiplying A.B or squaring it, A.A.

Or you can have the cross product, which is A X B, which gives you another vector, perpendicular to both Cross Product.

The reason there are two different ways to multiply vectors is to deal with different physical situations, so the dot product can be used for work, which is a scalar, F.D, and the cross product is used for torque, say, or angular velocity, which are vectors.

May I suggest a good read on geometric algebra(outer products, r-blades ,...). Check out " An introduction to Geometric Algebra and Calculus " but essentially the answer to your question can be shown using the following definition

$$\vec{a}^2=\vec{a}\vec{a}=\vec{a}\cdot\vec{a}+\vec{a}\bigwedge\vec{a} $$

However

$$\vec{a}\bigwedge\vec{a}=0$$

Hence $$\vec{a}^2=\vec{a}\cdot\vec{a}=||\vec{a}||^2$$