Rotating a Vector in 3D Space

If you want to rotate a vector you should construct what is known as a rotation matrix.

Rotation in 2D

Say you want to rotate a vector or a point by θ, then trigonometry states that the new coordinates are

    x' = x cos θ − y sin θ
    y' = x sin θ + y cos θ

To demo this, let's take the cardinal axes X and Y; when we rotate the X-axis 90° counter-clockwise, we should end up with the X-axis transformed into Y-axis. Consider

    Unit vector along X axis = <1, 0>
    x' = 1 cos 90 − 0 sin 90 = 0
    y' = 1 sin 90 + 0 cos 90 = 1
    New coordinates of the vector, <x', y'> = <0, 1>  ⟹  Y-axis

When you understand this, creating a matrix to do this becomes simple. A matrix is just a mathematical tool to perform this in a comfortable, generalized manner so that various transformations like rotation, scale and translation (moving) can be combined and performed in a single step, using one common method. From linear algebra, to rotate a point or vector in 2D, the matrix to be built is

    |cos θ   −sin θ| |x| = |x cos θ − y sin θ| = |x'|
    |sin θ    cos θ| |y|   |x sin θ + y cos θ|   |y'|

Rotation in 3D

That works in 2D, while in 3D we need to take in to account the third axis. Rotating a vector around the origin (a point) in 2D simply means rotating it around the Z-axis (a line) in 3D; since we're rotating around Z-axis, its coordinate should be kept constant i.e. 0° (rotation happens on the XY plane in 3D). In 3D rotating around the Z-axis would be

    |cos θ   −sin θ   0| |x|   |x cos θ − y sin θ|   |x'|
    |sin θ    cos θ   0| |y| = |x sin θ + y cos θ| = |y'|
    |  0       0      1| |z|   |        z        |   |z'|

around the Y-axis would be

    | cos θ    0   sin θ| |x|   | x cos θ + z sin θ|   |x'|
    |   0      1       0| |y| = |         y        | = |y'|
    |−sin θ    0   cos θ| |z|   |−x sin θ + z cos θ|   |z'|

around the X-axis would be

    |1     0           0| |x|   |        x        |   |x'|
    |0   cos θ    −sin θ| |y| = |y cos θ − z sin θ| = |y'|
    |0   sin θ     cos θ| |z|   |y sin θ + z cos θ|   |z'|

Note 1: axis around which rotation is done has no sine or cosine elements in the matrix.

Note 2: This method of performing rotations follows the Euler angle rotation system, which is simple to teach and easy to grasp. This works perfectly fine for 2D and for simple 3D cases; but when rotation needs to be performed around all three axes at the same time then Euler angles may not be sufficient due to an inherent deficiency in this system which manifests itself as Gimbal lock. People resort to Quaternions in such situations, which is more advanced than this but doesn't suffer from Gimbal locks when used correctly.

I hope this clarifies basic rotation.

Rotation not Revolution

The aforementioned matrices rotate an object at a distance r = √(x² + y²) from the origin along a circle of radius r; lookup polar coordinates to know why. This rotation will be with respect to the world space origin a.k.a revolution. Usually we need to rotate an object around its own frame/pivot and not around the world's i.e. local origin. This can also be seen as a special case where r = 0. Since not all objects are at the world origin, simply rotating using these matrices will not give the desired result of rotating around the object's own frame. You'd first translate (move) the object to world origin (so that the object's origin would align with the world's, thereby making r = 0), perform the rotation with one (or more) of these matrices and then translate it back again to its previous location. The order in which the transforms are applied matters. Combining multiple transforms together is called concatenation or composition.

Composition

I urge you to read about linear and affine transformations and their composition to perform multiple transformations in one shot, before playing with transformations in code. Without understanding the basic maths behind it, debugging transformations would be a nightmare. I found this lecture video to be a very good resource. Another resource is this tutorial on transformations that aims to be intuitive and illustrates the ideas with animation (caveat: authored by me!).

Rotation around Arbitrary Vector

A product of the aforementioned matrices should be enough if you only need rotations around cardinal axes (X, Y or Z) like in the question posted. However, in many situations you might want to rotate around an arbitrary axis/vector. The Rodrigues' formula (a.k.a. axis-angle formula) is a commonly prescribed solution to this problem. However, resort to it only if you’re stuck with just vectors and matrices. If you're using Quaternions, just build a quaternion with the required vector and angle. Quaternions are a superior alternative for storing and manipulating 3D rotations; it's compact and fast e.g. concatenating two rotations in axis-angle representation is fairly expensive, moderate with matrices but cheap in quaternions. Usually all rotation manipulations are done with quaternions and as the last step converted to matrices when uploading to the rendering pipeline. See Understanding Quaternions for a decent primer on quaternions.