Manifold of Density Matrices

Two-by-two semidefinite matrices A (without the trace condition) form a cone with a singularity at the origin. These can be embedded into three-by-three matrices by means of direct sum with the one-by-one matrix [1-tr(A)]. This suggests that your space will have singularities; this is turned into a proof below.

More specifically, the diagonal matrices in your space $M$ form a triangle $DEF$ in $\mathbb R^3$ in the first octant cut out by the relation $x+y+z=1$. The three vertices $D,E,F$ are in a single orbit when one acts by conjugation. This suggests that there might be non-smooth on the boundary of the space, as shown below.

The set of semidefinite matrices $A$ is convex because if $x^* A x\ge0$ and $x^* B x\ge0$ then also $x^* (A+B) x\ge 0$. Thus your space is a convex set and therefore a ball topologically. Using radial scaling you can identify it with a metric ball centered at an interior point, and pull back the smooth structure of the metric ball; in this sense, the space does admit "the structure of a smooth manifold".

However, the boundary of the convex set itself is not smooth. This can be seen as follows. The vertices and the edges of triangle $DEF$ above lie on the boundary of the convex set, since their determinant is zero. Meanwhile, each interior point of the triangle is also an interior point of the convex set, since any perturbation of it is still a positive definite matrix. If the boundary of the convex set were smooth at the vertex $D$, then the tangent plane at $D$ would in particular contain the directions $DE$ and $DF$, and would therefore also contain the entire triangle $DEF$. But we already know that the interior points of the triangle are not on the boundary. Hence the boundary of the convex set cannot be smooth at $D$.