Set of Linear equation has no solution or unique solution or infinite solution?

Probably the most straightforward method (to fully distinguish between the various possibilities) that I've seen is transforming the corresponding augmented matrix into row-reduced echelon form. In this case, you would start with: $$\left[\begin{array}{ccc|c}1 & 3 & -1 & -4\\4 & -1 & 2 & 3\\2 & -1 & -3 & 1\end{array}\right]$$ Subtracting $4$ times the first row from the second, and $2$ times the first row from the third, we have: $$\left[\begin{array}{ccc|c}1 & 3 & -1 & -4\\0 & -13 & 6 & 19\\0 & -7 & -1 & 9\end{array}\right]$$ Subtracting $2$ times the third row from the second, we have: $$\left[\begin{array}{ccc|c}1 & 3 & -1 & -4\\0 & 1 & 8 & 1\\0 & -7 & -1 & 9\end{array}\right]$$ Adding $7$ times the second row to the third, we have: $$\left[\begin{array}{ccc|c}1 & 3 & -1 & -4\\0 & 1 & 12 & 1\\0 & 0 & 55 & 16\end{array}\right]$$

At this point, we have only zeroes below the main diagonal, but no zeroes on the diagonal, so a unique solution exists. Continuing to reduce until the $3\times 3$ portion of the augmented matrix is just the $3\times 3$ identity matrix, we have $$\left[\begin{array}{ccc|c}1 & 0 & 0 & 3/11\\0 & 1 & 0 & -73/55\\0 & 0 & 1 & 16/55\end{array}\right]$$ This tells us that $x=3/11,$ $y=-73/55,$ $z=16/55$ is the unique solution to the system.

Let's consider another system: $$\begin{cases}x+3y-z=4\\4x-y+2z=8\\2x-7y+4z=-3,\end{cases}$$ which has corresponding matrix $$\left[\begin{array}{ccc|c}1 & 3 & -1 & 4\\4 & -1 & 2 & 8\\2 & -7 & 4 & -3\end{array}\right].$$ Starting out the same way gets us $$\left[\begin{array}{ccc|c}1 & 3 & -1 & 4\\0 & -13 & 6 & -8\\0 & -13 & 6 & -11\end{array}\right],$$ and subtracting the second row from the third gives us $$\left[\begin{array}{ccc|c}1 & 3 & -1 & 4\\0 & -13 & 6 & -8\\0 & 0 & 0 & -3\end{array}\right].$$ Now we have only zeroes below the main diagonal, but we have a zero on the main diagonal, too. This tells us that either there are no solutions or there are infinitely-many. Translated back into terms of $x,y,z$ this is the equivalent system $$\begin{cases}x+3y+-z=4\\0x-13y+6z=-8\\0x+0y+0z=-3,\end{cases}$$ or alternatively $$\begin{cases}x=-\frac5{13}z+\frac{28}{13}\\y=\frac6{13}z+\frac8{13}\\0=3,\end{cases}$$ but there is no solution to the last equation, so no solution to the system.

Upshot: We will have no solutions whenever we end up with one or more rows of all $0$s except in the last column as we reduce the augmented matrix.

By contrast, if we'd started with the system $$\begin{cases}x+3y-z=4\\4x-y+2z=8\\2x-7y+4z=0,\end{cases}$$ which has corresponding matrix $$\left[\begin{array}{ccc|c}1 & 3 & -1 & 4\\4 & -1 & 2 & 8\\2 & -7 & 4 & 0\end{array}\right],$$ then our reduction process will get us $$\left[\begin{array}{ccc|c}1 & 3 & -1 & 4\\0 & -13 & 6 & -8\\0 & 0 & 0 & 0\end{array}\right].$$ Again, we must have no solution or infinitely many. Continuing to row-reduce as much as possible gets us to $$\left[\begin{array}{ccc|c}1 & 0 & 5/13 & 28/13\\0 & 1 & -6/13 & 8/13\\0 & 0 & 0 & 0\end{array}\right].$$ Translated back into terms of $x,y,z$ this is the equivalent system $$\begin{cases}x+0y+\frac5{13}z=\frac{28}{13}\\0x+y-\frac6{13}z=\frac8{13}\\0=0.\end{cases}$$ One of these equations is always true, so one of our variables can take on any value. We might as well let $z$ take on any value, at which point the other two equations will tell us the values that $x$ and $y$ must take. Hence, we have infinitely-many solutions.

Upshot: We will have infinitely-many solutions whenever we end up with one or more rows of all $0$s as we reduce the augmented matrix, so long as we don't have any rows with all $0$s except in the last column.

Added: Simply taking the determinant of the unaugmented matrix of the system--meaning of $$\begin{bmatrix}1 & 3 & -1\\4 & -1 & 2\\2 & -1 & -3\end{bmatrix}$$ in the first example and of $$\begin{bmatrix}1 & 3 & -1\\4 & -1 & 2\\2 & -7 & 4\end{bmatrix}$$ in the other two examples--will give us part of the answer. If the determinant is $0$ (as in the second and third example), then the system either has no solution or infinitely-many, but we cannot (by this method alone) say which. Otherwise, the system has a unique solution, but we cannot (by this method alone) say what it might be. That's why I tend to prefer the first method I suggested, at least when dealing with only a few equations and a few variables: it tells us the whole story.

The important concept here is linear dependence versus linear independence. As shown in the examples posted by others, linear dependence occurs when one equation in the system of equations can be shown to be a multiple of another. This is ultimately what Gaussian elimination or computing the determinant reveals. In this instance, there is no unique solution to the system of equations.

Conversely, if the system of equations is linearly independent, then a unique solution does exist (though you still have to compute it, as is done in the examples in other answers).

This can be visualized by graphing the equations, assuming a low order for the system. Linear dependence implies that two or more of the lines obtained when graphing the system are parallel or one on top of the other. Linear independence will result in a graph in which the various lines intersect at one point, that point being the solution to the system of equations.