Find the value of k, (if any), for which the system below has unique, infinite or no solution. [duplicate]

As I mentioned in a comment, there is at least one another post about the same problem: System of Linear Equations - how many solutions?

Let me address your attempt. (I think that as soon as you are satisfied with the feedback you got on your attempt to solve the problem, this post might be closed as a duplicate of some other post asking the same question.)

$\left[\begin{array}{ccc|c}1&1&k&1\\k&1&1&1\\0&k-1&1-k&0\end{array}\right]$

This is where I got stuck, and I'm not sure how to continue.

Now a reasonable step would be to divide the last row by $(k-1)$. But this is only possible if $k-1\ne0$.

So you can:

• Solve the case $k=1$ separately.
• For $k\ne 1$ divide the last row by $(k-1)$ which leads you to a simpler matrix - you should try whether you are able to continue from there.

$$\left[\begin{array}{ccc|c}1&1&k&1\\k&1&1&1\\0&1&-1&0\end{array}\right]$$

I would restart from: $$\left[\begin{array}{ccc|c}1&1&k&1\\0&k-1&1-k&0\\k&1&1&1\end{array}\right]$$ then $\,R_3\leftarrow R_3-kR_1$: $$\left[\begin{array}{ccc|c}1&1&k&1\\0&k-1&1-k&0\\0&1-k&1-k^2&1-k\end{array}\right]$$ $R_3\leftarrow R_3+R_2$: $$\left[\begin{array}{ccc|c}1&1&k&1\\0&k-1&1-k&0\\0&0&(1-k)(2+k)&1-k\end{array}\right]\tag{1}$$ Suppose for now $\boldsymbol {k\neq 1,-2}.$ $R_2\leftarrow \frac{1}{k-1}R_2$: $$\left[\begin{array}{ccc|c}1&1&k&1\\0&1&-1&0\\0&0&(1-k)(2+k)&1-k\end{array}\right]$$ then $\,R_2\leftarrow R_2+\frac1{(1-k)(2+k)}R_3$ and $\,R_1\leftarrow R_1-\frac k{(1-k)(2+k)}R_3$: $$\left[\begin{array}{ccc|c}1&1&0&\dfrac2{2+k}\\0&1&0&\dfrac1{2+k}\\0&0&(1-k)(2+k)&1-k\end{array}\right]$$ and finally $\,R_1\leftarrow R_1-R_2$, $\,R_3\leftarrow \frac1{(1-k)(2+k)}R_3$

$$\left[\begin{array}{ccc|c}1&0&0&\dfrac1{2+k}\\0&1&0&\dfrac1{2+k}\\0&0&1&\dfrac1{2+k}\end{array}\right]$$ The solution is $\enspace x=y=z=\dfrac1{2+k}$.

If $\boldsymbol{k=1}$: the matrix (1) becomes $$\left[\begin{array}{ccc|c}1&1&1&1\\0&0&0&0\\0&0&0&0\end{array}\right]$$ so the solutions consists in the plane $\,x+y+z=1$.

If $\boldsymbol{k=-2}$: the matrix becomes: $$\left[\begin{array}{ccc|c}1&1&-2&1\\0&1&-1&0\\0&0&0&-1\end{array}\right]$$ and the last row shows there's no solution.