What is a coset?
Honestly. I just don't understand the word and keep running into highly technical explanations (wikipedia, I'm looking at you!). If somebody would be so awesome as to explain the concept assuming basic knowledge of group theory and high school algebra I would be delighted.
Let's start with a concrete example. Let our group be $\mathbb{Z}$, the integers with addition. There is a subgroup here of even integers, written $2\mathbb{Z}$. This is a subgroup because the sum of even integers is an even integer, and because the number $0$ is even. Since subgroups need to have identity elements, it is important that $0$ is in $2\mathbb{Z}$. Now, the integers also contain $2\mathbb{Z} + 1$, the collection of odd integers. This isn't quite a subgroup, because it doesn't contain an identity element. However, it looks like a shifted copy of the subgroup $2\mathbb{Z}$, which makes sense because it is a shifted copy of $2\mathbb{Z}$ -- we are shifting by adding $1$. What's more, every integer is either even or odd (but not both!), so the (disjoint) union of $2\mathbb{Z}$ with its shifted copy $2\mathbb{Z}+1$ is all of $\mathbb{Z}$.
One thing to observe is that the shift $2\mathbb{Z} + 1$ and $2\mathbb{Z}+3$ both give you the odd integers, so it is wrong to say that every element shifts a subgroup in a unique way.
If we took the subgroup $3\mathbb{Z}$ of multiplies of $3$, we would get two shifts: $3\mathbb{Z} + 1$ and $3\mathbb{Z}+2$. These are all disjoint from one another, they all look very similar, and their union is the entirety of $\mathbb{Z}$.
One more example before we move to generalities. Let $G$ be the group of symmetries of the square, which is a rather common example in group theory. We have a subgroup here of rotational symmetries $H$. $H$ has four elements: rotation by $0$ degrees, rotation by $90$ degrees, rotation by $180$ degrees, and rotation by $270$ degrees. Rotation by $0$ degrees is the identity element. $G$ contains an element $f$ which represents flipping the square upside down. $f$ is not an element of $H$. Suppose you "shift" $H$ by taking every symmetry in $H$ and following it by flipping the square upside down. We would write this $fH$, and this is not a group, since it does not contain the identity element. However, it looks like a shifted subgroup $H$, it has the same number of elements of $H$, it has no overlap with $H$, and $G$ is the union of $H$ and $fH$, because every symmetry is either a rotation or a rotation followed by a flip.
Speaking generally, let $G$ be a group. Given a subgroup $H$, you can shift $H$ by multiplying it by an element $g$ in $G$. If $g$ is also an element of $H$, the shift $gH$ is the same as $H$. But if $g$ is not in $H$, $gH \neq H$, and $gH$ is not a subgroup. The shift $gH$ is called a coset of $H$, and it looks a lot like a copy of $H$. Actually, $H$ is also a coset of $H$, since you can shift by the identity element. Just like every integer was even or odd, if you take all the cosets of $H$ they fill up the entirety of $G$, so you can break up any group into a collection of pieces that look like "shifted copies" of a subgroup.
Think of $G = \mathbb{R}^2$, and $H = \{(x,x) : x\in \mathbb{R}\}$ be the line $y=x$. Then any coset of $H$ is another line parallel to the line $y=x$. In group theory notation, one would write it as $$ (a,b)+H = \{(a,b) + (x,y) : (x,y)\in H\} = \{(x+a,x+b) : x\in \mathbb{R}\} $$
Let $G = \mathbb{C}^{\times}$, and $H = \{z\in \mathbb{C}^{\times} : |z| = 1\}$ be the unit circle. Then any coset of $H$ is another circle centred at the origin. Again this would be $$ w_0H = \{w_0z : z\in H\} = \{w_0z : |z| = 1 \} = \{w : |w| = |w_0|\} $$
Suppose $A$ is an $n\times n$ matrix, and $G = \mathbb{R}^n$. Let $$ H = \{x \in G : Ax = 0\} $$ For any $b\in \mathbb{R}^n$, the solution set $$ \{y \in \mathbb{R}^n : Ay = b \} $$ is a coset of $H$ (if it is non-empty)
A coset is what you get by taking a subgroup and shifting it by some element of the group.
For example, the line $y = 4x$ is a subgroup of the additive group of the points on the plane. If I shft the line upwards by adding $0\vec i + 1 \vec j$ to every point on the line, I get the line $y = 4x+1$. That is not a subgroup any longer, because it does not contain the identity $0\vec i + 0 \vec j$. But it has the same "shape" as the original subgroup, just shifted away from the origin.
For another example, in the group $\mathbb{Z}_{12}$, with modular addition, let $H$ be the subgroup $\{0,3,6,9\}$. If I shift it by adding $1$ to each element, I get a coset $1 + H = \{1,4,7,10\}$. If I shift it by adding $2$ to each element of $H$ I get the coset $2 + H = \{2,5,8,11\}$. These are not subgroups, because they do not contain $0$, but they have the same "shape" as the original subgroup $H$, in some intuitive sense.
Most of the proofs about cosets use the fact that the original subgroup actually was a subgroup. The coset is usually not a subgroup, but the fact that it came from a subgroup means that it has a similar type of internal structure that you can use to prove things about it.