$G \times H \cong G \times K$ , then $ K \cong H$

1) This is a nontrivial theorem of Laszlo Lovasz. Google his name along with direct product and you will find it.

2) Let $$G = \mathbb{Z} / p \mathbb{Z} \times \mathbb{Z} / p \mathbb{Z} \times \ldots$$ $$H = \mathbb{Z} / p \mathbb{Z} $$ $$K = \mathbb{Z} / p \mathbb{Z} \times \mathbb{Z} / p \mathbb{Z}.$$

You can see Hirshon's article On cancellation in groups, where the author proves that finite groups may be cancelled in direct products. Also I mentionned a simple proof of Vipul Naik here, who proved that finite groups may be cancelled in direct products of finite groups.

For your counterexample, you can take $$G \times (G \times G \times \cdots) \simeq \{1\} \times (G \times G \times \cdots)$$ where $G$ is any nontrivial group. Other examples can be found here; in particular, a finitely presented counterexample is given there.

There are a couple of questions from about a year ago in a similar vein to this one on math.SE. So I thought it would be useful to post this answer linking to two of them and relating them to your question.

The first question to be posed:

Does $G\times H\cong G\times K\Rightarrow H\cong K$?

Basic answer: No, take $G\times G\times G\times \cdots$, $H=G$ and $K=1$ (which is basically the same as the answers given here).

This led to a spin-off question, which is basically your question (1):

Let $G\times H\cong G\times K$ be finite isomorphic groups. Then are $H$ and $K$ isomorphic?

Answer: Yes. All the relevant answers cite a paper of Hirshon, with Serios' giving an elementary proof due to Naik.

Now, to answer your second question: Where does this fail? When the above two questions were asked this bugged me for a while. All the answers either said "finite groups work" of "some infinite groups don't work". The problem is that the infinite groups given were rubbish! They are not finitely generated! In the theory of infinite groups the challenge is often to find finitely generated, or, even better, finitely presented groups with pathological properties. See, for example, here, here and here. Now, it turns out that the property we are discussing fails in the simplest possible infinite case. Concretely:

If $G\times H\cong G\times K$ is finitely presented and $H\cong\mathbb{Z}$ is infinite cyclic then $K$ need not be infinite cyclic.

I review the proof of this (it is dead easy!) in an answer to the original question, here. The result is again due to Hirshon, and is given in the same paper as he proves the result about finite groups. Basically, Hishon proved the results for finite groups and asked "where does this break", and then told us. So, for a complete answer to your question, read Hirshon's paper!