What is the difference between the two sentences? (Have you been ... before? AND have you ever been...?)

The first question (Have you been to Japan before?) is most likely to be asked when you and your conversation partner are actually in Japan or are discussing a plan to visit it.

A similar question is (Have you ridden a horse before?). You are sitting on a horse or discussing a plan to ride one.

The second question (Have you ever been to Japan?) would be rather unlikely to be asked when you are actually in Japan. It is more likely to be an out-of-the-blue question similar to Have you ever ridden a horse? It is also possible when discussing a plan to visit Japan or to ride a horse.