Should the JUnit message state the condition of success or failure?

Solution 1:

I rarely even bother with a message, at least for assertEquals. Any sensible test runner will explain that you were using assertEquals and the two things which were meant to be equal. Neither of your messages give more information than that.

I usually find that unit test failures are transient things - I'll rapidly find out what's wrong and fix it. The "finding out what's wrong" usually involves enough detail that a single message isn't going to make much difference. Consider "time saved by having a message" vs "time spent thinking of messages" :)

EDIT: Okay, one case where I might use a message: when there's a compact description in text which isn't obvious from the string representation of the object.

For example: "Expected date to be December 1st" when comparing dates stored as milliseconds.

I wouldn't worry about how you express it exactly though: just make sure it's obvious from the message which way you mean. Either "should be" or "wasn't" is fine - just "December 1st" wouldn't be obvious.

Solution 2:

According to the junit API the message is the "the identifying message for the AssertionError" so its not a message describing the condition that should be met but a message describing what's wrong if the condition isn't met. So in your example "objects aren't identical" seems to be more conformant.

Solution 3:

Unlike many others I feel that using a message is extremely helpful for many reasons:

  1. The person looking at the logs of a failed test may not be the person who wrote the test. It can take time to read through the code and understand what case the assertion is meant to address. A helpful message will save time.

  2. Even in the event it is the developer of the test who is looking at the logs it may have been days or months since the test was written and, again, a message can save time.

My advice would be to write the message with a statement of the expected behavior. For example:

assertEquals("The method should be invoked 3 times", 3, invocationCount);

Solution 4:

I don't think it matters at all - You already know that a failure happened, and therefore it doesn't matter if the message states what should have happened, or what shouldn't happen.

The goal of the message is to help you when it can, not to obtain some completeness.

Obviously, in the case of assertEquals this is less important, but the message is important in the case of general asserts. The message should help you obtain enough context to understand right away what exactly failed.

However, the amount of needed context (and thus the details in the message) should depend on how you get the report. For example, if you get it in Eclipse, you can easily go and interact and see what happened, so the message is less imporrtant. However, if you get your reports emailed to you (e.g., from a continuous build server) then you want the message to provide enough information so that you will have an idea of what is going on before you even go to the corresponding source code.

Solution 5:

I would like to answer the question without considering, if a message in generel is useful.

If a test fails, something is wrong. I know this. I want to know why it is broken. That's very easy to find out because I just have to open the test case and the SUT. Like Jon said, it's very easy to fix it (hopefully ;-) ).

But what about the message? The message is for me an advice, what could be done to turn it into a green test case. So I would appreciate if there's an advice given in the message text, how to fix this problem or where to search for the problem.

Another interesting aspect would be the usage of positive expressions. It's worth a consideration to use positive text messages. In your example, I would use Objects should be identical. But that's a small reason.