What are the differences between BDD frameworks for Java? [closed]

What are the pros and cons of each Behavior Driven Development (BDD) framework for Java?

I've found some of them here, for example.

Does it make sense to use a BDD framework if I already use a mocking library (e.g. Mockito)?

Solution 1:

I've just finished comparing three BDD frameworks for Java. Obviously my findings have a fairly short use-by date.

Concordion

Very flexible
Very pretty report output
Nice plugin framework
Poorly documented. I had to read the source to figure it out (luckily its extremely good quality).
Fixtures seemed likely to end up tightly coupled to the html.

EasyB

Very shallow learning curve (even for non-Groovy Developers)
Extremely powerful DBUnit integration
Apparently no support for parameters (leads to either very vague stories or duplication between text and code (edit: actually there is but the documentation for it was very well hidden.)
Story and Code are very tightly coupled (same file)
Very basic report output
Couldn't get IntelliJ plugin to work
Inactive community (Maven plugin seems to have been broken for three months - not many code examples to draw on)

JBehave

Extremely powerful and flexible (eg reduction of boiler-plate through composition of stories as pre-requisites)
Extensive (if fragmented) documentation and examples
Extensive (if overwhelming) support for different frameworks and environments
Excellent separation of story files from code
Looks to have a pretty active community and much more examples and discussion of it on web.
Quite a steep learning curve (took me 3-4 times longer to figure out than Concordion/EasyB)

I didn't have the chance to try out Cuke4Duke of JDave as I would have liked, but will probably push for JBehave at this time.

Solution 2:

"pros and cons" might be different things for different people. I usually have a look at

development activity, e.g. are new releases likely or is the last release 2 years old.
maturity, e.g. how long has it been around, are there tutorials and maybe even books available. (I don't read these books, it's just a sign of adoption.)
tool support, e.g. is there an Eclipse plugin, Ant support, etc
size of the dependencies, I don't like frameworks that come with everything of their own. e.g. I want to chose my mocking framework myself.
kind of license, this is important for me because of legal terms in the company I work for.
compatibility with related tools, e.g. does it use Gherkin language or not.

And from some frameworks I had a look at

Instinct bad: last activity Mar 2010, good: ASF license
JDave bad: comes with matchers and mocks, good: last activity Jan 2011, ASF license
easyb bad: last activity Oct 2010, not sure: it uses Groovy. This might be ok, but would be a problem for adoption in my case.
beanspec bad: only one version in 2007, this is dead
bdoc bad: last activity Jan 2010, not sure: it looks like going the other way, creating a report from the code.
spock bad: maybe a bit extreme, this is a complete testing framework, not only BDD, good: very active, very cool.
jbehave, the "mother" of all BDD in Java, bad: very powerful = complex, incompatible license (for me), comes with almost every test library and much more, good: based on RSpec and therefore compatible, eclipse plugins, maven integration, very active community
ginkgo4j, a BDD framework for Java also based on Ruby's RSpec but using Java lambda's (instead of annotations) to allow you to create highly contextual, highly readable tests. Simple. Very powerful. Open source Apache 2 license.

Concerning the mocks: You definitely need a mocking framework as well. The BDD frameworks just help you in writing the specs, but some tests will need mocks or stubs, esp. when you design top down (from overview to detail).

Solution 3:

What's the best BDD framework to use with Java? Why? What are the pros and cons of each framework?

Here is an interesting link about Concordion vs. Cucumber and Java based Acceptance Testing

I've found couple of them here, but I'm not sure which one to choose.

Really, look at the one mentioned above.

Does it make sense to use a BDD framework if I already use a mocking library (e.g. Mockito)?

Short answer: yes, definitely. Actually, acceptance testing using a BDD framework and unit testing in isolation using mock objects are so different that I don't really get the question. Acceptance testing is black box testing, tests are used to verify that a business feature is working and are ideally written by business analyst. Unit tests in isolation using mocks is white box testing, tests are used to verify that a unit is working and are written by developers. Both are useful buty they have totally different purposes. In other words, using Mockito doesn't replace a BDD framework at all and the inverse is also true.

Solution 4:

I originally did my BDD with plain jUnit but I've been looking at JDave lately because it's almost 1:1 to what I was doing with jUnit. It also runs on top of jUnit so it already works on Eclipse and is also easy to configure to work on continuous integration systems such as Hudson. Can't really compare it with others but my experiences with JDave have been good so far.

Oh and it's never a stupid idea to use mocks! They're not tied to TDD/BDD specifically, their purpose is to ease the burden of testing in general.

Solution 5:

Wow, I see the topic is hot, lot of good answers...

Irony aside, I recently discovered BDD and found the concept interesting. Hey, it forces to write both tests... and specifications! As surprising as it might seem, the latter can be also missing in some projects... Or just lacking the precision that BDD forces to introduce.

The Behavior Driven Development article summarizes the concept and links to some good articles (like the one written by Andrew Glover). Moreover, to the topic of this thread, it gives a rather comprehensive (I suppose) listing of BDD frameworks, a good number of them being for Java.
It doesn't solve the problem of choosing the framework but at least it will ease the search...

Since BDD relies heavily on readability of test code, I suppose a good criterion of choice is to look at the quick tours/tutorial and see which one seems the more fitting your style. Other criteria could be the fact a framework leverage tools you are familiar with (unit test, mocking), usage with IDE, and so on.