Speeding up RSpec tests in a large Rails application
10 seconds is a very long time for any single test to run. My gut feeling is that your spec target is running both unit and integration tests at the same time. This is a typical thing that projects fall into and at some stage, you will need to overcome this technical debt if you want to produce more, faster. There are a number of strategies which can help you to do this... and I'll recommend a few that I have used in the past.
1. Separate Unit From Integration Tests
The first thing I would do is to separate unit from integration tests. You can do this either by:
- Moving them (into separate folders under the spec directory) - and modifying the rake targets
- Tagging them (rspec allows you to tag your tests)
The philosophy goes, that you want your regular builds to be quick - otherwise people won't be too happy to run them often. So get back to that territory. Get your regular tests to run quick, and use a continuous integration server to run the more complete build.
An integration test is a test that involves external dependencies (e.g. Database, WebService, Queue, and some would argue FileSystem). A unit test just tests the specific item of code that you want checked. It should run fast (9000 in 45 secs is possible), i.e. most of it should run in memory.
2. Convert Integration Tests To Unit Tests
If the bulk of your unit tests is smaller than your integration test suite, you have a problem. What this means is that inconsistencies will begin to appear more easily. So from here, start creating more unit tests to replace integration tests. Things you can do to help in this process are:
- Use a mocking framework instead of the real resource. Rspec has an inbuilt mocking framework.
- Run rcov on your unit test suite. Use that to gauge how thorough your unit test suite is.
Once you have a proper unit test(s) to replace an integration test - remove the integration test. Duplicate testing only makes maintenance worse.
3. Don't Use Fixtures
Fixtures are evil. Use a factory instead (machinist or factorybot). These systems can build more adaptable graphs of data, and more importantly, they can build in-memory objects which you can use, rather than load things from an external data source.
4. Add Checks To Stop Unit Tests Becoming Integration Tests
Now that you have faster testing in place, time to put in checks to STOP this from occurring again.
There are libraries which monkey patch active record to throw an error when trying to access the database (UnitRecord).
You could also try pairing and TDD which can help force your team to write faster tests because:
- Somebody's checking - so nobody gets lazy
- Proper TDD requires fast feedback. Slow tests just make the whole thing painful.
5. Use Other Libraries To Overcome The Problem
Somebody mentioned spork (speeds up load times for the test suite under rails3), hydra/parallel_tests - to run unit tests in parallel (across multiple cores).
This should probably be used LAST. Your real problem is all the way in step 1, 2, 3. Solve that and you will be in a better position to role out additional infrastructure.
For a great cookbook on improving the performance of your test suite, check out the Grease Your Suite presentation.
He documents a 45x speedup in test suite run time by utilizing techniques such as:
- fast_context
- quickerclip
- hydra
- scary file system tweaks
- tcmalloc
- Ruby Enterprise Edition GC tuning
You can use Spork. It has support for 2.3.x ,
https://github.com/sporkrb/spork
or ./script/spec_server which may work for 2.x
Also you can edit the database configuration ( which essentially speeds up the database queries etc ), which will also increase performance for tests.
10 seconds per example seems like a very long time. I've never seen a spec that took more than one second, and most take far less. Are you testing network connections? Database writes? Filesystem writes?
Use mocks and stubs as much as possible - they are much faster than writing code that hits the database. Unfortunately mocking and stubbing also take more time to write (and are harder to do correctly). You have to balance the time spent writing tests vs. the time spent running tests.
I second Andrew Grimm's comment about looking into a CI system which might allow you to parallelize your test suite. For something that size, it might be the only viable solution.
If you are using ActiveRecord models, you should also consider the cost of BCrypt encryption.
You can read more about it on this blog post: http://blog.syncopelabs.co.uk/2012/12/speed-up-rspec-test.html