Testing with JMeter: how to run N requests per second

I need to test if our system can perform N requests per second. Technically, it's 2 requests to one API, 2 requests to another, and 6 requests to third one. But the important thing that they should happen simultaneously - so 10 requests per second. So, in JMeter I've created three Thread Groups, first defines number of threads 1, and ramp-up time 0. Second thread group is the same, and third thread group defines number of threads 6 and ramp-up time 0. But that doesn't really guarantee it's going to run them per second How do I emulate that? And how do I see the results -- if it was able to perform or wasn't?

Thanks!

Solution 1:

You could use ConstantThroughputTimer.

Quote from JMeter help files below:

18.6.4 Constant Throughput Timer This timer introduces variable pauses, calculated to keep the total throughput (in terms of samples per minute) as close as possible to a give figure. Of course the throughput will be lower if the server is not capable of handling it, or if other timers or time-consuming test elements prevent it. N.B. although the Timer is called the Constant Throughput timer, the throughput value does not need to be constant. It can be defined in terms of a variable or function call, and the value can be changed during a test.

For example I've used it to generate 40 requests per second:

 <ConstantThroughputTimer guiclass="TestBeanGUI" testclass="ConstantThroughputTimer" testname="Constant Throughput Timer" enabled="true">
      <stringProp name="calcMode">all active threads in current thread group</stringProp>
      <doubleProp>
        <name>throughput</name>
        <value>2400.0</value>
        <savedValue>0.0</savedValue>
      </doubleProp>
    </ConstantThroughputTimer>

And thats a summary:

Created the tree successfully using performance/search-performance.jmx
Starting the test @ Tue Mar 15 16:28:39 CET 2011 (1300202919244)
Waiting for possible shutdown message on port 4445
Generate Summary Results +  3247 in  80,3s =   40,4/s Avg:    18 Min:     0 Max:  1328 Err:   108 (3,33%)
Generate Summary Results +  7199 in 180,0s =   40,0/s Avg:    15 Min:     1 Max:  2071 Err:   378 (5,25%)
Generate Summary Results = 10446 in 260,3s =   40,1/s Avg:    16 Min:     0 Max:  2071 Err:   486 (4,65%)
Generate Summary Results +  7200 in 180,0s =   40,0/s Avg:    14 Min:     0 Max:   152 Err:   399 (5,54%)
Generate Summary Results = 17646 in 440,4s =   40,1/s Avg:    15 Min:     0 Max:  2071 Err:   885 (5,02%)
Generate Summary Results +  7199 in 180,0s =   40,0/s Avg:    14 Min:     0 Max:  1797 Err:   436 (6,06%)
Generate Summary Results = 24845 in 620,4s =   40,0/s Avg:    15 Min:     0 Max:  2071 Err:  1321 (5,32%)

But I run this test inside my network.

Solution 2:

As with any network test, there's always going to be problems, especially with latency - even if you could send exactly 6 per second, they're going to be sent sequentially (that's just how packets get sent) and may not all hit in that second, plus processing time.

Generally when performance metrics specific x per second, it's measured over a period of time. Your API may even have a buffer - so you could technically send 6 per second, but process 5 per second, with a buffer of 20, meaning it'd be fine for 20 seconds of traffic, as you'd have sent 120, which would take 120/5 = 24 seconds to process. But any more than that would overflow the buffer. So to just send exactly 6 in a second to test is insufficient.

In the thread group, you're right setting number of threads (users) to 6. Then run it looping forever (tick it or put it in a while loop) and add a listener like aggregate report and results tree. The results you can use to check the right stuff is being sent and responded to (assuming you validate the responses) and in the aggregate report, you can see how many of each activity is happening per hour (obviously multiply by 3600 for seconds, but because of this inaccuracy it's best to run it for a good length of time).

The initial load test can now be run, and as a more accurate test, you can leave it running for longer (soak test) to see if any other problems surface - buffer overflows, memory leaks, or other unexpected events.