How to calculate percentage improvement in response time for performance testing

Solution 1:

There are two ways to interpret "percentage improvement in response time". One is the classic and ubiquitous formula for computing a percentage change in a data point from an old value to a new value, which looks like this:

(new - old)/old*100%

So for your case:

(799 - 15306)/15306*100% = -94.78%

That means the new value is 94.78% smaller (faster, since we're talking about response time) than the old value.

The second way of interpreting the statement is to take the percentage of the old value that the new value "covers" or "reaches":

new/old*100%

For your case:

799/15306*100% = 5.22%

That means the new value is just 5.22% of the old value, which, for response time, means it takes just 5.22% of the time to respond, compared to the old response time.

The use of the word "improvement" suggests that you want the 94.78% value, as that shows how much of the lag in the old response time was eliminated ("improved") by the new code. But when it comes to natural language, it can be difficult to be certain about precise meaning without careful clarification.

Solution 2:

I think the accepted answer suffers from the original question not having nice round numbers and that there are 3 different ways to state the result.

Let's assume that the old time was 10 seconds and the new time is 5 seconds.

There's clearly a 50% reduction (or decrease) in the new time:

(old-new)/old x 100% = (10-5)/10 x 100% = 50%

But when you talk about an increase in performance, where a bigger increase is clearly better, you can't use the formula above. Instead, the increase in performance is 100%:

(old-new)/new x 100% = (10-5)/5 x 100% = 100%

The 5 second time is 2x faster than the 10 second time. Said a different way, you can do the task twice (2x) now for every time you used to be able to do it.

old/new = 10/5 = 2.0

So now let's consider the original question

The old time was 15306 ms and the new time is 799 ms.

There is a 94.7% reduction in time.

(old-new)/old x 100% = (15306-799)/15306 x 100% = 94.7%

There is a 1816% increase in performance:

(old-new)/new x 100% = (15306-799)/799 x 100% = 1815.6%

Your new time is 19x faster:

old/new = 15306/799 = 19.16