Unit testing c++. How to test private members?
I would like to make unit tests for my C++ application.
What is the correct form to test private members of a class? Make a friend class which will test the private members, use a derived class, or some other trick?
Which technique does the testing APIs use?
Solution 1:
Typically, one only tests the public interface as discussed in the question's comments.
There are times however when it is helpful to test private or protected methods. For example, the implementation may have some non-trivial complexities that are hidden from users and that can be tested more precisely with access to non-public members. Often it's better to figure out a way to remove that complexity or figure out how to expose the relevant portions publicly, but not always.
One way to allow unit tests access to non-public members is via the friend construct.
Solution 2:
Answering this question touches many other topics. Beside any religiosity in CleanCode, TDD and others:
There are several ways to access private members. In any case you have to overrule the tested code! This is possible on both levels of parsing C++ (preprocessor and language itself):
Define all to public
By using the preprocessor you are able to break encapsulation.
#define private public
#define protected public
#define class struct
The disadvantage is, that the class of the delivered code is not the same as in the test! The C++ Standard in chapter 9.2.13 says:
The order of allocation of non-static data members with different access control is unspecified.
This means, that the compiler has the right to reorder the member variables and virtual functions for the test. You may struggle, that this won't harm your classes if no buffer overflow happens, but it means, that you won't test the same code as you deliver. It means, that if you access members of an object, that was initialized by code, compiled with private not defined to public, the offset of your member may differ!
Friends
This method needs to change the tested class for befriending it with the test class or the test function. Some testing frameworks like gtest (FRIEND_TEST(..);) have special functionality to support this way of accessing private things.
class X
{
private:
friend class Test_X;
};
It opens the class only for the test and does not open up the world, but you have to modify the code that gets delivered. In my opinion this is a bad thing, because a test should never change the tested code. As a further disadvantage it gives other classes of the delivered code the possibility to intrude your class by naming themselves like a test class (this would also harm the ODR rule of the C++ Standard).
Declaring the private things protected and derive from the class for tests
Not a very elegant way, very intrusive, but works also:
class X
{
protected:
int myPrivate;
};
class Test_X: public X
{
// Now you can access the myPrivate member.
};
Any other way with macros
Works, but has the same disadvantages on standard conformity like the first way. e.g.:
class X
{
#ifndef UNITTEST
private:
#endif
};
I think that the last both ways are no alternatives to the first two ways, because they have no advantages over the first ones, but are more intrusive on the tested code. The first way is very risky, so you may use the befriending approach.
Some words on the never-test-private-things-discussion. One of the upsides of unit testing at all is, that you will reach very early the point, where you have to improve the design of your code. This is also sometimes one of the downsides of unit testing. It makes object orientation sometimes more complicated, than it has to be. Especially if you follow the rule to design classes in the same way the real world objects are.
Then you have to change the code sometimes into something ugly, because the unit testing approach forces you to do so. Working on complex frameworks, that are used to control physical processes, is one example. There you want to map the code on the physical process, because often parts of the process are already very complex. The dependency list on that processes gets sometimes very long. This is one possible moment, where testing private members is getting nice. You have to trade-off with the advantages and disadvantages of each approach.
Classes are getting sometimes complex! Then you have to decide to split them or to take them as they are. Sometimes the second decision makes more sense. In the end it is always a question of which goals you want to achieve (e.g. perfect design, quick incorporation times, low development costs...).
My Opinion
My decision process for accessing private members looks like this:
- Do you need to test private members themselves? (Often this reduces the total number of tests needed)
- If yes, do you see any design advantage to refactor the class?
- If no, befriend the test in your class (use this because of the missing alternatives).
I don't like the befriending approach, because it changes the tested code, but the risk to test something, that may not be the same as delivered (as possible with the first approach), will not justify the cleaner code.
BTW: Testing only the public interface is also a fluent matter, because in my experience it changes as often as the private implementation does. So you have no advantage to reduce the test on public members.
Solution 3:
I haven't found a golden solution myself, but you can use friend to test private members, if you know how the testing framework names it's methods. I use the following to test private members with Google test. While this works quite well, note that it's a hack, and I don't use it in production code.
In the header of the code I want to test (stylesheet.h), I have:
#ifndef TEST_FRIENDS
#define TEST_FRIENDS
#endif
class Stylesheet {
TEST_FRIENDS;
public:
// ...
private:
// ...
};
and in the test I have:
#include <gtest/gtest.h>
#define TEST_FRIENDS \
friend class StylesheetTest_ParseSingleClause_Test; \
friend class StylesheetTest_ParseMultipleClauses_Test;
#include "stylesheet.h"
TEST(StylesheetTest, ParseSingleClause) {
// can use private members of class Stylesheet here.
}
You always add a new line to TEST_FRIENDS if you add a new test that accesses private members. The benefits of this technique are that it is fairly unobstrusive in the tested code, as you only add a few #defines, which have no effect when not testing. The downside is that it is a bit verbose in the tests.
Now one word as to why you would want to do this. Ideally of course, you have small classes with well-defined responsibilities, and the classes have easily testable interfaces. However, in practice that's not always easy. If you are writing a library, what is private and public is dictated by what you want the consumer of the library to be able to use (your public API), and not by what's in need of testing or not. You can have invariants that are very unlikely to change, and need to be tested, but are of no interest to the consumer of your API. Then, black-box-testing of the API is not enough. Also if you encounter bugs and write additional tests to prevent regressions, it can be neccessary to test private stuff.
Solution 4:
Sometimes, it is required to test private methods. Testing can be done by adding FRIEND_TEST to the class.
// Production code
// prod.h
#include "gtest/gtest_prod.h"
...
class ProdCode
{
private:
FRIEND_TEST(ProdTest, IsFooReturnZero);
int Foo(void* x);
};
//Test.cpp
// TestCode
...
TEST(ProdTest, IsFooReturnZero)
{
ProdCode ProdObj;
EXPECT_EQ(0, ProdObj.Foo(NULL)); //Testing private member function Foo()
}
Adding more info, since many are not aware of gtest features.
This is from gtest/gtest_prod.h
https://github.com/google/googletest/blob/master/googletest/include/gtest/gtest_prod.h
// Copyright 2006, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Google C++ Testing and Mocking Framework definitions useful in production code.
// GOOGLETEST_CM0003 DO NOT DELETE
#ifndef GTEST_INCLUDE_GTEST_GTEST_PROD_H_
#define GTEST_INCLUDE_GTEST_GTEST_PROD_H_
// When you need to test the private or protected members of a class,
// use the FRIEND_TEST macro to declare your tests as friends of the
// class. For example:
//
// class MyClass {
// private:
// void PrivateMethod();
// FRIEND_TEST(MyClassTest, PrivateMethodWorks);
// };
//
// class MyClassTest : public testing::Test {
// // ...
// };
//
// TEST_F(MyClassTest, PrivateMethodWorks) {
// // Can call MyClass::PrivateMethod() here.
// }
//
// Note: The test class must be in the same namespace as the class being tested.
// For example, putting MyClassTest in an anonymous namespace will not work.
#define FRIEND_TEST(test_case_name, test_name)\
friend class test_case_name##_##test_name##_Test
#endif // GTEST_INCLUDE_GTEST_GTEST_PROD_H_