Cross Platform Test Driven Development Environment Using CMake (Part 3)

Published April 08, 2013 by All8Up
Do you see issues with this article? Let us know.
Advertisement
In the last part I showed how to start adding static libraries and setup to share information between the different portions of the CMake build system. This part will cover the last of the build targets, shared libraries which are a little more difficult than the static libraries. The difficulty is not on the CMake side but instead due to Windows being a bit of a pain in this area and requiring some additional work. Once the shared libraries are complete though, the final portion of the environment will be added, the unit test library. Unit tests are a great benefit to nearly any project. A set of proper tests can help in many ways, the most obvious being to catch bugs early. Unfortunately a number of the test libraries end up requiring a lot of boilerplate code and become a pain to write. As such, when looking for a testing library the number one goal was to find something as minimal and non-intrusive as possible. At the same time, a mocking system is also useful to have for larger and more complicated testing purposes. The Google Mock library googlemock supplies both the mocking and unit test libraries and it is very simple to use. Download CMakeProject.zip and uncompress it somewhere if you don't have it from Part 2 of the series. Parts 1 2 3 4 5

Shared Libraries

The final target type which we have not implemented as of yet is the shared library. At the basic level it is no different than using a static library and in fact, on Linux and OsX it is identical except it is a shared library. Unfortunately Windows is significantly more complicated due to the requirements of importing and exporting functions. We'll start by ignoring the problems on Windows for now and getting the basics functioning on Linux and OsX. The first thing to do is add a new set of directories for the shared library: Create the following directories under CMakeProject/libraries: World Include Source Now edit the CMakeProject/libraries/CMakeList.txt as follows: ADD_SUBDIRECTORY( libraries/Hello ) ADD_SUBDIRECTORY( libraries/World ) Finally add the new files: CMakeProject/libraries/World/CMakeLists.txt: SET( INCLUDE_DIRS ${CMAKE_CURRENT_LIST_DIR}/Include ) INCLUDE_DIRECTORIES( ${INCLUDE_DIRS} ) SET( INCLUDE_FILES Include/World.hpp ) SET( SOURCE_FILES Source/World.cpp ) ADD_LIBRARY( World SHARED ${INCLUDE_FILES} ${SOURCE_FILES} ) # Export the include directory. SET( WORLD_INCLUDE_DIRS ${INCLUDE_DIRS} PARENT_SCOPE ) CMakeProject/libraries/World/Include/World.hpp: #pragma once const char* const WorldString(); CMakeProject/libraries/World/Source/World.cpp: #include const char* const WorldString() { return "World!"; } Regenerate the build and you now have a shared library. If you are using Linux or OsX, your library is complete and can be used by simply adding it to a target the same way the static library was added. Let's make a couple minor modifications in order to use this library and link against it. CMakeProject/libraries/Hello/Source/Hello.cpp: #include const char* const HelloString() { return "Hello"; } CMakeProject/tests/Hello2/main.cpp: /* Hello World program */ #include #include #include #include int main( int argc, char** argv ) { std::cout << HelloString() << " " << WorldString(); return 0; } CMakeProject/tests/CMakeLists.txt: CMAKE_MINIMUM_REQUIRED( VERSION 2.6.4 ) INCLUDE_DIRECTORIES( ${HELLO_INCLUDE_DIRS} ${WORLD_INCLUDE_DIRS} ) PROJECT( HelloWorld ) ADD_EXECUTABLE( Hello2 main.cpp ) TARGET_LINK_LIBRARIES( Hello2 Hello World ) With the given changes, Linux and OsX can build and run the modified 'Hello2' project and the string for "Hello" comes from a static library function and the "World!" comes from a shared library function. Unfortunately Windows is not so lucky and will fail to link. CMakeProjectWindowsBroken.zip

The Windows DLL

As with the OsX specific fix, using shared libraries with Windows is something of a black art. I don't intend to explain all the details, that would take another article. I will simply explain the basics of the fix and make the code work on all three platforms again. The basic problem with using shared libraries on Windows requires a little description. First off they are called dynamic link libraries (aka DLL) on windows. Second they are split into two pieces, the actual DLL portion which is the shared library where the code lives and a link library which is like a static library. Confusingly the link library usually has the file extention 'lib' just like an actual static library so don't get them confused. At this time, we are not telling the compiler to generate this intermediate file.
To add more confusion, the lib "is" actually just a static library. It simply contains meta data to tell Windows the DLL to load and stub functions which are fixed up after load. Now forget this and continue on with the simple description.
Generating the intermediate file is simple enough but you have to deal with it in a cross platform nature or you break Linux and OsX builds. Exporting the library symbol and making the link library is as simple as changing the function declaration to be the following: __declspec( dllexport ) const char* const WorldString(); If you go ahead and make this change, the project compiles on Windows, but it will not run because it can't find the DLL. This problem is an annoyance with how VC tends to layout the projects when divided up in the manner suggested in this series of articles. Fixing this digs into the darker recesses of CMake properties which, for the moment we will avoid by cheating. For the time being, just copy the dll manually: Copy from CMakeProject/build/vc11/libraries/World/Debug/World.dll to CMakeProject/build/vc11/tests/Hello2/Debug/World.dll At this point Windows is functioning again, though with obvious cheats and actually a glaring error, though the error doesn't cause a problem at the moment. What's the error? Well, to be proper you are supposed to use '__declspec( dllexport )' only in the code building the DLL and then '__declspec( dllimport )' in code which uses the library. While it works as it is, it is best to follow the rules as closely as possible so as not to get strange behaviors/errors at a later time. So, we'll extend the code to correct the issue of import versus export: #ifdef BUILDING_WORLD # define WORLD_EXPORT __declspec( dllexport ) #else # define WORLD_EXPORT __declspec( dllimport ) #endif WORLD_EXPORT const char* const WorldString(); Rebuild and now you get a warning: "warning C4273: 'WorldString' : inconsistent dll linkage" The reason is, we have not defined 'BUILDING_WORLD'. Since we want to define this only for the 'World' library, we edit the CMakeProject/libraries/World/CMakeLists.txt as follows: SET( INCLUDE_DIRS ${CMAKE_CURRENT_LIST_DIR}/Include ) INCLUDE_DIRECTORIES( ${INCLUDE_DIRS} ) SET( INCLUDE_FILES Include/World.hpp ) SET( SOURCE_FILES Source/World.cpp ) ADD_DEFINITIONS( -DBUILDING_WORLD ) ADD_LIBRARY( World SHARED ${INCLUDE_FILES} ${SOURCE_FILES} ) # Export the include directory. SET( WORLD_INCLUDE_DIRS ${INCLUDE_DIRS} PARENT_SCOPE ) Notice the new CMake command: 'ADD_DEFINITIONS'. As the name states, we are adding a definition to the macro preprocessor named 'BUILDING_WORLD' to the 'Hello2' target. Windows is happy, we are following the rules and other than having to manually copy the DLL for the moment, it functions. There is one last issue, the macro will be applied to all platforms, something we don't want. So back in CMakeProject/libraries/World/Include/World.hpp, make the following change: #ifdef _WINDOWS # ifdef BUILDING_WORLD # define WORLD_EXPORT __declspec( dllexport ) # else # define WORLD_EXPORT __declspec( dllimport ) # endif #else # define WORLD_EXPORT #endif WORLD_EXPORT const char* const WorldString(); CMake inserts per OS specific definitions based on the platform being targetted. In this case, as with using Visual Studio without CMake, '_WINDOWS' is defined by default. So, we can rely on this and now the code is safely cross platform and ready for commit.

Changing Output Directories

In order to finish correcting the Windows build we have to use a bit of CMake voodoo which is not often documented particularly well. Changing output directories. What we want is to output the executables and the libraries (shared and static) to a single output directory while leaving intermediate files wherever CMake/VC wants to store them. This is not required for OsX and Linux as the output generators there follow a more simplified directory structure. We also want to do this only for Visual Studio and not change the default behavior of nmake, CodeBlocks and other generators. Modify the CMakeProject/CMakeLists.txt to be the following: CMAKE_MINIMUM_REQUIRED( VERSION 2.6.4 ) IF( APPLE ) SET( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -stdlib=libc++" ) SET( CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -stdlib=libc++" ) SET( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -stdlib=libc++" ) SET( CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -stdlib=libc++" ) ENDIF( APPLE ) PROJECT( CMakeProject ) IF( MSVC ) SET( CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/Binaries ) SET( CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/Binaries ) SET( CMAKE_LIBRARY_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/Binaries ) ENDIF( MSVC ) INCLUDE( libraries/CMakeLists.txt ) ADD_SUBDIRECTORY( tests ) Note that the detection for Visual Studio is performed after the 'PROJECT' command. There is a reason for this. CMake does not initialize many variables until after the 'PROJECT' command is issued. Among the variables not initialized is of course 'MSVC'. The reason for the delayed definition is that CMake reads the locally catched data at the point where it see's the 'PROJECT' command. So, while knowing the platform being run on and being able to modify the compile and linker flags can be done before the 'PROJECT' command, the specifics such as the generator in use are not known until the cache data is processed. The next item to note is: why three directories? CMake divides the directories into three pieces to match the three target types, runtime for the executables, archive for static libraries and library for shared/dynamic libraries. While you could just set the executable and shared library paths, I include the static libraries in case something external to this build system wanted to link against the libraries. It is not required but makes things easier in advanced usages. Finally, what is the 'PROJECT_BINARY_DIR' variable being used? Before we define any of the targets, this variable simply points to the directory CMake was launched from as it's out of source build location. In my case on Windows, I'm using '/CMakeProject/build/Vc11' and as such that is what 'PROJECT_BINARY_DIR' is initialized with. There are many other variables, some which will also point to this location and could have been used. I just like this one because it is related to what I'm trying to accomplish, which is put all the final compiled binaries in one place. Regenerate the project, rebuild all and you will now have a single directory with all the binary outputs within it. You can run the examples using Visual Studio and it will no longer complain that it can not find the DLL to load.
On some installations of Visual Studio when used with CMake, Visual Studio will complain that it can't find a file to debug. My Win7 desktop has this problem where my Win7 laptop does not. The easy fix, if you run into this, is to simply switch the dropdown Debug Type from 'Auto' to 'Mixed'. VC now has no problems. I do not know the source of this issue or I would fix it.
CMakeProjectFullFix.zip

Unit Testing

With a title that states "Test Driven Development" in it, you would probably think that adding unit tests would have been done a bit earlier. The funny thing is that you have had a unit testing framework the entire time, it is built right into CMake itself called CTest. The downside is that I don't much like the way the built in functionality works and incorporating an external library is a good learning experience. So, we will start using googlemock in a little bit. During the course of writing this series I initially thought that I would stick to an 'all lower case' naming convention. In the process of writing, I mostly added code quickly and as such fell back on my habitual way of doing things without paying much attention. Other than to say I'm going to use camel case for the names, we'll skip the reasons and simply start with a fresh empty environment based on all the work we have done and also adding the final target divisions, so download the following and we'll start adding the unit test library: CMakeEnvironment.zip

The New Environment

There is not much changed in the new environment other than applying the mentioned capitalizations and adding a couple new subdirectories along with the removal of 'tests'. The directories are intended as follows:
  • Build: A placeholder for where you can use CMake to store its cache files and such. I tend to use subdirectories such as: Vc11, Xcode, Makefiles, etc.
  • External: The location we will be storing external libraries such as googlemock.
  • Libraries: Static and shared libraries used by the projects.
  • Applications: Your game, potentially another game and anything which is not a tool but uses the libraries.
  • Tools: Obviously things which are used in the creation of the game. Exporters, converters, big level builders, etc.

Go Get GoogleMock

Download the gmock-1.6.0.zip file from googlemock and decompress it into the 'CMakeEnvironment/External' directory. You should end up with a new folder: 'CMakeEnvironment/External/gmock-1.6.0' which contains the library source. If you go looking around in the folder you will notice that it uses autoconf and m4 scripting to build on Linux/OsX and supplies a solution and project file for Visual Studio. This is normally fine for simple things but I want it integrated with the CMake build environment more tightly such that if I change global compiler flags it will rebuild along with the other sections of the project. As you will see, all the little steps we've gone through in setting up the environment with CMake in a clean manner will start to pay off as we integrate the library. Thankfully, as with many other Google libraries, the source contains a 'fused-src' directory which means we can add two headers and a source file to our project and almost be done with it. I say almost, since I want to make sure we integrate the library within the environment as cleanly as possible and make no modifications to the unzipped directory. Start by adding the following (and creating the new directories): CMakeEnvironment/External/CMake/GMock/CMakeLists.txt: # Use a variable to point to the root of googlemock within # the External directory. This makes it easier to update # to new releases as desired. SET( GOOGLEMOCK_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/../../gmock-1.6.0" ) # Make the file lists. SET( INCLUDE_FILES ${GOOGLEMOCK_ROOT}/fused-src/gtest/gtest.h ${GOOGLEMOCK_ROOT}/fused-src/gmock/gmock.h ) SET( SOURCE_FILES ${GOOGLEMOCK_ROOT}/fused-src/gmock-gtest-all.cc ) # Setup some build definitions for VC2012. IF( MSVC ) ADD_DEFINITIONS( -D_VARIADIC_MAX=10 ) ENDIF( MSVC ) # Add as a static library target. ADD_LIBRARY( GMock STATIC ${INCLUDE_FILES} ${SOURCE_FILES} ) This will allow GMock and GTest to compile on Windows and Linux. Unfortunately nothing ever seems to work without problems. The library refuses to compile on OsX due to Clang being detected as GCC and making a bad assumption about the availability of tr1 files. After arguing with it and even hacking the source I decided that for the time being, the easiest and most time effective method was to cheat. The benefit of cheating is under rated when you need to get things done and don't want to hack on external code. So, what exactly is this cheat? We'll add a directory for tr1 and a file for tuple, the file will simply alias the to C++11 tuple in order to meet the requirements. (I won't detail the changes, check out the various CMakeLists.txt if you are curious.) Additionally, we have a link problem with the Linux environments to fix. If you tried to link against the GMock library and build you will get link errors, given the output it becomes pretty apparent that we've missed the threading libraries. In order to fix this, we get to learn a new command in CMake: FIND_PACKAGE. Anytime you need a set of specific includes and/or libraries to link against, there is usually a module which CMake supplies (or the community supplies) for the item. In this case we need to find the 'Threads' package. Once we do this, we add a new target link library to our binaries: '${CMAKE_THREAD_LIBS_INIT}'. This will take care of the linking on Linux and also because it is a variable, it will be empty on other platforms which don't need explicit linkage to an additional library. As with static libraries in general, we know it is easy to link against the library but a bit more of a challenge to get the include paths setup. Additionally, the variadic work around for VC needs to be pushed out as a global item for all projects, otherwise they will fail to compile. This is all stuff we've dealt with before so it becomes easy. In the root CMake file move the 'ADD_DEFINITIONS' command to be just under the output path changes we made and also insert the FIND_PACKAGE command for all builds: CMakeEnvironment/CMakeLists.txt: # ############################################### # Make VC happy and group all the binary outputs, # also make sure GMock headers will compile in # all targets. IF( MSVC ) SET( CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/Binaries ) SET( CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/Binaries ) SET( CMAKE_LIBRARY_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/Binaries ) ADD_DEFINITIONS( -D_VARIADIC_MAX=10 ) ENDIF( MSVC ) # ############################################### # Find the threading package for this machine. FIND_PACKAGE( Threads ) Remove the definition line from the GMock/CMakeLists.txt and add the following to the bottom of the file: SET( GMOCK_INCLUDE_DIRS ${GOOGLEMOCK_ROOT}/fused-src PARENT_SCOPE )

Controlling Tests

The last thing to be done is to control if unit tests are compiled and built. When you start working this is not a big deal of course as you will likely only have a couple libraries and tests. After a while though, you may end up with more little "test" targets than actual libraries and applications. So, it is a good idea to go ahead and add the ability to disable tests from day one. Add the following item: CMakeEnvironment/CMakeLists.txt: # ############################################### # Allow unit tests to be disabled via command # line or the CMake GUI. OPTION( BUILD_UNIT_TESTS "Build unit tests." ON ) Add this just under the 'PROJECT' command and it will show up in the CMake GUI as something you can turn on and off.

Putting The Results To Use

Two and a half articles later, we are finally ready to put our work to good use. I'm going to start by writing a 3D vector class since it is a common item and easily unit tested. The first thing to do is of course get the completed environment. This file includes the googlemock library so it's quite a bit larger than the prior items: CMakeEnvironmentComplete.zip

The Math Library

I'm going to keep this very simple for the purposes of this article. I will not be implementing the actual class, just enough to show a unit test in action. Future articles will likely build off this work but this is going to close out the CMake work for the time being so we want to keep things simple. Let's start by adding the directories for the new 'Math' library: CMakeEnvironment Libraries Math Include Math Source Tests Ok, so the first question is likely to be why did I duplicate the directory 'Math' under the 'Include' directory? This is simply a polution prevention item. Let's say you have a custom 'Collections' library and within that there is a 'Vector.hpp' file and of course the math library could have a 'Vector.hpp' file. If you include without the prefix of 'Collections' or 'Math', which file are you including? With the way we will setup the libraries, this problem is solved by forcing the users of the libraries to qualify the includes as follows: #include #include This is another one of the personal preference items but I prefer avoiding problems from day one and not simply assuming they won't happen. If you don't like this, once we get done feel free to remove the nested directory. But be warned that in future articles you'll see some good reasons to use this pattern which can greatly simplify normally difficult processes. Let's fill in the hierarchy with files now, add the following: CMakeEnvironment/Libraries/Math/CMakeLists.txt: SET( INCLUDE_DIRS ${CMAKE_CURRENT_LIST_DIR}/Include ) INCLUDE_DIRECTORIES( ${INCLUDE_DIRS} ) SET( INCLUDE_FILES Include/Math/Math.hpp Include/Math/Vector3.hpp ) SET( SOURCE_FILES Source/Vector3.cpp ) ADD_DEFINITIONS( -DBUILDING_WORLD ) ADD_LIBRARY( Math STATIC ${INCLUDE_FILES} ${SOURCE_FILES} ) # Export the include directory. SET( MATH_INCLUDE_DIRS ${INCLUDE_DIRS} PARENT_SCOPE ) Add empty files for: Include/Math/Math.hpp Include/Math/Vector3.hpp Source/Vector3.cpp Tests/Main.cpp Regenerate the environment and you should have your new Math library.

The First Unit Test

The first thing we want to do is add a unit test holder before we add any code to the library. What we need is a simple executable which can include the files from the library. Since this is part of the math library, we can simply add the test within 'CMakeEnvironment/Libraries/Math/CMakeLists.txt'. Additionally, we want the test to honor the flag we setup such that the test is not included when tests are disabled. Add the following to the end of the Math libraries CMake file: # Make a unit test holder for the Math library. IF( BUILD_UNIT_TESTS ) # Add the gmock include directories. INCLUDE_DIRECTORIES( ${GMOCK_INCLUDE_DIRS} ) ADD_EXECUTABLE( _TestMath Tests/Main.cpp ) TARGET_LINK_LIBRARIES( _TestMath Math GMock ${CMAKE_THREAD_LIBS_INIT} ) ADD_TEST( NAME _TestMath COMMAND _TestMath ) ENDIF( BUILD_UNIT_TESTS ) Regenerate and you get a new target '_TestMath'. In the CMake GUI if you set 'BUILD_UNIT_TESTS' to 'FALSE' and regenerate, the new target goes away as we desired. But, there is actually one more thing we want to do to make this even better in the future. In the root CMake file with the 'OPTION' command defining the 'BUILD_UNIT_TESTS' variable, add the following bit right afterward: # ############################################### # Enable the CMake built in CTest system if unit # tests are enabled. IF( BUILD_UNIT_TESTS ) ENABLE_TESTING() ENDIF( BUILD_UNIT_TESTS ) Make sure to set 'BUILD_UNIT_TESTS' back to 'TRUE' and regenerate. A new target shows up in the build: "RUN_TESTS". This is a nice little target when you have many more tests then we will have here. Basically if you attempt to build this target is runs all the executables you add with 'ADD_TEST'. It is also great for automated build/continuous integration environments since running the target is easy compared to finding all the individual tests. Now, lets add the actual unit test code: CMakeEnvironment/Libraries/Math/Tests/Main.cpp #include int main( int argc, char** argv ) { ::testing::InitGoogleTest( &argc, argv ); return RUN_ALL_TESTS(); } This is a do nothing bit of code right now until we add some code to the Math library and of course add some tests. So, let's fill in the vector class real quick: CMakeEnvironment/Libraries/Math/Include/Math/Vector3.hpp: #pragma once #include namespace Math { class Vector3f { public: Vector3f() {} Vector3f( float x, float y, float z ); float X() const {return mX;} float Y() const {return mY;} float Z() const {return mZ;} private: float mX; float mY; float mZ; }; } CMakeEnvironment/Libraries/Math/Source/Vector3.cpp: #include using namespace Math; Vector3f::Vector3f( float x, float y, float z ) : mX( x ) , mY( y ) , mZ( z ) { }
Yes, the default constructor does not initialize to zero's. That is a topic for another article.
Rebuild and the library should build, the test case should build and even run with the following output: [==========] Running 0 tests from 0 test cases. [==========] 0 tests from 0 test cases ran. (1 ms total) [ PASSED ] 0 tests. So it is time to add our first tests. Add the following files: CMakeEnvironment/Libraries/Math/Tests/TestAll.hpp: #include #include "TestConstruction.hpp" CMakeEnvironment/Libraries/Math/Tests/TestConstruction.hpp: TEST( Math, Vector3f ) { Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 0.0f, test0.X() ); EXPECT_EQ( 0.0f, test0.Y() ); EXPECT_EQ( 0.0f, test0.Z() ); Math::Vector3f test1x( 1.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); EXPECT_EQ( 0.0f, test0.Y() ); EXPECT_EQ( 0.0f, test0.Z() ); Math::Vector3f test1y( 0.0f, 1.0f, 0.0f ); EXPECT_EQ( 0.0f, test0.X() ); EXPECT_EQ( 1.0f, test0.Y() ); EXPECT_EQ( 0.0f, test0.Z() ); Math::Vector3f test1z( 0.0f, 0.0f, 1.0f ); EXPECT_EQ( 0.0f, test0.X() ); EXPECT_EQ( 0.0f, test0.Y() ); EXPECT_EQ( 1.0f, test0.Z() ); } Modify CMakeEnvironment/Libraries/Math/CMakeLists.txt by adding the test headers: ADD_EXECUTABLE( _TestMath Tests/Main.cpp Tests/TestAll.hpp Tests/TestConstruction.hpp ) And include the 'TestAll.hpp' from the file CMakeEnvironment/Libraries/Math/Tests/Main.cpp: #include #include "TestAll.hpp" int main( int argc, char** argv ) { ::testing::InitGoogleTest( &argc, argv ); return RUN_ALL_TESTS(); } Regenerate, build and run. The test should output the following: [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from Math [ RUN ] Math.Vector3f [ OK ] Math.Vector3f (0 ms) [----------] 1 test from Math (1 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (5 ms total) [ PASSED ] 1 test. Congratulations, you have a unit test for your new Vector3 class. The completed environment and example unit test can be downloaded here CMakeEnvironmentWithTest.zip Please provide feedback on any build problems you might have and I'll attempt to get things fixed.

Conclusion

In this article we covered dealing with OS specific difficulties involving Windows and a couple more OsX issues. It has been a long haul to get to this point dealing with nitpicky items and a lot of explanation. The final result is a fairly easy to use and maintain environment which you can use to build unit tests and apply test driven development features to your coding. In a future article I intend to expand on the Math library and show some of the big utilities of using an environment such as this day to day. For now, my fingers are tired and my own code calls to me.
Cancel Save
0 Likes 17 Comments

Comments

slimshader

Last few weeks I am moving from Win-only (Visual Studio) build to portable CMake version. One of few things to still figure out is unit testing integrated into the process. I came back after Easter and see this series - best gift I could ask for :) Great work and thanks for doing this!

April 02, 2013 04:25 PM
slimshader

I might be missing something but:

1) should gmock/gtest results be displayed in VS "Output" window during the build?

2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior?

April 02, 2013 06:17 PM
slimshader

Since you mentioned ADD_DEFINITIONS...: how do you deal with #defines that have to be added only in specific variant of the build (like debug)?

April 02, 2013 06:34 PM
All8Up

I might be missing something but:

1) should gmock/gtest results be displayed in VS "Output" window during the build?

2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior?

The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed.

For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. :)

April 03, 2013 12:47 AM
All8Up

Since you mentioned ADD_DEFINITIONS...: how do you deal with #defines that have to be added only in specific variant of the build (like debug)?

This one gets to be a bit more difficult. I was considering if I should cover this later as at this point you get into some of the less well understood areas of CMake. But, there is a relatively simple solution if you need per build type definitions. CMake will add "_DEBUG" flags in all of the generators. So, if you need to differentiate some flags, just "#if _DEBUG" for the debug versions "#else" for the release versions.

April 03, 2013 01:12 AM
slimshader

I might be missing something but:

1) should gmock/gtest results be displayed in VS "Output" window during the build?

2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior?

The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed.

For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. smile.png

Actually that is not what I meant. ATM my VS build is configured in this pretty sweet way: http://leefrancis.org/2010/11/17/google-test-gtest-setup-with-microsoft-visual-studio-2008-c/

gtest_main executable is executed as post-build step of dependent target which most importantly causes fail of main build (tests binary returns non-zero to VS) on failed test. On top of that, since testing is now part of the build, every test result is displayed in "Output" window and on error you can just double-click in it and it jumps to failed ASSERT/EXPECT (like wit normal errors and warnings).

April 03, 2013 07:21 AM
Peter Mrozek

I just love articles that explain a single problem in much detail, especially when it comes to the most basic thing before you start developing: the environment itself. Combined with the TDD approach, I'll definitely be using this in my own project. :)

One little thing: could you please correct the navigation links? The link to part 2 points to part 1. ;)

April 03, 2013 01:42 PM
All8Up

I might be missing something but:

1) should gmock/gtest results be displayed in VS "Output" window during the build?

2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior?

The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed.

For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. smile.png

Actually that is not what I meant. ATM my VS build is configured in this pretty sweet way: http://leefrancis.org/2010/11/17/google-test-gtest-setup-with-microsoft-visual-studio-2008-c/

gtest_main executable is executed as post-build step of dependent target which most importantly causes fail of main build (tests binary returns non-zero to VS) on failed test. On top of that, since testing is now part of the build, every test result is displayed in "Output" window and on error you can just double-click in it and it jumps to failed ASSERT/EXPECT (like wit normal errors and warnings).

I took a look at the description you mentioned. That seems like a nice integration. I'll look into it further.

The one thing about the integration though is I will often have a couple tests which fail as I do refactoring work, so I don't want the tests to cause a failure when I run things most of the time. I just want to know which ones failed as I make changes. Especially when you have say 100+ tests going, a single failure may not be a horrible problem though knowing about it is a good idea.

April 03, 2013 04:41 PM
All8Up

I just love articles that explain a single problem in much detail, especially when it comes to the most basic thing before you start developing: the environment itself. Combined with the TDD approach, I'll definitely be using this in my own project. smile.png

One little thing: could you please correct the navigation links? The link to part 2 points to part 1. ;)

Err, oops.. :) I'm traveling at the moment but I'll try to get it fixed up. Thanks for the feedback though, glad you like it.

As a note, I haven't yet decided what to do with the fourth article. The original intention was to get to an x-plat SIMD capable math library and use that to show some of the useful bits of the setup and why I use the embedded extra named directory. I'm trying to decide if I should continue on with the CMake first though as the setup and such has updated quite a lot. There are some things in the posted one done in less than the best manners, functional yes but intended for simplicity instead of using the absolute best practices. (With CMake it is not always obvious what the best case is of course. :))

I may attach the more advanced version up as a little after note to part 3 but the features will take another article to describe. Some of the interesting bits for folks would likely be the use of a configure system to generate a "Config.hpp" specific to your target and settings. The new settings include IOS support, control over SIMD (SSE1-AVX for Intel and Neon for Arm) and general cleanup and moving things around. Also of note was that In the posted version I forgot to properly cover the Xcode specific attribute variables which means Xcode can sometimes be a bit annoying since some things may fail once in a while. (It's just a random "I don't support Cxx11' compiles, the next time it will likely work.)

Anyway, that is a bunch more stuff which could take another part added to the article. Probably a good idea to go ahead and do that and push the other stuff a bit later.

April 03, 2013 05:14 PM
l0calh05t

I might be missing something but:

1) should gmock/gtest results be displayed in VS "Output" window during the build?

2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior?

The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed.

For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. smile.png

Actually that is not what I meant. ATM my VS build is configured in this pretty sweet way: http://leefrancis.org/2010/11/17/google-test-gtest-setup-with-microsoft-visual-studio-2008-c/

gtest_main executable is executed as post-build step of dependent target which most importantly causes fail of main build (tests binary returns non-zero to VS) on failed test. On top of that, since testing is now part of the build, every test result is displayed in "Output" window and on error you can just double-click in it and it jumps to failed ASSERT/EXPECT (like wit normal errors and warnings).

If you add executing the individual test as a post-build step, the test will be run as a part of the build (and rerun every time the test is recompiled due to changes).


# Make a unit test holder for the Math library.
IF( BUILD_UNIT_TESTS )
	# Add the gmock include directories.
	INCLUDE_DIRECTORIES( ${GMOCK_INCLUDE_DIRS} )

	ADD_EXECUTABLE( _TestMath
		Tests/Main.cpp
	)
	TARGET_LINK_LIBRARIES( _TestMath
		Math
		GMock
		${CMAKE_THREAD_LIBS_INIT}
	)
	ADD_CUSTOM_COMMAND(
		TARGET _TestMath
		POST_BUILD
		COMMAND _TestMath
	)

	ADD_TEST( NAME
		_TestMath
		COMMAND
		_TestMath
	)
ENDIF( BUILD_UNIT_TESTS )

As this will make the compile fail if any test fails (and make will not build anything else if that happens unless started with -i), you should wrap the ADD_CUSTOM_COMMAND with an IF and only enable it on request (for use with IDEs).

EDIT: Note that MSVC will happily build the other targets even if tests fail, so the test failing during development isn't really an issue (but you get the information right away which is good!)

April 07, 2013 11:59 AM
All8Up


	ADD_CUSTOM_COMMAND(
		TARGET _TestMath
		POST_BUILD
		COMMAND _TestMath
	)

	ADD_TEST( NAME
		_TestMath
		COMMAND
		_TestMath
	)
ENDIF( BUILD_UNIT_TESTS )

A very good point, unfortunately I had not introduced that CMake command and I didn't want to suggest anything which I had not documented and shown a more substantial example of. I want to keep this to the level of "practical" and not dig into every little feature of CMake, I figured a later article could show integration with CruiseControl or something like that and these items are a great place to discuss the custom commands.

For me personally, I just prefer to right click run tests when I'm done editing in VC since "building" is part of my process, even though I may not run the result yet, it is just a double check of my code to that point. So a failure during the build because I broke unit tests would annoy my normal working pattern. Not to mention that a couple of my tests are stressful and can take 10 seconds each to complete. Those are mock based tests though they get merged into the test results as normal.

Anyway, it is a good point and I wanted to revisit this now that I'm back from my trip. We'll see where it goes after Part 4 is posted.

April 07, 2013 11:36 PM
abigagli

Hi,
very interesting series, and really useful as I'm in the process of moving to CMake for a couple of projects I'm currently working on.

  • First, just a typo I suspect, as you say that all tests should pass, but the code for CMakeEnvironment/Libraries/Math/Tests/TestConstruction.hpp always uses test0.X(), test0.Y(), test0.Y() in the EXPECT_EQ clauses, even when the object under test is test1x, test1y, test1z[/size]?
  • Second, regarding the problem of letting know to the upper levels the proper INCLUDE_DIRECTORIES, how does it sound to have a PROJECT(libname) directive in each library and then in the root CMakeLists.txt have something like INCLUDE_DIRECTORIES(${libname_SOURCE_DIR/<where_are_the_includes>})? Of course this doesn't work for the external libraries, but I see it as a cleaner solution for in-project libraries. What do you think?[/size]
April 21, 2013 02:12 PM
All8Up

hi Abi,

First item: Oops! :) Think I fixed it later but didn't update the various zip file attachments. GameDev needs a better method of integrating code into articles as the zip files are a pain to keep maintained.

Second: I'm not sure I'm following the idea. Though I suspect perhaps you are saying something like:

a .cmake file in the projects/libraries etc directories we include and each of those then defines what to include/use so we are only at one step removed? I might have to play with this, but honestly, I'm not sure it really changes the problem, just moves it. I'll have to think about it, I still have a couple more articles going so perhaps I'll hit a CMake part 5 with suggestions and fixes included. :)

April 25, 2013 02:46 AM
abigagli

[...]

Second: I'm not sure I'm following the idea. Though I suspect perhaps you are saying something like:

a .cmake file in the projects/libraries etc directories we include and each of those then defines what to include/use so we are only at one step removed? I might have to play with this, but honestly, I'm not sure it really changes the problem, just moves it. I'll have to think about it, I still have a couple more articles going so perhaps I'll hit a CMake part 5 with suggestions and fixes included. smile.png

Well, I'm just saying that after a PROJECT (<MYLIBNAME>) directive, it seems like cmake makes the <MYLIBNAME>_SOURCE_DIR variable defined to point where sources for that project are, so if <MYLIBNAME>'s includes are in a well defined relative path from its sources, you can get a valid pointer to their location everywhere by just using such variable e.g. <MYLIBNAME>_SOURCE_DIR/../Includes if there's an "Includes" folder at the same level where a Sources folder is.

Hope this clarifies my idea a bit, and looking forward parts >=5 for additional great content!

April 25, 2013 10:17 AM
All8Up


[...]

Second: I'm not sure I'm following the idea. Though I suspect perhaps you are saying something like:

Well, I'm just saying that after a PROJECT (<MYLIBNAME>) directive, it seems like cmake makes the <MYLIBNAME>_SOURCE_DIR variable defined to point where sources for that project are, so if <MYLIBNAME>'s includes are in a well defined relative path from its sources, you can get a valid pointer to their location everywhere by just using such variable e.g. <MYLIBNAME>_SOURCE_DIR/../Includes if there's an "Includes" folder at the same level where a Sources folder is.

Hope this clarifies my idea a bit, and looking forward parts >=5 for additional great content!

Think I understand now. Unfortunately it leaves a couple problems to be solved. There are three things which need to be exported from the libraries:

:The library to link against, and any which it may need to add to the build. For instance, if I link in a network library, I'd also need to pull ws_32.dll on windows to make it function.

:The various extra defines for the library, sometimes you have some extra's required, again picking on Window's you might need to define an OS version.

:The include directory of course but it can actually be multiple directories. Say for instance you have a renderer, it may need to include D3D/OpenGL directories also.

I'll keep looking at this but for the time being I think I like the explicit nature of the current version. It is a bit verbose but that let's it adapt to any edge cases. I'll keep thinking about it though.

April 26, 2013 06:54 AM
TomRyba
Hi AllEightUp,
I hit the same problem as you did with TR1 tuple lib in Xcode. Apparently google guys already saw that coming and added few compiler flags.

Just add one of the following to root CMakeLists.txt
# To force Google Test to use its own tuple library
ADD_DEFINITIONS( -DGTEST_USE_OWN_TR1_TUPLE=1 )
# or if you don't want Google Test to use tuple at all
ADD_DEFINITIONS( -DGTEST_HAS_TR1_TUPLE=0 )

Hope that helps.
May 08, 2013 03:09 AM
All8Up

I had tried to use the various flags, such as that one, but never got a working build for some reason. I decided that a little cheatery was just as easy as figuring out the problems. I will give it another try at some point but I was hoping the library would be updated before I tried again.

May 08, 2013 06:45 PM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!

In Part 3, it is time to finish off the basic build environment by adding the last target type, shared libraries. Unfortunately this portion involves Windows specifics and takes a while to make fully cross platform. When everything is functional it will be time to add the unit testing framework. The remainder of the article will cover how to setup the unit tests with CMake and various methods of using the build environment.

Advertisement
Advertisement