CMake: In which order are files parsed (cache, toolchain, etc.)?

There’s no official documentation about this particular inner workings of CMake, so please find below a summary of what I’ve learned about CMake so far …

What files are parsed depends on the

1. The host and target operating system
2. The target compiler
3. Your host computer’s environment (variables, registry, installed software)
4. Your project’s CMake script files, which could include
   1. Your toolchain file
   2. Your selected programming languages
   3. Any external projects/libraries/files/scripts

There are a lot of possible combinations of those parameters, but most of the time CMake does all the magic of automatically detecting the correct settings for you and you don’t need to bother how it’s done. The good news is – when you need to know – it follows certain intrinsic patterns.

Interesting is that it only marginally depends on the CMake generator you are selecting.

Initial Step: Compiler Detection and Verification

This mainly starts with the project() command. Taking CXX language as an example, the main files for compiler detection are (see also the root files in the question’s trace output):

• share/cmake-x.y/Modules/CMakeDetermineCXXCompiler.cmake

  This basically tries to determine the compiler executable’s location and does call it to get a more specific compiler id.

  Furthermore it e.g. defines source/output file extensions based on the host computer environment and target operating system.

• share/cmake-x.y/Modules/CMakeCXXCompiler.cmake.in

  This is the template to store the result of the compiler detection in ${CMAKE_BINARY_DIR}/${CMAKE_FILES_DIRECTORY}/x.y.z/CMakeCXXCompiler.cmake.

  Mainly those variables are: CMAKE_CXX_COMPILER, CMAKE_CXX_SOURCE_FILE_EXTENSIONS, CMAKE_CXX_IGNORE_EXTENSIONS and CMAKE_CXX_COMPILER_ENV_VAR

• share/cmake-x.y/Modules/CMakeCXXInformation.cmake

  This file sets the basic flags for the compiler. It’s also where the compiler, host and target does have the most influence on the setup with calls like this:

  include(Platform/${CMAKE_SYSTEM_NAME}-${CMAKE_CXX_COMPILER_ID}-CXX-${CMAKE_SYSTEM_PROCESSOR} OPTIONAL)
  include(Platform/${CMAKE_SYSTEM_NAME}-${CMAKE_CXX_COMPILER_ID}-CXX OPTIONAL)
  include(Platform/${CMAKE_SYSTEM_NAME}-${CMAKE_BASE_NAME} OPTIONAL)
  include(Platform/${CMAKE_SYSTEM_NAME} OPTIONAL)        
  

• share/cmake-x.y/Modules/CMakeTestCXXCompiler.cmake

  This does test everything and e.g. determine compiler features by actually calling the compiler in a simple generated CMake projects.

The results of those steps are stored in cached variables and those files are special in such case the they are safeguarded by variables like CMAKE_CXX_COMPILER_LOADED, CMAKE_CXX_INFORMATION_LOADED or CMAKE_CXX_COMPILER_WORKS to not run with each consecutive CMake configuration step again.

Project Configuration Files: Modify the Defaults

There are several ways you could change a CMake default values without actually having to touch your project’s CMakeLists.txt files.

• -C <initial-cache> command line option

  This can be used if you want to give some preset values (you would normally give via -D ... option) trough several projects over and over again. Like some library search paths on your computer or some presets used in your company.

• CMakeCache.txt through e.g. cmake-gui

  cmake-gui lets you manually modify your project’s options (editing all non-internal variables in CMakeCache.txt) before you finally generate the build environment.

• CMAKE_TOOLCHAIN_FILE

  Mainly used for cross-compiling, but it can more generally describes as preset values per compiler toolchain used.

• PreLoad.cmake

  More or less the same as the “initial cache” option (see above), but it’s not given through a command line option. It has just to be in the same directory as your project’s CMakeLists.txt.

  Note: It supports all CMake script commands like if() calls, but PreLoad.cmake has its

  • own variable scope (everything non-cached here is not visible in your main CMakeLists.txt)
  • limitations what is already known (it runs before everything else, so mostly you can check against CMAKE_GENERATOR)

• CMAKE_USER_MAKE_RULES_OVERRIDE, CMAKE_USER_MAKE_RULES_OVERRIDE_<LANG>

  This allows to modify non-cached default values after the automatic detection by CMake.

  Example: Extending the valid CXX source file extensions by .c files

  MakeRulesOverwrite.cmake

  list(APPEND CMAKE_CXX_SOURCE_FILE_EXTENSIONS c)
  

  Then you can call cmake with something like

  > cmake -D CMAKE_USER_MAKE_RULES_OVERRIDE:PATH=..\MakeRulesOverwrite.cmake ..
  

• CMAKE_PROJECT_ParserTest_INCLUDE

  This is meant to “inject custom code into project builds without modifying their source” directly after your project() command was processed (and the build environment was detected).

Toolchain.cmake: Parsed Multiple Times

A toolchain file is read multiple time while determining the system, compiler, etc.

Important to know is:

• It’s read with each try_compile() call. And since try compile must produce a valid executable, you may need – if you are e.g. cross-compiling – to

  • CMAKE_TRY_COMPILE_TARGET_TYPE to STATIC_LIBRARY (CMake version 3.6 or above)
  • Check IN_TRY_COMPILE global property to add additional options

• If you change your toolchain file, CMake will re-trigger the compiler detection (as in the trace above). Which profoundly helps to play with your compiler settings.

CMake Re-Configurations: Everything comes from the Cache

Last but not least, it’s important to know that the trace above only shows the initial step. All consecutive project configurations will take almost everything from cached variables and therefore will read much less files in the re-configuration runs.

References

• The Architecture of Open Source Applications: CMake
• Generic rule from makefile to CMake
• CMake error at CMakeLists.txt:30 (project): No CMAKE_C_COMPILER could be found