Idea: CPM.cmake-based dependency management

[TheLartians]

Hey @Neumann-A, great work on this project! I believe that simple json-based project manifests would be a huge improvement to the CMake ecosystem.

It may be interesting to see if this project can be combined with CPM.cmake for a simple CMake dependency management solution. Dependencies could be defined in an object using a { "<target-name>": <source-string> } syntax, where <source-string> is a string using CPM's single-argument syntax or is an object containing CPM.cmake parameters. This would allow for a simple and reproducible dependency management solution for many git-based dependencies without the need for any external tools or system-dependent package managers.

Example

{
    "homepage" : "<someurl>",
    "description" : "<somedescription>",
    "version" : "0.1.0",
    "languages" : ["CXX"],
    "dependencies" : {
        "Catch2": "gh:catchorg/Catch2@2.13.4",
        "linenoise": "gh:antirez/linenoise#1.0",
        "Boost::boost": "gh:Orphis/boost-cmake@1.67.0",
        "fmt": {
            "git_tag": "7.1.3",
            "github_repository": "fmtlib/fmt",
            "options": ["FMT_INSTALL YES"]
        }
    }
}

Implementation

Without having looked at the details of CMakeJSON, I believe the implementation should be very simple. After CPM.cmake has been included, we would only need to call CPMAddPackage(<source-string>) and afterwards target_link_libraries(target <target-name>) for each dependency - in the single argument case. In the object case, we would need to transform the parameters into a single string that can be forwarded to CPMAddPacakge.

While I don't have much time to work on it atm, I'd be happy to support where I can, if you are interested.

[Neumann-A]

also related: cpm-cmake/CPM.cmake#229 (comment)

---

Main design decision:

• I don't want to have CPM somewhere in the CMakeList.txt. I would be fine having it in a Find<ModuleName>.(cmake|json)
• I am a bit concerned about 'hard coding' the version. Would rather try to use the version supplied by PACKAGE_FIND_VERSION(_*) for lookup
• If the version is hard coded it should be somehow moved out of the Find<ModuleName>.(cmake|json) into a Find<ModuleName>.cpm.json or similar
• CPM needs to be optional so include(CPM OPTIONAL RESULT_VARIABLE CPM_FOUND) or similar will/should be used
• Important: Keep it agnostic to any package manager. So Find<ModuleName>.json needs to be looking behind the scenes for a Find<ModuleName>.cpm.json. Just like it would work for vcpkg. Main difference to vcpkg -> vcpkg does install deps on project(); CPM builds on build as far as I understood it.

---

Questions to answer before integrating CPM:

• How is it different from FetchContent?
• How is it different from ExternalProject_Add?
• Does it actually work from Find<ModuleName>.cmake ? Maybe I need to design Find<ModuleName>.json and then figure out how it could be integrated.

[TheLartians]

Good feedback, some (opinionated) thoughts.

I don't want to have CPM somewhere in the CMakeList.txt. I would be fine having it in a Find.(cmake|json)

The recommended way of including CPM.cmake is through the get_cpm.cmake script. This allows users to add the script without adding additional bloat to the git repo. I can actually imagine adding the CMakeJSON script using CPM as well, to allow non-intrusive updating and avoiding code bloat in projects using the JSON manifest.

I am a bit concerned about 'hard coding' the version. Would rather try to use the version supplied by PACKAGE_FIND_VERSION(_*) for lookup

I disagree, leaving the version open to later interpretation will result in unreliable builds and unexpected issues when users try to use your project on their systems. Hard-coding the version is preferable imo as builds are guaranteed to work the same, no matter the current user's system configuration.

If the version is hard coded it should be somehow moved out of the Find.(cmake|json) into a Find.cpm.json or similar

This is similar to a planned/semi-implemented feature of CPM.cmake, that would be to allow users to specify a version range and use an external version-lock file added to the project. This would be similar to npm's package-lock.json.

CPM needs to be optional so include(CPM OPTIONAL RESULT_VARIABLE CPM_FOUND) or similar will/should be used
Important: Keep it agnostic to any package manager. So Find.json needs to be looking behind the scenes for a Find.cpm.json. Just like it would work for vcpkg. Main difference to vcpkg -> vcpkg does install deps on project(); CPM builds on build as far as I understood it.

CPM.cmake itself can be made pm agnostic: if you defined the env variable CPM_USE_LOCAL_PACKAGES, it will search for a local installation using find_package before adding the source. Note though that this removes the advantages of reproducible builds (and possibly cross-compilation), as discussed above.

How is it different from FetchContent? How is it different from ExternalProject_Add?

Check out the CPM.cmake Readme for a comparison.

Does it actually work from Find.cmake ? Maybe I need to design Find.json and then figure out how it could be integrated.

I don't see why not. Personally though, I'd still recommend avoiding find_package where possible, as it makes the whole build system-dependent.

[Neumann-A]

The recommended way of including CPM.cmake is through the get_cpm.cmake script.

Hmm seems like CMakeJSON should provide something similar to that.

I disagree, leaving the version open to later interpretation will result in unreliable builds and unexpected issues when users try to use your project on their systems. Hard-coding the version is preferable imo as builds are guaranteed to work the same, no matter the current user's system configuration.

This seems to go against CMake's COMPATIBILITY idea. Also most version constrains I have seen in pkg-config files are typically minimum requirements >= and not fixed. I don't mind hard coded versions constrains in files directly related to the package manager e.g. something like vcpkg does with manifests.
What I don't want to do is lock users into whatever package managing system/solution the project normally uses as a default ...
I know there are reason to do it because it reduces the amount of issues user experience but I want to keep the associated CMakeLists.txt/project files as clean as possible. If users want easy setup CMake presets are probably the way to go nowadays (e.g. provide a preset for using vcpkg, cpm and/or conan).

I consider package management as an addon/plugin to buildscripts which should normally be done in a superbuild like script instead.

if you defined the env variable CPM_USE_LOCAL_PACKAGES

As described above the default should be opt in not opt out.

This is similar to a planned/semi-implemented feature of CPM.cmake, that would be to allow users to specify a version range and use an external version-lock file added to the project.

This seems like the thing I would be looking for. Documentation/Examples seems to be lacking though.

I don't see why not. Personally though, I'd still recommend avoiding find_package where possible, as it makes the whole build system-dependent.

Not really it makes it dependent on the package manager you use which is the correct way to go. If you use your system package manager of course it will then be system-dependent. The real problem is that most Find<ModuleName>.cmake are insufficient, incomplete or simply incorrect which is why Find<ModuleName>.json will provide some abstraction around it for libraries.
Also I doubt that CPM as the ability to correctly vendor dependencies into a project since this requires a lot of extra steps

But let me first figure out Find<ModuleName>.json then I can start looking beyond that.