Skip to content

fix(cmake): Split CMake settings generation from library install tasks to support multi-project dependency builds. #88

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

fix(cmake): Split CMake settings generation from library install tasks to support multi-project dependency builds. #88

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
60e4436
fix cmake settings
Bill-hbrhbr Oct 5, 2025
2e05b9f
Add fixxing quote
Bill-hbrhbr Oct 5, 2025
025e070
typo fix
Bill-hbrhbr Oct 5, 2025
00bb47b
Add labels for task that runs when changed
Bill-hbrhbr Oct 6, 2025
1126ba6
Split out generate cmake settings from the install task
Bill-hbrhbr Oct 6, 2025
022fe92
Boost rework
Bill-hbrhbr Oct 6, 2025
73d6e96
Avoid concurrent cmake file writing
Bill-hbrhbr Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 20 additions & 31 deletions exports/taskfiles/utils/boost.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ set: ["u", "pipefail"]
shopt: ["globstar"]

tasks:
# Generates `GENERATE_DIR` by copying `SOURCE_DIR` and then running boost's bootstrap step.
# Generates `GENERATE_DIR` by copying `SOURCE_DIR` and then running Boost's bootstrap step.
#
# @param {string} SOURCE_DIR Project source directory.
# @param {string} GENERATE_DIR Directory in which to generate `b2` and the build configuration.
Expand Down Expand Up @@ -54,19 +54,17 @@ tasks:
INCLUDE_PATTERNS:
- "{{.GENERATE_DIR}}"

# Runs boost's b2 build and install step, and creates a CMake settings file. The caller must have
# Runs Boost's b2 build and install step, and creates a CMake settings file. The caller must have
# previously called `generate` on `SOURCE_DIR` (to produce the `GENERATE_DIR`) for this task to
# succeed.
#
# @param {string} GENERATE_DIR Directory containing boost's source and build files.
# @param {string} BUILD_DIR Directory in which to build boost.
# @param {string} GENERATE_DIR Directory containing Boost's source and build files.
# @param {string} BUILD_DIR Directory in which to build Boost.
# @param {string} INSTALL_PREFIX Path prefix of where the project should be installed.
# @param {string[]} [EXTRA_ARGS] Any additional arguments to pass to the build command.
# @param {int} [JOBS] The maximum number of concurrent processes to use when building. If
# omitted, the b2 default number is used. Before 1.76.0, the number was 1. Since 1.76.0, the
# default is the number of cores.
# @param {string} [CMAKE_SETTINGS_DIR] If set, the directory where the project's CMake settings
# file should be stored.
build-and-install:
internal: true
vars:
Expand All @@ -89,79 +87,71 @@ tasks:
"-j{{.JOBS}}"
{{- end}};
popd;
- >-
{{- if .CMAKE_SETTINGS_DIR}}
echo "set(Boost_ROOT
\"{{.INSTALL_PREFIX}}\"
CACHE PATH
\"Package root for Boost.\"
)" >> "{{.CMAKE_SETTINGS_DIR}}/Boost.cmake"
{{- end}}

# Downloads boost from `URL` and installs boost.
# Downloads Boost from `URL` and installs Boost.
#
# General parameters
# @param {string} [WORK_DIR={{.ROOT_DIR}}] Base directory in which to store the source, generate,
# build, and install directories.
# @param {string} [SOURCE_DIR={{.WORK_DIR}}/boost-src] Directory in which to extract the tar
# @param {string} [SOURCE_DIR={{.WORK_DIR}}/Boost-src] Directory in which to extract the tar
# file.
#
# Download parameters
# @param {string} FILE_SHA256 Content hash to verify the downloaded tar file against.
# @param {string} URL
#
# Boost generate parameters
# @param {string} [GENERATE_DIR={{.WORK_DIR}}/boost-generate] Directory in which to generate the
# @param {string} [GENERATE_DIR={{.WORK_DIR}}/Boost-generate] Directory in which to generate the
# project build files.
# @param {string} [GENERATE_CHECKSUM_FILE={{.WORK_DIR}}/boost-generate.md5] Checksum file for the
# @param {string} [GENERATE_CHECKSUM_FILE={{.WORK_DIR}}/Boost-generate.md5] Checksum file for the
# generate directory.
# @param {string} [INSTALL_PREFIX={{.WORK_DIR}}/boost-install] Path prefix of where the project
# @param {string} [INSTALL_PREFIX={{.WORK_DIR}}/Boost-install] Path prefix of where the project
# should be installed.
# @param {string[]} [TARGETS] Target libraries to build.
# @param {string[]} [GENERATE_ARGS] Any additional arguments to pass to the generate command.
#
# Boost build-and-install parameters
# @param {string} [BUILD_DIR={{.WORK_DIR}}/boost-build] Directory in which to build the project.
# @param {string} [BUILD_DIR={{.WORK_DIR}}/Boost-build] Directory in which to build the project.
# @param {int} [JOBS] The maximum number of concurrent processes to use when building. If
# omitted, the b2 default number is used. Before 1.76.0, the number was 1. Since 1.76.0, the
# default is the number of cores.
# @param {string[]} [BUILD_AND_INSTALL_ARGS] Any additional arguments to pass to boost's build
# @param {string[]} [BUILD_AND_INSTALL_ARGS] Any additional arguments to pass to Boost's build
# and install command.
# @param {string} [CMAKE_SETTINGS_DIR] If set, the directory where the project's CMake settings
# file should be stored.
download-and-install:
internal: true
label: "{{.TASK}}:{{.URL}}-{{.INSTALL_PREFIX}}"
vars:
# General parameters
BOOST_PACKAGE_NAME: "Boost"
WORK_DIR: >-
{{default .ROOT_DIR .WORK_DIR}}
SOURCE_DIR: >-
{{default (printf "%s/boost-src" .WORK_DIR) .SOURCE_DIR}}
{{default (printf "%s/%s-src" .WORK_DIR .BOOST_PACKAGE_NAME) .SOURCE_DIR}}

# Boost generate parameters
GENERATE_DIR: >-
{{default (printf "%s/boost-generate" .WORK_DIR) .GENERATE_DIR}}
{{default (printf "%s/%s-generate" .WORK_DIR .BOOST_PACKAGE_NAME) .GENERATE_DIR}}
GENERATE_CHECKSUM_FILE: >-
{{default (printf "%s/boost-generate.md5" .WORK_DIR) .GENERATE_CHECKSUM_FILE}}
{{default (printf "%s.md5" .GENERATE_DIR) .GENERATE_CHECKSUM_FILE}}
INSTALL_PREFIX: >-
{{default (printf "%s/boost-install" .WORK_DIR) .INSTALL_PREFIX}}
{{default (printf "%s/%s-install" .WORK_DIR .BOOST_PACKAGE_NAME) .INSTALL_PREFIX}}
TARGETS:
ref: "default (list) .TARGETS"
GENERATE_ARGS:
ref: "default (list) .GENERATE_ARGS"

# Boost build-and-install parameters
BUILD_DIR: >-
{{default (printf "%s/boost-build" .WORK_DIR) .BUILD_DIR}}
{{default (printf "%s/%s-build" .WORK_DIR .BOOST_PACKAGE_NAME) .BUILD_DIR}}
BUILD_AND_INSTALL_ARGS:
ref: "default (list) .BUILD_AND_INSTALL_ARGS"
JOBS: >-
{{default "" .JOBS}}
CMAKE_SETTINGS_DIR: >-
{{default "" .CMAKE_SETTINGS_DIR}}
requires:
vars: ["FILE_SHA256", "URL"]
# TODO: use install directory checksum to properly decide if the task is up-to-date. Right now,
# the task gets re-run for each unique task label.
run: "when_changed"
deps:
- task: "remote:download-and-extract-tar"
vars:
Expand All @@ -187,4 +177,3 @@ tasks:
EXTRA_ARGS:
ref: ".BUILD_AND_INSTALL_ARGS"
JOBS: "{{.JOBS}}"
CMAKE_SETTINGS_DIR: "{{.CMAKE_SETTINGS_DIR}}"
90 changes: 48 additions & 42 deletions exports/taskfiles/utils/cmake.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
version: "3"

includes:
misc: "misc.yaml"
remote: "remote.yaml"

set: ["u", "pipefail"]
shopt: ["globstar"]

tasks:
# Runs the CMake build step for the given build directory. The caller must have previously called
# `generate` on `BUILD_DIR` for this task to succeed. We purposely omit `sources` and `generates`
# as we defer to `cmake` to decide whether it should perform any actions.
# `generate` on `BUILD_DIR` for this task to succeed.
#
# NOTE: We purposely omit `sources` and `generates` as we defer to `cmake` to decide whether it
# should perform any actions.
#
# @param {string} BUILD_DIR Directory containing the generated build system to use.
# @param {string[]} [EXTRA_ARGS] Any additional arguments to pass to the build command.
Expand Down Expand Up @@ -60,9 +63,10 @@ tasks:
ref: ".EXTRA_ARGS"
TARGETS: ["clean"]

# Runs the CMake generate step for the given source and build directories. We purposely omit
# `sources` and `generates` as we defer to `cmake` to decide whether it should perform any
# actions.
# Runs the CMake generate step for the given source and build directories.
#
# NOTE: We purposely omit `sources` and `generates` as we defer to `cmake` to decide whether it
# should perform any actions.
#
# @param {string} BUILD_DIR Directory in which to generate the build system.
# @param {string} SOURCE_DIR Project source directory containing the CMakeLists.txt file.
Expand All @@ -85,38 +89,22 @@ tasks:
{{- end}}

# Runs the CMake install step for the given build directory. The caller must have previously
# called `build` on `BUILD_DIR` for this task to succeed. If `CMAKE_SETTINGS_DIR` is set, a
# settings file will be created in that directory, containing a `{{.CMAKE_PACKAGE_NAME}}_ROOT`
# CMake variable that points to `INSTALL_PREFIX`.
# called `build` on `BUILD_DIR` for this task to succeed.
#
# NOTE: We purposely omit `sources` and `generates` as we defer to `cmake` to decide whether it
# should perform any actions.
#
# @param {string} BUILD_DIR Directory containing the completed build to use.
# @param {string} INSTALL_PREFIX Path prefix of where the project should be installed.
# @param {string} [CMAKE_PACKAGE_NAME] CMake project name (used in directory names and the CMake
# settings file). Required if `CMAKE_SETTINGS_DIR` is set.
# @param {string} [CMAKE_SETTINGS_DIR] If set, the directory where the project's CMake settings
# file should be stored.
# @param {string[]} [EXTRA_ARGS] Any additional arguments to pass to the install command.
install:
internal: true
label: "{{.TASK}}:{{.BUILD_DIR}}-{{.INSTALL_PREFIX}}-{{.EXTRA_ARGS}}"
vars:
CMAKE_SETTINGS_DIR: >-
{{default "" .CMAKE_SETTINGS_DIR}}
EXTRA_ARGS:
ref: "default (list) .EXTRA_ARGS"
requires:
vars: ["BUILD_DIR", "INSTALL_PREFIX"]
preconditions:
- sh: >-
{{if .CMAKE_SETTINGS_DIR}}
{{not (empty .CMAKE_PACKAGE_NAME)}}
{{end}}
msg: |-
CMAKE_PACKAGE_NAME must be set if CMAKE_SETTINGS_DIR is set.
CMAKE_SETTINGS_DIR: {{.CMAKE_SETTINGS_DIR}}
cmds:
- >-
cmake
Expand All @@ -125,18 +113,12 @@ tasks:
{{- range .EXTRA_ARGS}}
{{.}}
{{- end}}
- >-
{{- if .CMAKE_SETTINGS_DIR}}
echo "set({{.CMAKE_PACKAGE_NAME}}_ROOT
\"{{.INSTALL_PREFIX}}\"
CACHE PATH
\"Package root for {{.CMAKE_PACKAGE_NAME}}.\"
)" >> "{{.CMAKE_SETTINGS_DIR}}/{{.CMAKE_PACKAGE_NAME}}.cmake"
{{- end}}

# Downloads a CMake project tar file from `TAR_URL` and then generates, builds, and installs the
# project. We purposely omit `sources` and `generates` as we defer to `cmake` to decide whether it
# should perform any actions. However, the download and extraction will be skipped if unnecessary.
# project.
#
# NOTE: We purposely omit `sources` and `generates` as we defer to `cmake` to decide whether it
# should perform any actions.
#
# Required parameters
# @param {string} CMAKE_PACKAGE_NAME CMake package name.
Expand All @@ -154,8 +136,6 @@ tasks:
# command.
# @param {int} [CMAKE_JOBS] The maximum number of concurrent processes to use when building. If
# omitted, the native build tool's default number is used. See `man cmake`.
# @param {string} [CMAKE_SETTINGS_DIR] The directory where the project's CMake settings file
# should be stored.
# @param {string} [CMAKE_SOURCE_DIR=.] The path, within the extraction directory, containing the
# project's top level CMakeLists.txt.
# @param {string[]} [CMAKE_TARGETS] A list of specific targets to build instead of the default
Expand All @@ -173,8 +153,6 @@ tasks:
ref: "default (list) .CMAKE_INSTALL_ARGS"
CMAKE_JOBS: >-
{{default "" .CMAKE_JOBS}}
CMAKE_SETTINGS_DIR: >-
{{default "" .CMAKE_SETTINGS_DIR}}
CMAKE_SOURCE_DIR: >-
{{default "." .CMAKE_SOURCE_DIR}}
CMAKE_TARGETS:
Expand Down Expand Up @@ -211,12 +189,33 @@ tasks:
- task: "install"
vars:
BUILD_DIR: "{{.BUILD_DIR}}"
CMAKE_PACKAGE_NAME: "{{.CMAKE_PACKAGE_NAME}}"
CMAKE_SETTINGS_DIR: "{{.CMAKE_SETTINGS_DIR}}"
EXTRA_ARGS:
ref: ".CMAKE_INSTALL_ARGS"
INSTALL_PREFIX: "{{.INSTALL_PREFIX}}"

# Create a settings file in CMAKE_SETTINGS_DIR, containing a `{{.CMAKE_PACKAGE_NAME}}_ROOT` CMake
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Create a settings file in CMAKE_SETTINGS_DIR, containing a `{{.CMAKE_PACKAGE_NAME}}_ROOT` CMake
# Creates a settings file in `CMAKE_SETTINGS_DIR`, containing a `{{.CMAKE_PACKAGE_NAME}}_ROOT` CMake

# variable that points to `PACKAGE_INSTALL_PREFIX`.
#
# Required parameters
# @param {string} CMAKE_PACKAGE_NAME CMake package name.
# @param {string} CMAKE_SETTINGS_DIR The directory where the project's CMake settings file should
# be stored.
# @param {string} PACKAGE_INSTALL_PREFIX The location where the project is installed.
generate-cmake-settings:
internal: true
label: "{{.TASK}}:{{.CMAKE_PACKAGE_NAME}}-{{.CMAKE_SETTINGS_DIR}}-{{.PACKAGE_INSTALL_PREFIX}}"
requires:
vars: ["CMAKE_PACKAGE_NAME", "CMAKE_SETTINGS_DIR", "PACKAGE_INSTALL_PREFIX"]
run: "when_changed"
cmds:
- "mkdir -p {{.CMAKE_SETTINGS_DIR}}"
- >-
echo "set({{.CMAKE_PACKAGE_NAME}}_ROOT
\"{{.PACKAGE_INSTALL_PREFIX}}\"
CACHE PATH
\"Package root for {{.CMAKE_PACKAGE_NAME}}.\"
)" >> "{{.CMAKE_SETTINGS_DIR}}/{{.CMAKE_PACKAGE_NAME}}.cmake"

# Sets up all CMake dependencies for a project by:
#
# 1. creating a directory to contain all CMake settings files (CMAKE_SETTINGS_DIR).
Expand All @@ -227,8 +226,7 @@ tasks:
# @param {string} CMAKE_SETTINGS_DIR The directory where CMake settings files should be stored.
# @param {string} DEP_TASK The task to run that will install all dependencies. NOTE:
# - The task name must be qualified from the root of the project.
# - The task must not require any arguments (to use a task with arguments create a new task that
# calls the original with any arguments set).
Comment on lines -230 to -231
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this is a little misleading. We should say that the task must take two arguments CMAKE_SETTINGS_DIRS and CMAKE_SETTINGS_FILE, and cannot have other required arguments.

# - The task takes in CMAKE_SETTINGS_DIR and CMAKE_SETTINGS_FILE as optional arguments.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From previous part of the docstring and the task's requires, it looks like CMAKE_SETTINGS_DIR is not optional.

# - Dependencies must write their settings file to CMAKE_SETTINGS_DIR in order to have them
# included in CMAKE_SETTINGS_FILE.
# @param {string} [CMAKE_SETTINGS_FILE={{.CMAKE_SETTINGS_DIR}}/all.cmake] The file in which to
Expand All @@ -242,12 +240,20 @@ tasks:
requires:
vars: ["CMAKE_SETTINGS_DIR", "DEP_TASK"]
cmds:
- "rm -rf {{.CMAKE_SETTINGS_DIR}}"
- "mkdir -p {{.CMAKE_SETTINGS_DIR}}"
- task: "misc:reset-dir"
vars:
DIR_PATH: "{{.CMAKE_SETTINGS_DIR}}"
- task: "misc:reset-file"
vars:
FILE_PATH: "{{.CMAKE_SETTINGS_FILE}}"

# NOTE: We prefix DEP_TASK with `::` assuming that this taskfile is included through the
# `utils` taskfile, and that in turn is included in the user's taskfile.
- task: "::{{.DEP_TASK}}"
vars:
CMAKE_SETTINGS_DIR: "{{.CMAKE_SETTINGS_DIR}}"
CMAKE_SETTINGS_FILE: "{{.CMAKE_SETTINGS_FILE}}"

- |-
for file in {{.CMAKE_SETTINGS_DIR}}/*.cmake; do
if [[ "$file" != "{{.CMAKE_SETTINGS_FILE}}" ]]; then
Expand Down
22 changes: 22 additions & 0 deletions exports/taskfiles/utils/misc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,25 @@ tasks:
. "{{.OUTPUT_DIR}}/bin/activate"
pip3 install --upgrade pip
pip3 install --upgrade -r "{{.REQUIREMENTS_FILE}}"

reset-dir:
desc: "Removes the given directory and create it empty. Runs once per directory path."
internal: true
label: "{{.TASK}}:{{.DIR_PATH}}"
requires:
vars: ["DIR_PATH"]
run: "when_changed"
cmds:
- "rm -rf {{.DIR_PATH}}"
- "mkdir -p {{.DIR_PATH}}"

reset-file:
desc: "Removes the given file. Runs once per file path."
internal: true
label: "{{.TASK}}:{{.FILE_PATH}}"
requires:
vars: ["FILE_PATH"]
run: "when_changed"
cmds:
- "rm -rf {{.FILE_PATH}}"
- "mkdir -p $(dirname -- {{.FILE_PATH}})"
Comment on lines +45 to +65
Copy link
Contributor

@coderabbitai coderabbitai bot Oct 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Quote the reset commands to avoid clobbering the wrong paths

Both reset-dir and reset-file call rm -rf/mkdir -p without quoting the expanded variables. If the caller passes a path containing spaces or glob tokens, the shell will split the argument and you could delete or recreate unintended locations. Please add the usual quoting around the templated paths and around the dirname call.

-      - "rm -rf {{.DIR_PATH}}"
-      - "mkdir -p {{.DIR_PATH}}"
+      - 'rm -rf "{{.DIR_PATH}}"'
+      - 'mkdir -p "{{.DIR_PATH}}"'
...
-      - "rm -rf {{.FILE_PATH}}"
-      - "mkdir -p $(dirname -- {{.FILE_PATH}})"
+      - 'rm -rf "{{.FILE_PATH}}"'
+      - 'mkdir -p "$(dirname -- "{{.FILE_PATH}}")"'

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In exports/taskfiles/utils/misc.yaml around lines 45 to 65 the rm -rf and mkdir
-p commands use unquoted templated paths which allows word-splitting and glob
expansion (dangerous for paths with spaces or glob chars); fix by quoting the
templated path variables in both commands (e.g. quote {{.DIR_PATH}} and
{{.FILE_PATH}}) and also quote the dirname expression used for the file case,
and include the standard "--" sentinel to dirname to safely handle names
beginning with "-" so the shell receives a single, quoted path argument.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Bill-hbrhbr This makes sense to me. Could you fix this by applying what create-venv is doing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!