summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYifan Hong <elsk@google.com>2024-03-13 15:55:39 -0700
committerYifan Hong <elsk@google.com>2024-03-20 10:45:40 -0700
commitccfb5394430125d80c3cb66d4fe1fe4c0037a41f (patch)
treea33735ed94cc9d1863967615d5c076c93dba1630
parentbb0e3a313cccec31639dc7efacc16058c5ce0286 (diff)
downloadbuild-ccfb5394430125d80c3cb66d4fe1fe4c0037a41f.tar.gz
kleaf: kernel_filegroup impl ddk_config_env
This resolves the following error in DDKv2 when we build a ddk_module against Kleaf tooling in the @kleaf module: Traceback (most recent call last): File "<...>/build/kernel/kleaf/impl/ddk/ddk_config.bzl", line 33, column 24, in _ddk_config_impl _create_main_action( File "<...>/build/kernel/kleaf/impl/ddk/ddk_config.bzl", line 151, column 69, in _create_main_action ddk_config_env = ctx.attr.kernel_build[KernelBuildExtModuleInfo].ddk_config_env Error: 'KernelBuildExtModuleInfo' value has no field or method 'ddk_config_env' Bug: 291918087 Change-Id: I2202a518173a9a3d77b9427f6f64f9f0b803db00
-rw-r--r--kleaf/docs/api_reference/kernel.md12
-rw-r--r--kleaf/impl/common_providers.bzl5
-rw-r--r--kleaf/impl/kernel_build.bzl4
-rw-r--r--kleaf/impl/kernel_config.bzl54
-rw-r--r--kleaf/impl/kernel_env.bzl26
-rw-r--r--kleaf/impl/kernel_filegroup.bzl71
-rw-r--r--kleaf/impl/kernel_filegroup_declaration.bzl7
7 files changed, 145 insertions, 34 deletions
diff --git a/kleaf/docs/api_reference/kernel.md b/kleaf/docs/api_reference/kernel.md
index a2410e7..141751a 100644
--- a/kleaf/docs/api_reference/kernel.md
+++ b/kleaf/docs/api_reference/kernel.md
@@ -258,10 +258,11 @@ Define an executable that creates `compile_commands.json` from a `kernel_build`.
## kernel_filegroup
<pre>
-kernel_filegroup(<a href="#kernel_filegroup-name">name</a>, <a href="#kernel_filegroup-deps">deps</a>, <a href="#kernel_filegroup-srcs">srcs</a>, <a href="#kernel_filegroup-collect_unstripped_modules">collect_unstripped_modules</a>, <a href="#kernel_filegroup-ddk_module_defconfig_fragments">ddk_module_defconfig_fragments</a>,
- <a href="#kernel_filegroup-debug">debug</a>, <a href="#kernel_filegroup-exec_platform">exec_platform</a>, <a href="#kernel_filegroup-gki_artifacts">gki_artifacts</a>, <a href="#kernel_filegroup-images">images</a>, <a href="#kernel_filegroup-kasan">kasan</a>, <a href="#kernel_filegroup-kasan_generic">kasan_generic</a>, <a href="#kernel_filegroup-kasan_sw_tags">kasan_sw_tags</a>,
- <a href="#kernel_filegroup-kcsan">kcsan</a>, <a href="#kernel_filegroup-kernel_release">kernel_release</a>, <a href="#kernel_filegroup-kernel_uapi_headers">kernel_uapi_headers</a>, <a href="#kernel_filegroup-lto">lto</a>, <a href="#kernel_filegroup-module_outs_file">module_outs_file</a>,
- <a href="#kernel_filegroup-protected_modules_list">protected_modules_list</a>, <a href="#kernel_filegroup-target_platform">target_platform</a>, <a href="#kernel_filegroup-trim_nonlisted_kmi">trim_nonlisted_kmi</a>)
+kernel_filegroup(<a href="#kernel_filegroup-name">name</a>, <a href="#kernel_filegroup-deps">deps</a>, <a href="#kernel_filegroup-srcs">srcs</a>, <a href="#kernel_filegroup-collect_unstripped_modules">collect_unstripped_modules</a>, <a href="#kernel_filegroup-config_out_dir">config_out_dir</a>, <a href="#kernel_filegroup-config_out_dir_files">config_out_dir_files</a>,
+ <a href="#kernel_filegroup-ddk_module_defconfig_fragments">ddk_module_defconfig_fragments</a>, <a href="#kernel_filegroup-debug">debug</a>, <a href="#kernel_filegroup-env_setup_script">env_setup_script</a>, <a href="#kernel_filegroup-exec_platform">exec_platform</a>,
+ <a href="#kernel_filegroup-gki_artifacts">gki_artifacts</a>, <a href="#kernel_filegroup-images">images</a>, <a href="#kernel_filegroup-kasan">kasan</a>, <a href="#kernel_filegroup-kasan_generic">kasan_generic</a>, <a href="#kernel_filegroup-kasan_sw_tags">kasan_sw_tags</a>, <a href="#kernel_filegroup-kcsan">kcsan</a>, <a href="#kernel_filegroup-kernel_release">kernel_release</a>,
+ <a href="#kernel_filegroup-kernel_uapi_headers">kernel_uapi_headers</a>, <a href="#kernel_filegroup-lto">lto</a>, <a href="#kernel_filegroup-module_outs_file">module_outs_file</a>, <a href="#kernel_filegroup-protected_modules_list">protected_modules_list</a>, <a href="#kernel_filegroup-target_platform">target_platform</a>,
+ <a href="#kernel_filegroup-trim_nonlisted_kmi">trim_nonlisted_kmi</a>)
</pre>
**EXPERIMENTAL.** The API of `kernel_filegroup` rapidly changes and
@@ -286,8 +287,11 @@ It can be used in the `base_kernel` attribute of a [`kernel_build`](#kernel_buil
| <a id="kernel_filegroup-deps"></a>deps | A list of additional labels that participates in implementing the providers.<br><br>This usually contains a list of prebuilts.<br><br>Unlike srcs, these labels are NOT added to the [`DefaultInfo`](https://docs.bazel.build/versions/main/skylark/lib/DefaultInfo.html) | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="kernel_filegroup-srcs"></a>srcs | The list of labels that are members of this file group.<br><br>This usually contains a list of prebuilts, e.g. `vmlinux`, `Image.lz4`, `kernel-headers.tar.gz`, etc.<br><br>Not to be confused with [`kernel_srcs`](#kernel_filegroup-kernel_srcs). | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="kernel_filegroup-collect_unstripped_modules"></a>collect_unstripped_modules | See [`kernel_build.collect_unstripped_modules`](#kernel_build-collect_unstripped_modules).<br><br>Unlike `kernel_build`, this has default value `True` because [`kernel_abi`](#kernel_abi) sets [`define_abi_targets`](#kernel_abi-define_abi_targets) to `True` by default, which in turn sets `collect_unstripped_modules` to `True` by default. | Boolean | optional | `True` |
+| <a id="kernel_filegroup-config_out_dir"></a>config_out_dir | Directory to support `kernel_config` | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
+| <a id="kernel_filegroup-config_out_dir_files"></a>config_out_dir_files | Files in `config_out_dir` | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="kernel_filegroup-ddk_module_defconfig_fragments"></a>ddk_module_defconfig_fragments | Additional defconfig fragments for dependant DDK modules. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="kernel_filegroup-debug"></a>debug | - | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `"@kleaf//build/kernel/kleaf:debug"` |
+| <a id="kernel_filegroup-env_setup_script"></a>env_setup_script | Setup script from `kernel_env` | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="kernel_filegroup-exec_platform"></a>exec_platform | Execution platform, where the build is executed.<br><br>See https://bazel.build/extending/platforms. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="kernel_filegroup-gki_artifacts"></a>gki_artifacts | A list of files that were built from the [`gki_artifacts`](#gki_artifacts) target. The `gki-info.txt` file should be part of that list.<br><br>If `kernel_release` is set, this attribute has no effect. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="kernel_filegroup-images"></a>images | A label providing files similar to a [`kernel_images`](#kernel_images) target. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
diff --git a/kleaf/impl/common_providers.bzl b/kleaf/impl/common_providers.bzl
index a47f77c..22694b2 100644
--- a/kleaf/impl/common_providers.bzl
+++ b/kleaf/impl/common_providers.bzl
@@ -307,6 +307,11 @@ KernelBuildFilegroupDeclInfo = provider(
[`File`](https://bazel.build/rules/lib/File)s containing
archives of UAPI headers.""",
"arch": "[`kernel_build.arch`](#kernel_build-arch)",
+ "env_setup_script": """A [depset](https://bazel.build/extending/depsets) of
+ [`File`](https://bazel.build/rules/lib/File)s to replay the `kernel_config` environment.
+
+ See [`KernelConfigInfo`](#KernelConfigInfo).""",
+ "config_out_dir": """The output directory of `kernel_config`.""",
},
)
diff --git a/kleaf/impl/kernel_build.bzl b/kleaf/impl/kernel_build.bzl
index 90a4576..013cbd5 100644
--- a/kleaf/impl/kernel_build.bzl
+++ b/kleaf/impl/kernel_build.bzl
@@ -42,6 +42,7 @@ load(
"KernelBuildUapiInfo",
"KernelBuildUnameInfo",
"KernelCmdsInfo",
+ "KernelConfigInfo",
"KernelEnvAndOutputsInfo",
"KernelEnvAttrInfo",
"KernelEnvMakeGoalsInfo",
@@ -1993,6 +1994,8 @@ def _create_infos(
ddk_module_defconfig_fragments = ddk_module_defconfig_fragments,
kernel_uapi_headers = kernel_uapi_headers_depset,
arch = ctx.attr.arch,
+ env_setup_script = ctx.attr.config[KernelConfigInfo].env_setup_script,
+ config_out_dir = ctx.file.config,
)
default_info_files = all_output_files["outs"].values() + all_output_files["module_outs"].values()
@@ -2122,6 +2125,7 @@ _kernel_build = rule(
KernelToolchainInfo,
],
doc = "the kernel_config target",
+ allow_single_file = True,
),
"keep_module_symvers": attr.bool(
doc = "If true, a copy of `Module.symvers` is kept, with the name `{name}_Module.symvers`",
diff --git a/kleaf/impl/kernel_config.bzl b/kleaf/impl/kernel_config.bzl
index b8e0b3f..14fae04 100644
--- a/kleaf/impl/kernel_config.bzl
+++ b/kleaf/impl/kernel_config.bzl
@@ -456,20 +456,6 @@ def _kernel_config_impl(ctx):
)
post_setup_deps = [out_dir]
- post_setup = """
- [ -z ${{OUT_DIR}} ] && echo "FATAL: configs post_env_info setup run without OUT_DIR set!" >&2 && exit 1
- # Restore kernel config inputs
- mkdir -p ${{OUT_DIR}}/include/
- rsync -aL {out_dir}/.config ${{OUT_DIR}}/.config
- rsync -aL --chmod=D+w {out_dir}/include/ ${{OUT_DIR}}/include/
- rsync -aL --chmod=F+w {out_dir}/localversion ${{OUT_DIR}}/localversion
-
- # Restore real value of $ROOT_DIR in auto.conf.cmd
- sed -i'' -e 's:${{ROOT_DIR}}:'"${{ROOT_DIR}}"':g' ${{OUT_DIR}}/include/config/auto.conf.cmd
- """.format(
- out_dir = out_dir.path,
- )
-
if trim_nonlisted_kmi_utils.get_value(ctx):
# Ensure the dependent action uses the up-to-date abi_symbollist.raw
# at the absolute path specified in abi_symbollist.raw.abspath
@@ -479,14 +465,9 @@ def _kernel_config_impl(ctx):
serialized_env_info_setup_script = ctx.actions.declare_file("{name}/{name}_setup.sh".format(name = ctx.attr.name))
ctx.actions.write(
output = serialized_env_info_setup_script,
- content = """
- {pre_setup}
- {eval_restore_out_dir_cmd}
- {post_setup}
- """.format(
- pre_setup = ctx.attr.env[KernelEnvInfo].setup,
- eval_restore_out_dir_cmd = kernel_utils.eval_restore_out_dir_cmd(),
- post_setup = post_setup,
+ content = get_config_setup_command(
+ env_setup_command = ctx.attr.env[KernelEnvInfo].setup,
+ out_dir = out_dir,
),
)
@@ -574,6 +555,35 @@ def _get_config_script(ctx, inputs):
runfiles = runfiles,
)
+def get_config_setup_command(
+ env_setup_command,
+ out_dir):
+ """Returns the content of `<kernel_build>_config_setup.sh`, given the parameters.
+
+ Args:
+ env_setup_command: command to set up environment from `kernel_env`
+ out_dir: output directory from `kernel_config`
+ """
+
+ return """
+ {env_setup_command}
+ {eval_restore_out_dir_cmd}
+
+ [ -z ${{OUT_DIR}} ] && echo "FATAL: configs post_env_info setup run without OUT_DIR set!" >&2 && exit 1
+ # Restore kernel config inputs
+ mkdir -p ${{OUT_DIR}}/include/
+ rsync -aL {out_dir}/.config ${{OUT_DIR}}/.config
+ rsync -aL --chmod=D+w {out_dir}/include/ ${{OUT_DIR}}/include/
+ rsync -aL --chmod=F+w {out_dir}/localversion ${{OUT_DIR}}/localversion
+
+ # Restore real value of $ROOT_DIR in auto.conf.cmd
+ sed -i'' -e 's:${{ROOT_DIR}}:'"${{ROOT_DIR}}"':g' ${{OUT_DIR}}/include/config/auto.conf.cmd
+ """.format(
+ env_setup_command = env_setup_command,
+ eval_restore_out_dir_cmd = kernel_utils.eval_restore_out_dir_cmd(),
+ out_dir = out_dir.path,
+ )
+
def _kernel_config_additional_attrs():
return dicts.add(
kernel_config_settings.of_kernel_config(),
diff --git a/kleaf/impl/kernel_env.bzl b/kleaf/impl/kernel_env.bzl
index 2011a6e..7163c3d 100644
--- a/kleaf/impl/kernel_env.bzl
+++ b/kleaf/impl/kernel_env.bzl
@@ -286,14 +286,10 @@ def _kernel_env_impl(ctx):
command = command,
)
- setup = hermetic_tools.setup
- setup += """
- source {build_utils_sh}
- # source the build environment
- source {env}
- """.format(
- build_utils_sh = ctx.file._build_utils_sh.path,
- env = out_file.path,
+ setup = get_env_info_setup_command(
+ hermetic_tools_setup = hermetic_tools.setup,
+ build_utils_sh = ctx.file._build_utils_sh,
+ env_setup_script = out_file,
)
setup_tools = [
@@ -337,6 +333,20 @@ def _kernel_env_impl(ctx):
DefaultInfo(files = depset([out_file])),
]
+def get_env_info_setup_command(hermetic_tools_setup, build_utils_sh, env_setup_script):
+ """Returns text for KernelEnvInfo.setup"""
+
+ return """
+ {hermetic_tools_setup}
+ source {build_utils_sh}
+ # source the build environment
+ source {env_setup_script}
+ """.format(
+ hermetic_tools_setup = hermetic_tools_setup,
+ build_utils_sh = build_utils_sh.path,
+ env_setup_script = env_setup_script.path,
+ )
+
def _get_env_setup_cmds(ctx):
pre_env = ""
if ctx.attr._debug_annotate_scripts[BuildSettingInfo].value:
diff --git a/kleaf/impl/kernel_filegroup.bzl b/kleaf/impl/kernel_filegroup.bzl
index bc6dd10..7ceb899 100644
--- a/kleaf/impl/kernel_filegroup.bzl
+++ b/kleaf/impl/kernel_filegroup.bzl
@@ -26,6 +26,7 @@ load(
"KernelBuildUnameInfo",
"KernelEnvAttrInfo",
"KernelImagesInfo",
+ "KernelSerializedEnvInfo",
"KernelToolchainInfo",
"KernelUnstrippedModulesInfo",
)
@@ -37,7 +38,9 @@ load(
)
load(":debug.bzl", "debug")
load(":hermetic_toolchain.bzl", "hermetic_toolchain")
+load(":kernel_config.bzl", "get_config_setup_command")
load(":kernel_config_settings.bzl", "kernel_config_settings")
+load(":kernel_env.bzl", "get_env_info_setup_command")
load(":kernel_toolchains_utils.bzl", "kernel_toolchains_utils")
load(
":utils.bzl",
@@ -94,17 +97,68 @@ def _get_kernel_release(ctx):
)
return kernel_release
+def _get_ddk_config_env(ctx):
+ """Returns `KernelBuildExtModuleInfo.ddk_config_env`."""
+
+ if not ctx.file.config_out_dir or not ctx.file.env_setup_script:
+ return None
+
+ hermetic_tools = hermetic_toolchain.get(ctx)
+ toolchains = kernel_toolchains_utils.get(ctx)
+
+ env_setup_command = """
+ KLEAF_REPO_WORKSPACE_ROOT={kleaf_repo_workspace_root}
+ """.format(
+ kleaf_repo_workspace_root = Label(":kernel_filegroup.bzl").workspace_root,
+ )
+ env_setup_command += get_env_info_setup_command(
+ hermetic_tools_setup = hermetic_tools.setup,
+ build_utils_sh = ctx.file._build_utils_sh,
+ env_setup_script = ctx.file.env_setup_script,
+ )
+
+ ddk_config_env_setup_command = get_config_setup_command(
+ env_setup_command = env_setup_command,
+ out_dir = ctx.file.config_out_dir,
+ )
+
+ ddk_config_env_setup_script = ctx.actions.declare_file(
+ "{name}/{name}_ddk_config_setup.sh".format(name = ctx.attr.name),
+ )
+ ctx.actions.write(
+ output = ddk_config_env_setup_script,
+ content = ddk_config_env_setup_command,
+ )
+ ddk_config_env = KernelSerializedEnvInfo(
+ setup_script = ddk_config_env_setup_script,
+ inputs = depset([
+ ddk_config_env_setup_script,
+ ctx.file.env_setup_script,
+ ctx.version_file,
+ ], transitive = [target.files for target in ctx.attr.config_out_dir_files]),
+ tools = depset([
+ ctx.file._build_utils_sh,
+ ], transitive = [
+ hermetic_tools.deps,
+ toolchains.all_files,
+ ]),
+ )
+ return ddk_config_env
+
def _kernel_filegroup_impl(ctx):
hermetic_tools = hermetic_toolchain.get(ctx)
all_deps = ctx.files.srcs + ctx.files.deps
+ ddk_config_env = _get_ddk_config_env(ctx)
+
# TODO(b/219112010): Implement KernelSerializedEnvInfo properly
kernel_module_dev_info = KernelBuildExtModuleInfo(
modules_staging_archive = utils.find_file(MODULES_STAGING_ARCHIVE, all_deps, what = ctx.label),
# TODO(b/211515836): module_scripts might also be downloaded
# Building kernel_module (excluding ddk_module) on top of kernel_filegroup is unsupported.
# module_hdrs = None,
+ ddk_config_env = ddk_config_env,
collect_unstripped_modules = ctx.attr.collect_unstripped_modules,
ddk_module_defconfig_fragments = depset(transitive = [
target.files
@@ -309,12 +363,29 @@ default, which in turn sets `collect_unstripped_modules` to `True` by default.
allow_single_file = True,
doc = "A file providing the kernel release string. This is preferred over `gki_artifacts`.",
),
+ "config_out_dir_files": attr.label_list(
+ doc = "Files in `config_out_dir`",
+ allow_files = True,
+ ),
+ "config_out_dir": attr.label(
+ allow_single_file = True,
+ doc = "Directory to support `kernel_config`",
+ ),
+ "env_setup_script": attr.label(
+ allow_single_file = True,
+ doc = "Setup script from `kernel_env`",
+ ),
"_debug_print_scripts": attr.label(default = "//build/kernel/kleaf:debug_print_scripts"),
"_cache_dir_config_tags": attr.label(
default = "//build/kernel/kleaf/impl:cache_dir_config_tags",
executable = True,
cfg = "exec",
),
+ "_build_utils_sh": attr.label(
+ allow_single_file = True,
+ default = Label("//build/kernel:build_utils"),
+ cfg = "exec",
+ ),
"ddk_module_defconfig_fragments": attr.label_list(
doc = "Additional defconfig fragments for dependant DDK modules.",
allow_empty = True,
diff --git a/kleaf/impl/kernel_filegroup_declaration.bzl b/kleaf/impl/kernel_filegroup_declaration.bzl
index 72080e0..e29c4aa 100644
--- a/kleaf/impl/kernel_filegroup_declaration.bzl
+++ b/kleaf/impl/kernel_filegroup_declaration.bzl
@@ -95,6 +95,9 @@ kernel_filegroup(
kernel_release = {kernel_release_repr},
protected_modules_list = {protected_modules_repr},
ddk_module_defconfig_fragments = {ddk_module_defconfig_fragments_repr},
+ config_out_dir_files = glob([{config_out_dir_repr} + "/**"]),
+ config_out_dir = {config_out_dir_repr},
+ env_setup_script = {env_setup_script_repr},
target_platform = {target_platform_repr},
exec_platform = {exec_platform_repr},
visibility = ["//visibility:public"],
@@ -150,6 +153,8 @@ def _write_filegroup_decl_file(ctx, info, deps_files, kernel_uapi_headers, templ
info.ddk_module_defconfig_fragments,
**(join | pkg)
)
+ sub.add_joined("{config_out_dir_repr}", depset([info.config_out_dir]), **(one | pkg))
+ sub.add_joined("{env_setup_script_repr}", depset([info.env_setup_script]), **(one | pkg))
sub.add("{target_platform_repr}", repr(ctx.attr.kernel_build.label.name + "_platform_target"))
sub.add("{exec_platform_repr}", repr(ctx.attr.kernel_build.label.name + "_platform_exec"))
sub.add("{arch}", info.arch)
@@ -177,6 +182,8 @@ def _create_archive(ctx, info, deps_files, kernel_uapi_headers, filegroup_decl_f
info.module_outs_file,
info.kernel_release,
kernel_uapi_headers,
+ info.config_out_dir,
+ info.env_setup_script,
]
if info.src_protected_modules_list:
direct_inputs.append(info.src_protected_modules_list)