diff options
author | Yifan Hong <elsk@google.com> | 2024-03-13 15:55:39 -0700 |
---|---|---|
committer | Yifan Hong <elsk@google.com> | 2024-03-20 10:45:40 -0700 |
commit | ccfb5394430125d80c3cb66d4fe1fe4c0037a41f (patch) | |
tree | a33735ed94cc9d1863967615d5c076c93dba1630 | |
parent | bb0e3a313cccec31639dc7efacc16058c5ce0286 (diff) | |
download | build-ccfb5394430125d80c3cb66d4fe1fe4c0037a41f.tar.gz |
kleaf: kernel_filegroup impl ddk_config_env
This resolves the following error in DDKv2 when
we build a ddk_module against Kleaf tooling in
the @kleaf module:
Traceback (most recent call last):
File "<...>/build/kernel/kleaf/impl/ddk/ddk_config.bzl", line 33, column 24, in _ddk_config_impl
_create_main_action(
File "<...>/build/kernel/kleaf/impl/ddk/ddk_config.bzl", line 151, column 69, in _create_main_action
ddk_config_env = ctx.attr.kernel_build[KernelBuildExtModuleInfo].ddk_config_env
Error: 'KernelBuildExtModuleInfo' value has no field or method 'ddk_config_env'
Bug: 291918087
Change-Id: I2202a518173a9a3d77b9427f6f64f9f0b803db00
-rw-r--r-- | kleaf/docs/api_reference/kernel.md | 12 | ||||
-rw-r--r-- | kleaf/impl/common_providers.bzl | 5 | ||||
-rw-r--r-- | kleaf/impl/kernel_build.bzl | 4 | ||||
-rw-r--r-- | kleaf/impl/kernel_config.bzl | 54 | ||||
-rw-r--r-- | kleaf/impl/kernel_env.bzl | 26 | ||||
-rw-r--r-- | kleaf/impl/kernel_filegroup.bzl | 71 | ||||
-rw-r--r-- | kleaf/impl/kernel_filegroup_declaration.bzl | 7 |
7 files changed, 145 insertions, 34 deletions
diff --git a/kleaf/docs/api_reference/kernel.md b/kleaf/docs/api_reference/kernel.md index a2410e7..141751a 100644 --- a/kleaf/docs/api_reference/kernel.md +++ b/kleaf/docs/api_reference/kernel.md @@ -258,10 +258,11 @@ Define an executable that creates `compile_commands.json` from a `kernel_build`. ## kernel_filegroup <pre> -kernel_filegroup(<a href="#kernel_filegroup-name">name</a>, <a href="#kernel_filegroup-deps">deps</a>, <a href="#kernel_filegroup-srcs">srcs</a>, <a href="#kernel_filegroup-collect_unstripped_modules">collect_unstripped_modules</a>, <a href="#kernel_filegroup-ddk_module_defconfig_fragments">ddk_module_defconfig_fragments</a>, - <a href="#kernel_filegroup-debug">debug</a>, <a href="#kernel_filegroup-exec_platform">exec_platform</a>, <a href="#kernel_filegroup-gki_artifacts">gki_artifacts</a>, <a href="#kernel_filegroup-images">images</a>, <a href="#kernel_filegroup-kasan">kasan</a>, <a href="#kernel_filegroup-kasan_generic">kasan_generic</a>, <a href="#kernel_filegroup-kasan_sw_tags">kasan_sw_tags</a>, - <a href="#kernel_filegroup-kcsan">kcsan</a>, <a href="#kernel_filegroup-kernel_release">kernel_release</a>, <a href="#kernel_filegroup-kernel_uapi_headers">kernel_uapi_headers</a>, <a href="#kernel_filegroup-lto">lto</a>, <a href="#kernel_filegroup-module_outs_file">module_outs_file</a>, - <a href="#kernel_filegroup-protected_modules_list">protected_modules_list</a>, <a href="#kernel_filegroup-target_platform">target_platform</a>, <a href="#kernel_filegroup-trim_nonlisted_kmi">trim_nonlisted_kmi</a>) +kernel_filegroup(<a href="#kernel_filegroup-name">name</a>, <a href="#kernel_filegroup-deps">deps</a>, <a href="#kernel_filegroup-srcs">srcs</a>, <a href="#kernel_filegroup-collect_unstripped_modules">collect_unstripped_modules</a>, <a href="#kernel_filegroup-config_out_dir">config_out_dir</a>, <a href="#kernel_filegroup-config_out_dir_files">config_out_dir_files</a>, + <a href="#kernel_filegroup-ddk_module_defconfig_fragments">ddk_module_defconfig_fragments</a>, <a href="#kernel_filegroup-debug">debug</a>, <a href="#kernel_filegroup-env_setup_script">env_setup_script</a>, <a href="#kernel_filegroup-exec_platform">exec_platform</a>, + <a href="#kernel_filegroup-gki_artifacts">gki_artifacts</a>, <a href="#kernel_filegroup-images">images</a>, <a href="#kernel_filegroup-kasan">kasan</a>, <a href="#kernel_filegroup-kasan_generic">kasan_generic</a>, <a href="#kernel_filegroup-kasan_sw_tags">kasan_sw_tags</a>, <a href="#kernel_filegroup-kcsan">kcsan</a>, <a href="#kernel_filegroup-kernel_release">kernel_release</a>, + <a href="#kernel_filegroup-kernel_uapi_headers">kernel_uapi_headers</a>, <a href="#kernel_filegroup-lto">lto</a>, <a href="#kernel_filegroup-module_outs_file">module_outs_file</a>, <a href="#kernel_filegroup-protected_modules_list">protected_modules_list</a>, <a href="#kernel_filegroup-target_platform">target_platform</a>, + <a href="#kernel_filegroup-trim_nonlisted_kmi">trim_nonlisted_kmi</a>) </pre> **EXPERIMENTAL.** The API of `kernel_filegroup` rapidly changes and @@ -286,8 +287,11 @@ It can be used in the `base_kernel` attribute of a [`kernel_build`](#kernel_buil | <a id="kernel_filegroup-deps"></a>deps | A list of additional labels that participates in implementing the providers.<br><br>This usually contains a list of prebuilts.<br><br>Unlike srcs, these labels are NOT added to the [`DefaultInfo`](https://docs.bazel.build/versions/main/skylark/lib/DefaultInfo.html) | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` | | <a id="kernel_filegroup-srcs"></a>srcs | The list of labels that are members of this file group.<br><br>This usually contains a list of prebuilts, e.g. `vmlinux`, `Image.lz4`, `kernel-headers.tar.gz`, etc.<br><br>Not to be confused with [`kernel_srcs`](#kernel_filegroup-kernel_srcs). | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` | | <a id="kernel_filegroup-collect_unstripped_modules"></a>collect_unstripped_modules | See [`kernel_build.collect_unstripped_modules`](#kernel_build-collect_unstripped_modules).<br><br>Unlike `kernel_build`, this has default value `True` because [`kernel_abi`](#kernel_abi) sets [`define_abi_targets`](#kernel_abi-define_abi_targets) to `True` by default, which in turn sets `collect_unstripped_modules` to `True` by default. | Boolean | optional | `True` | +| <a id="kernel_filegroup-config_out_dir"></a>config_out_dir | Directory to support `kernel_config` | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` | +| <a id="kernel_filegroup-config_out_dir_files"></a>config_out_dir_files | Files in `config_out_dir` | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` | | <a id="kernel_filegroup-ddk_module_defconfig_fragments"></a>ddk_module_defconfig_fragments | Additional defconfig fragments for dependant DDK modules. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` | | <a id="kernel_filegroup-debug"></a>debug | - | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `"@kleaf//build/kernel/kleaf:debug"` | +| <a id="kernel_filegroup-env_setup_script"></a>env_setup_script | Setup script from `kernel_env` | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` | | <a id="kernel_filegroup-exec_platform"></a>exec_platform | Execution platform, where the build is executed.<br><br>See https://bazel.build/extending/platforms. | <a href="https://bazel.build/concepts/labels">Label</a> | required | | | <a id="kernel_filegroup-gki_artifacts"></a>gki_artifacts | A list of files that were built from the [`gki_artifacts`](#gki_artifacts) target. The `gki-info.txt` file should be part of that list.<br><br>If `kernel_release` is set, this attribute has no effect. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` | | <a id="kernel_filegroup-images"></a>images | A label providing files similar to a [`kernel_images`](#kernel_images) target. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` | diff --git a/kleaf/impl/common_providers.bzl b/kleaf/impl/common_providers.bzl index a47f77c..22694b2 100644 --- a/kleaf/impl/common_providers.bzl +++ b/kleaf/impl/common_providers.bzl @@ -307,6 +307,11 @@ KernelBuildFilegroupDeclInfo = provider( [`File`](https://bazel.build/rules/lib/File)s containing archives of UAPI headers.""", "arch": "[`kernel_build.arch`](#kernel_build-arch)", + "env_setup_script": """A [depset](https://bazel.build/extending/depsets) of + [`File`](https://bazel.build/rules/lib/File)s to replay the `kernel_config` environment. + + See [`KernelConfigInfo`](#KernelConfigInfo).""", + "config_out_dir": """The output directory of `kernel_config`.""", }, ) diff --git a/kleaf/impl/kernel_build.bzl b/kleaf/impl/kernel_build.bzl index 90a4576..013cbd5 100644 --- a/kleaf/impl/kernel_build.bzl +++ b/kleaf/impl/kernel_build.bzl @@ -42,6 +42,7 @@ load( "KernelBuildUapiInfo", "KernelBuildUnameInfo", "KernelCmdsInfo", + "KernelConfigInfo", "KernelEnvAndOutputsInfo", "KernelEnvAttrInfo", "KernelEnvMakeGoalsInfo", @@ -1993,6 +1994,8 @@ def _create_infos( ddk_module_defconfig_fragments = ddk_module_defconfig_fragments, kernel_uapi_headers = kernel_uapi_headers_depset, arch = ctx.attr.arch, + env_setup_script = ctx.attr.config[KernelConfigInfo].env_setup_script, + config_out_dir = ctx.file.config, ) default_info_files = all_output_files["outs"].values() + all_output_files["module_outs"].values() @@ -2122,6 +2125,7 @@ _kernel_build = rule( KernelToolchainInfo, ], doc = "the kernel_config target", + allow_single_file = True, ), "keep_module_symvers": attr.bool( doc = "If true, a copy of `Module.symvers` is kept, with the name `{name}_Module.symvers`", diff --git a/kleaf/impl/kernel_config.bzl b/kleaf/impl/kernel_config.bzl index b8e0b3f..14fae04 100644 --- a/kleaf/impl/kernel_config.bzl +++ b/kleaf/impl/kernel_config.bzl @@ -456,20 +456,6 @@ def _kernel_config_impl(ctx): ) post_setup_deps = [out_dir] - post_setup = """ - [ -z ${{OUT_DIR}} ] && echo "FATAL: configs post_env_info setup run without OUT_DIR set!" >&2 && exit 1 - # Restore kernel config inputs - mkdir -p ${{OUT_DIR}}/include/ - rsync -aL {out_dir}/.config ${{OUT_DIR}}/.config - rsync -aL --chmod=D+w {out_dir}/include/ ${{OUT_DIR}}/include/ - rsync -aL --chmod=F+w {out_dir}/localversion ${{OUT_DIR}}/localversion - - # Restore real value of $ROOT_DIR in auto.conf.cmd - sed -i'' -e 's:${{ROOT_DIR}}:'"${{ROOT_DIR}}"':g' ${{OUT_DIR}}/include/config/auto.conf.cmd - """.format( - out_dir = out_dir.path, - ) - if trim_nonlisted_kmi_utils.get_value(ctx): # Ensure the dependent action uses the up-to-date abi_symbollist.raw # at the absolute path specified in abi_symbollist.raw.abspath @@ -479,14 +465,9 @@ def _kernel_config_impl(ctx): serialized_env_info_setup_script = ctx.actions.declare_file("{name}/{name}_setup.sh".format(name = ctx.attr.name)) ctx.actions.write( output = serialized_env_info_setup_script, - content = """ - {pre_setup} - {eval_restore_out_dir_cmd} - {post_setup} - """.format( - pre_setup = ctx.attr.env[KernelEnvInfo].setup, - eval_restore_out_dir_cmd = kernel_utils.eval_restore_out_dir_cmd(), - post_setup = post_setup, + content = get_config_setup_command( + env_setup_command = ctx.attr.env[KernelEnvInfo].setup, + out_dir = out_dir, ), ) @@ -574,6 +555,35 @@ def _get_config_script(ctx, inputs): runfiles = runfiles, ) +def get_config_setup_command( + env_setup_command, + out_dir): + """Returns the content of `<kernel_build>_config_setup.sh`, given the parameters. + + Args: + env_setup_command: command to set up environment from `kernel_env` + out_dir: output directory from `kernel_config` + """ + + return """ + {env_setup_command} + {eval_restore_out_dir_cmd} + + [ -z ${{OUT_DIR}} ] && echo "FATAL: configs post_env_info setup run without OUT_DIR set!" >&2 && exit 1 + # Restore kernel config inputs + mkdir -p ${{OUT_DIR}}/include/ + rsync -aL {out_dir}/.config ${{OUT_DIR}}/.config + rsync -aL --chmod=D+w {out_dir}/include/ ${{OUT_DIR}}/include/ + rsync -aL --chmod=F+w {out_dir}/localversion ${{OUT_DIR}}/localversion + + # Restore real value of $ROOT_DIR in auto.conf.cmd + sed -i'' -e 's:${{ROOT_DIR}}:'"${{ROOT_DIR}}"':g' ${{OUT_DIR}}/include/config/auto.conf.cmd + """.format( + env_setup_command = env_setup_command, + eval_restore_out_dir_cmd = kernel_utils.eval_restore_out_dir_cmd(), + out_dir = out_dir.path, + ) + def _kernel_config_additional_attrs(): return dicts.add( kernel_config_settings.of_kernel_config(), diff --git a/kleaf/impl/kernel_env.bzl b/kleaf/impl/kernel_env.bzl index 2011a6e..7163c3d 100644 --- a/kleaf/impl/kernel_env.bzl +++ b/kleaf/impl/kernel_env.bzl @@ -286,14 +286,10 @@ def _kernel_env_impl(ctx): command = command, ) - setup = hermetic_tools.setup - setup += """ - source {build_utils_sh} - # source the build environment - source {env} - """.format( - build_utils_sh = ctx.file._build_utils_sh.path, - env = out_file.path, + setup = get_env_info_setup_command( + hermetic_tools_setup = hermetic_tools.setup, + build_utils_sh = ctx.file._build_utils_sh, + env_setup_script = out_file, ) setup_tools = [ @@ -337,6 +333,20 @@ def _kernel_env_impl(ctx): DefaultInfo(files = depset([out_file])), ] +def get_env_info_setup_command(hermetic_tools_setup, build_utils_sh, env_setup_script): + """Returns text for KernelEnvInfo.setup""" + + return """ + {hermetic_tools_setup} + source {build_utils_sh} + # source the build environment + source {env_setup_script} + """.format( + hermetic_tools_setup = hermetic_tools_setup, + build_utils_sh = build_utils_sh.path, + env_setup_script = env_setup_script.path, + ) + def _get_env_setup_cmds(ctx): pre_env = "" if ctx.attr._debug_annotate_scripts[BuildSettingInfo].value: diff --git a/kleaf/impl/kernel_filegroup.bzl b/kleaf/impl/kernel_filegroup.bzl index bc6dd10..7ceb899 100644 --- a/kleaf/impl/kernel_filegroup.bzl +++ b/kleaf/impl/kernel_filegroup.bzl @@ -26,6 +26,7 @@ load( "KernelBuildUnameInfo", "KernelEnvAttrInfo", "KernelImagesInfo", + "KernelSerializedEnvInfo", "KernelToolchainInfo", "KernelUnstrippedModulesInfo", ) @@ -37,7 +38,9 @@ load( ) load(":debug.bzl", "debug") load(":hermetic_toolchain.bzl", "hermetic_toolchain") +load(":kernel_config.bzl", "get_config_setup_command") load(":kernel_config_settings.bzl", "kernel_config_settings") +load(":kernel_env.bzl", "get_env_info_setup_command") load(":kernel_toolchains_utils.bzl", "kernel_toolchains_utils") load( ":utils.bzl", @@ -94,17 +97,68 @@ def _get_kernel_release(ctx): ) return kernel_release +def _get_ddk_config_env(ctx): + """Returns `KernelBuildExtModuleInfo.ddk_config_env`.""" + + if not ctx.file.config_out_dir or not ctx.file.env_setup_script: + return None + + hermetic_tools = hermetic_toolchain.get(ctx) + toolchains = kernel_toolchains_utils.get(ctx) + + env_setup_command = """ + KLEAF_REPO_WORKSPACE_ROOT={kleaf_repo_workspace_root} + """.format( + kleaf_repo_workspace_root = Label(":kernel_filegroup.bzl").workspace_root, + ) + env_setup_command += get_env_info_setup_command( + hermetic_tools_setup = hermetic_tools.setup, + build_utils_sh = ctx.file._build_utils_sh, + env_setup_script = ctx.file.env_setup_script, + ) + + ddk_config_env_setup_command = get_config_setup_command( + env_setup_command = env_setup_command, + out_dir = ctx.file.config_out_dir, + ) + + ddk_config_env_setup_script = ctx.actions.declare_file( + "{name}/{name}_ddk_config_setup.sh".format(name = ctx.attr.name), + ) + ctx.actions.write( + output = ddk_config_env_setup_script, + content = ddk_config_env_setup_command, + ) + ddk_config_env = KernelSerializedEnvInfo( + setup_script = ddk_config_env_setup_script, + inputs = depset([ + ddk_config_env_setup_script, + ctx.file.env_setup_script, + ctx.version_file, + ], transitive = [target.files for target in ctx.attr.config_out_dir_files]), + tools = depset([ + ctx.file._build_utils_sh, + ], transitive = [ + hermetic_tools.deps, + toolchains.all_files, + ]), + ) + return ddk_config_env + def _kernel_filegroup_impl(ctx): hermetic_tools = hermetic_toolchain.get(ctx) all_deps = ctx.files.srcs + ctx.files.deps + ddk_config_env = _get_ddk_config_env(ctx) + # TODO(b/219112010): Implement KernelSerializedEnvInfo properly kernel_module_dev_info = KernelBuildExtModuleInfo( modules_staging_archive = utils.find_file(MODULES_STAGING_ARCHIVE, all_deps, what = ctx.label), # TODO(b/211515836): module_scripts might also be downloaded # Building kernel_module (excluding ddk_module) on top of kernel_filegroup is unsupported. # module_hdrs = None, + ddk_config_env = ddk_config_env, collect_unstripped_modules = ctx.attr.collect_unstripped_modules, ddk_module_defconfig_fragments = depset(transitive = [ target.files @@ -309,12 +363,29 @@ default, which in turn sets `collect_unstripped_modules` to `True` by default. allow_single_file = True, doc = "A file providing the kernel release string. This is preferred over `gki_artifacts`.", ), + "config_out_dir_files": attr.label_list( + doc = "Files in `config_out_dir`", + allow_files = True, + ), + "config_out_dir": attr.label( + allow_single_file = True, + doc = "Directory to support `kernel_config`", + ), + "env_setup_script": attr.label( + allow_single_file = True, + doc = "Setup script from `kernel_env`", + ), "_debug_print_scripts": attr.label(default = "//build/kernel/kleaf:debug_print_scripts"), "_cache_dir_config_tags": attr.label( default = "//build/kernel/kleaf/impl:cache_dir_config_tags", executable = True, cfg = "exec", ), + "_build_utils_sh": attr.label( + allow_single_file = True, + default = Label("//build/kernel:build_utils"), + cfg = "exec", + ), "ddk_module_defconfig_fragments": attr.label_list( doc = "Additional defconfig fragments for dependant DDK modules.", allow_empty = True, diff --git a/kleaf/impl/kernel_filegroup_declaration.bzl b/kleaf/impl/kernel_filegroup_declaration.bzl index 72080e0..e29c4aa 100644 --- a/kleaf/impl/kernel_filegroup_declaration.bzl +++ b/kleaf/impl/kernel_filegroup_declaration.bzl @@ -95,6 +95,9 @@ kernel_filegroup( kernel_release = {kernel_release_repr}, protected_modules_list = {protected_modules_repr}, ddk_module_defconfig_fragments = {ddk_module_defconfig_fragments_repr}, + config_out_dir_files = glob([{config_out_dir_repr} + "/**"]), + config_out_dir = {config_out_dir_repr}, + env_setup_script = {env_setup_script_repr}, target_platform = {target_platform_repr}, exec_platform = {exec_platform_repr}, visibility = ["//visibility:public"], @@ -150,6 +153,8 @@ def _write_filegroup_decl_file(ctx, info, deps_files, kernel_uapi_headers, templ info.ddk_module_defconfig_fragments, **(join | pkg) ) + sub.add_joined("{config_out_dir_repr}", depset([info.config_out_dir]), **(one | pkg)) + sub.add_joined("{env_setup_script_repr}", depset([info.env_setup_script]), **(one | pkg)) sub.add("{target_platform_repr}", repr(ctx.attr.kernel_build.label.name + "_platform_target")) sub.add("{exec_platform_repr}", repr(ctx.attr.kernel_build.label.name + "_platform_exec")) sub.add("{arch}", info.arch) @@ -177,6 +182,8 @@ def _create_archive(ctx, info, deps_files, kernel_uapi_headers, filegroup_decl_f info.module_outs_file, info.kernel_release, kernel_uapi_headers, + info.config_out_dir, + info.env_setup_script, ] if info.src_protected_modules_list: direct_inputs.append(info.src_protected_modules_list) |