Add documentation for rule-based bazel toolchain configuration

PiperOrigin-RevId: 606434760
Change-Id: Ie238b5513144e4289186af470e7503f05dd87890

2 files changed, 282 insertions, 1 deletions

diff --git a/cc/cc_toolchain_config_lib.bzl b/cc/cc_toolchain_config_lib.bzl
index 3a259de..876ed4d 100644
--- a/cc/cc_toolchain_config_lib.bzl
+++ b/cc/cc_toolchain_config_lib.bzl
@@ -492,7 +492,7 @@ def tool(path = None, with_features = [], execution_requirements = [], tool = No
         execution_requirements: Requirements on the execution environment for
             the execution of this tool, to be passed as out-of-band "hints" to
             the execution backend.
-            Ex. "requires-darwin"
+            Ex. "requires-mem:24g"
 
     Returns:
         A ToolInfo provider.
diff --git a/cc/toolchains/README.md b/cc/toolchains/README.md
new file mode 100644
index 0000000..976dd14
--- /dev/null
+++ b/cc/toolchains/README.md
@@ -0,0 +1,281 @@
+# Writing a custom rule_based C++ toolchain with rule-based definition.
+
+Work in progress!
+
+This document serves two purposes:
+* Until complete, this serves as an agreement for the final user-facing API. 
+* Once complete, this will serve as onboarding documentation.
+
+This section will be removed once complete.
+
+## Step 1: Define tools
+A tool is simply a binary. Just like any other bazel binary, a tool can specify
+additional files required to run.
+
+We can use any bazel binary as an input to anything that requires tools. In the
+example below, you could use both clang and ld as tools.
+
+```
+# @sysroot//:BUILD
+cc_tool(
+    name = "clang",
+    exe = ":bin/clang",
+    execution_requirements = ["requires-mem:24g"],
+    data = [...],
+)
+
+sh_binary(
+    name = "ld",
+    srcs = ["ld_wrapper.sh"],
+    data = [":bin/ld"],
+)
+    
+```
+
+## Step 2: Generate action configs from those tools
+An action config is a mapping from action to:
+
+* A list of tools, (the first one matching the execution requirements is used).
+* A list of flags and features that are always enabled for the action
+* A set of additional files required for the action
+
+Each action can only be specified once in the toolchain. Specifying multiple
+actions in a single `cc_action_config` is just a shorthand for specifying the
+same config for every one of those actions.
+
+If you're already familiar with how to define toolchains, the additional files
+is a replacement for `compile_files`, `link_files`, etc.
+
+Additionally, to replace `all_files`, we add `cc_additional_files_for_actions`.
+This allows you to specify that particular files are required for particular
+actions.
+
+We provide `additional_files` on the `cc_action_config` as a shorthand for 
+specifying `cc_additional_files_for_actions`
+
+Warning: Implying a feature that is not listed directly in the toolchain will throw
+an error. This is to ensure you don't accidentally add a feature to the
+toolchain.
+
+```
+cc_action_config(
+    name  = "c_compile",
+    actions = ["@rules_cc//actions:all_c_compile"],
+    tools = ["@sysroot//:clang"],
+    flag_sets = [":my_flag_set"],
+    implies = [":my_feature"],
+    additional_files = ["@sysroot//:all_header_files"],
+)
+
+cc_additional_files_for_actions(
+    name = "all_action_files",
+    actions = ["@rules_cc//actions:all_actions"],
+    additional_files = ["@sysroot//:always_needed_files"]
+)
+```
+
+## Step 3: Define some flag sets
+Flag sets are just sets of flags to be associated with actions. Most flag sets
+are simple, so we provide the shorthand `flags`. However, sometimes you
+need to do more complex things, for which we support `flag_groups` instead.
+
+Flag groups work exactly the same as the existing toolchain definition.
+
+Flag sets are a combination of both `flag_set` and `env_set` from the existing
+toolchain definition.
+
+`cc_flag_set_list` is simply a list of flag sets. This can be used to group
+flag sets together, and preserves ordering.
+
+```
+cc_flag_set(
+    name = "simple",
+    actions = ["@rules_cc//actions:all_cpp_compile_actions"],
+    flags = ["--foo"],
+    envs = {"FOO": "bar"},
+)
+
+cc_flag_group(
+    name = "complex_flag_group",
+    # API TBD
+)
+cc_flag_set(
+    name = "complex",
+    actions = ["@rules_cc//actions:c_compile"],
+    flag_groups = [":complex_flag_group"],
+)
+
+cc_flag_set_list(
+    name = "all_flags",
+    flag_sets = [":simple", ":complex"],
+)
+```
+
+## Step 4: Define some features
+A feature is a set of flags and configurations that can be enabled or disabled.
+
+Although the existing toolchain recommends using features to avoid duplication
+of definitions, we recommend avoiding using features unless you want the user to
+be able to enable / disable the feature themselves. This is because we provide
+alternatives such as `cc_flag_set_list` to allow combining flag sets and
+specifying them on each action in the action config.
+
+```
+cc_feature(
+    name = "my_feature",
+    feature_name = "my_feature",
+    flag_sets = [":all_flags"],
+    implies = [":other_feature"],
+)
+```
+
+## Step 5: Generate the toolchain
+The `cc_toolchain` macro:
+
+* Performs validation on the inputs (eg. no two action configs for a single
+  action)
+* Converts the type-safe providers to the unsafe ones in
+  `cc_toolchain_config_lib.bzl`
+* Generates a set of providers for each of the filegroups respectively
+* Generates the appropriate `native.cc_toolchain` invocation.
+
+```
+cc_toolchain(
+    name = "toolchain",
+    features = [":my_feature"]
+    unconditional_flag_sets = [":all_warnings"],
+    action_configs = [":c_compile"],
+    additional_files = [":all_action_files"],
+)
+```
+
+# Ancillary components for type-safe toolchains.
+## Well-known features
+Well-known features will be defined in `@rules_cc//features/well_known:*`.
+Any feature with `feature_name` in the well known features will have to specify
+overrides.
+
+`cc_toolchain` is aware of the builtin / well-known features. In order to
+ensure that a user understands that this overrides the builtin opt feature (I
+originally thought that it added extra flags to opt, but you still got the
+default ones, so that can definitely happen), and to ensure that they don't
+accidentally do so, we will force them to explicitly specify that it overrides
+the builtin one. This is essentially just an acknowledgement of "I know what
+I'm doing".
+
+Warning: Specifying two features with the same name is an error, unless one
+overrides the other. 
+
+```
+cc_feature(
+    name = "opt",
+    ...,
+    overrides = "@rules_cc//features/well_known:opt",
+)
+```
+
+In addition to well-known features, we could also consider in future iterations
+to also use known features for partial migrations, where you still imply a
+feature that's still defined by the legacy API:
+
+```
+# Implementation
+def cc_legacy_features(name, features):
+  for feature in features:
+    cc_known_feature(name = name + "_" + feature.name)
+  cc_legacy_features(name = name, features = FEATURES)
+
+
+# Build file
+FOO = feature(name = "foo", flag_sets=[flag_group(...)])
+FEATURES = [FOO]
+cc_legacy_features(name = "legacy_features", features = FEATURES)
+
+cc_feature(name = "bar", implies = [":legacy_features_foo"])
+
+cc_toolchain(
+  name = "toolchain",
+  legacy_features = ":legacy_features",
+  features = [":bar"],
+)
+```
+
+## Mutual exclusion
+Features can be mutually exclusive.
+
+We allow two approaches to mutual exclusion - via features or via categories.
+
+The existing toolchain uses `provides` for both of these. We rename it so that
+it makes more sense semantically.
+
+```
+cc_feature(
+   name = "incompatible_with_my_feature",
+   feature_name = "bar",
+   mutually_exclusive = [":my_feature"],
+)
+
+
+# This is an example of how we would define compilation mode.
+# Since it already exists, this wouldn't work.
+cc_mutual_exclusion_category(
+    name = "compilation_mode",
+)
+
+cc_feature(
+    name = "opt",
+    ...
+    mutually_exclusive = [":compilation_mode"],
+)
+cc_feature(
+    name = "dbg",
+    ...
+    mutually_exclusive = [":compilation_mode"],
+)
+```
+
+## Feature requirements
+Feature requirements can come in two formats.
+
+For example:
+
+* Features can require some subset of features to be enabled.
+* Flag sets can require some subset of features to be enabled, but others to be
+  disabled.
+
+This is very confusing for toolchain authors, so we will simplify things with
+the use of providers:
+
+* `cc_feature` will provide `feature`, `feature_set`, and `with_feature`
+* `cc_feature_set` will provide `feature_set` and `with_feature`.
+* `cc_feature_constraint` will provide `with_features` only.
+
+We will rename all `with_features` and `requires` to `requires_any_of`, to make
+it very clear that only one of the requirements needs to be met.
+
+```
+cc_feature_set(
+    name = "my_feature_set",
+    all_of = [":my_feature"],
+)
+
+cc_feature_constraint(
+    name = "my_feature_constraint",
+    all_of = [":my_feature"],
+    none_of = [":my_other_feature"],
+)
+
+cc_flag_set(
+   name = "foo",
+   # All of these provide with_feature.
+   requires_any_of = [":my_feature", ":my_feature_set", ":my_feature_constraint"]
+)
+
+# my_feature_constraint would be an error here.
+cc_feature(
+   name = "foo",
+   # Both of these provide feature_set.
+   requires_any_of = [":my_feature", ":my_feature_set"]
+   implies = [":my_other_feature", :my_other_feature_set"],
+)
+```