aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorGreg Roodt <groodt@gmail.com>2022-09-25 06:04:15 +1000
committerGitHub <noreply@github.com>2022-09-25 06:04:15 +1000
commitf0efec5cf8c0ae16483ee677a09ec70737a01bf5 (patch)
tree79e958ef1e97b1941260b84283db615678a153dc /README.md
parenta364fcac97b7ccadb5c5a703ec8ce2f2ecef0d97 (diff)
downloadbazelbuild-rules_python-f0efec5cf8c0ae16483ee677a09ec70737a01bf5.tar.gz
Standardise on pip_parse (#807)upstream/0.13.0
Diffstat (limited to 'README.md')
-rw-r--r--README.md81
1 files changed, 28 insertions, 53 deletions
diff --git a/README.md b/README.md
index 7359a2a..944493c 100644
--- a/README.md
+++ b/README.md
@@ -7,8 +7,7 @@
This repository is the home of the core Python rules -- `py_library`,
`py_binary`, `py_test`, and related symbols that provide the basis for Python
-support in Bazel. It also contains packaging rules for integrating with PyPI
-(`pip`). Documentation lives in the
+support in Bazel. It also contains package installation rules for integrating with PyPI and other package indices. Documentation lives in the
[`docs/`](https://github.com/bazelbuild/rules_python/tree/main/docs)
directory and in the
[Bazel Build Encyclopedia](https://docs.bazel.build/versions/master/be/python.html).
@@ -24,7 +23,7 @@ Once they are fully migrated to rules_python, they may evolve at a different
rate, but this repository will still follow
[semantic versioning](https://semver.org).
-The packaging rules (`pip_install`, etc.) are less stable. We may make breaking
+The package installation rules (`pip_install`, `pip_parse` etc.) are less stable. We may make breaking
changes as they evolve.
This repository is maintained by the Bazel community. Neither Google, nor the
@@ -101,14 +100,14 @@ py_binary(
)
```
-## Using the packaging rules
+## Using the package installation rules
Usage of the packaging rules involves two main steps.
-1. [Installing `pip` dependencies](#installing-pip-dependencies)
-2. [Consuming `pip` dependencies](#consuming-pip-dependencies)
+1. [Installing third_party packages](#installing-third_party-packages)
+2. [Using third_party packages as dependencies](#using-third_party-packages-as-dependencies)
-The packaging rules create two kinds of repositories: A central external repo that holds
+The package installation rules create two kinds of repositories: A central external repo that holds
downloaded wheel files, and individual external repos for each wheel's extracted
contents. Users only need to interact with the central external repo; the wheel repos
are essentially an implementation detail. The central external repo provides a
@@ -116,56 +115,13 @@ are essentially an implementation detail. The central external repo provides a
`BUILD` files that translates a pip package name into the label of a `py_library`
target in the appropriate wheel repo.
-### Installing `pip` dependencies
+### Installing third_party packages
To add pip dependencies to your `WORKSPACE`, load the `pip_install` function, and call it to create the
central external repo and individual wheel external repos.
```python
-load("@rules_python//python:pip.bzl", "pip_install")
-
-# Create a central external repo, @my_deps, that contains Bazel targets for all the
-# third-party packages specified in the requirements.txt file.
-pip_install(
- name = "my_deps",
- requirements = "//path/to:requirements.txt",
-)
-```
-
-Note that since `pip_install` is a repository rule and therefore executes pip at WORKSPACE-evaluation time, Bazel has no
-information about the Python toolchain and cannot enforce that the interpreter
-used to invoke pip matches the interpreter used to run `py_binary` targets. By
-default, `pip_install` uses the system command `"python3"`. This can be overridden by passing the
-`python_interpreter` attribute or `python_interpreter_target` attribute to `pip_install`.
-
-You can have multiple `pip_install`s in the same workspace. This will create multiple external repos that have no relation to
-one another, and may result in downloading the same wheels multiple times.
-
-As with any repository rule, if you would like to ensure that `pip_install` is
-re-executed in order to pick up a non-hermetic change to your environment (e.g.,
-updating your system `python` interpreter), you can force it to re-execute by running
-`bazel sync --only [pip_install name]`.
-
-### Fetch `pip` dependencies lazily
-
-One pain point with `pip_install` is the need to download all dependencies resolved by
-your requirements.txt before the bazel analysis phase can start. For large python monorepos
-this can take a long time, especially on slow connections.
-
-`pip_parse` provides a solution to this problem. If you can provide a lock
-file of all your python dependencies `pip_parse` will translate each requirement into its own external repository.
-Bazel will only fetch/build wheels for the requirements in the subgraph of your build target.
-
-There are API differences between `pip_parse` and `pip_install`:
-1. `pip_parse` requires a fully resolved lock file of your python dependencies. You can generate this by using the `compile_pip_requirements` rule,
- running `pip-compile` directly, or using virtualenv and `pip freeze`. `pip_parse` uses a label argument called `requirements_lock` instead of
- `requirements` to make this distinction clear.
-2. `pip_parse` translates your requirements into a starlark macro called `install_deps`. You must call this macro in your WORKSPACE to
- declare your dependencies.
-
-
-```python
load("@rules_python//python:pip.bzl", "pip_parse")
# Create a central repo that knows about the dependencies needed from
@@ -174,14 +130,33 @@ pip_parse(
name = "my_deps",
requirements_lock = "//path/to:requirements_lock.txt",
)
-
# Load the starlark macro which will define your dependencies.
load("@my_deps//:requirements.bzl", "install_deps")
# Call it to define repos for your requirements.
install_deps()
```
-### Consuming `pip` dependencies
+Note that since `pip_parse` is a repository rule and therefore executes pip at WORKSPACE-evaluation time, Bazel has no
+information about the Python toolchain and cannot enforce that the interpreter
+used to invoke pip matches the interpreter used to run `py_binary` targets. By
+default, `pip_parse` uses the system command `"python3"`. This can be overridden by passing the
+`python_interpreter` attribute or `python_interpreter_target` attribute to `pip_parse`.
+
+You can have multiple `pip_parse`s in the same workspace. This will create multiple external repos that have no relation to
+one another, and may result in downloading the same wheels multiple times.
+
+As with any repository rule, if you would like to ensure that `pip_parse` is
+re-executed in order to pick up a non-hermetic change to your environment (e.g.,
+updating your system `python` interpreter), you can force it to re-execute by running
+`bazel sync --only [pip_parse name]`.
+
+Note: The `pip_install` rule is deprecated. `pip_parse` offers identical functionality and both `pip_install`
+and `pip_parse` now have the same implementation. The name `pip_install` may be removed in a future version of the rules.
+The maintainers have taken all reasonable efforts to faciliate a smooth transition, but some users of `pip_install` will
+need to replace their existing `requirements.txt` with a fully resolved set of dependencies using a tool such as
+`pip-tools` or the `compile_pip_requirements` repository rule.
+
+### Using third_party packages as dependencies
Each extracted wheel repo contains a `py_library` target representing
the wheel's contents. There are two ways to access this library. The