Repository navigation
pip.parse vendored support #2462
Description
Activity
Here is a quick test example to show the differences in the requirements.bzl
Using WORKSPACE and pip_parse:
# WORKSPACE python_register_toolchains( name = "python_interpreter", ignore_root_user_error = True, python_version = "3.10.11", ) load("@python_interpreter//:defs.bzl", "interpreter") pip_parse( name = "test_lib", enable_implicit_namespace_pkgs = True, python_interpreter_target = interpreter, requirements_lock = "//:requirements.txt", ) # BUILD genrule( name = "test_bzl", srcs = ["@test_lib//:requirements.bzl"], outs = ["test.bzl"], cmd = "cp $< $@", )Using the new MODULE.bazel and pip.parse:
# MODULE.bazel python.toolchain( ignore_root_user_error = True, is_default = True, python_version = "3.10.11", ) use_repo(python, "python_3_10_11") pip.parse( hub_name = "test_lib", enable_implicit_namespace_pkgs = True, python_version = "3.10.11", requirements_lock = "//:requirements.txt", ) # BUILD genrule( name = "test_bzl", srcs = ["@test_lib//:requirements.bzl"], outs = ["test.bzl"], cmd = "cp $< $@", )You can see the new test.bzl misses:
load("@rules_python//python/pip_install:pip_repository.bzl", "whl_library") _packages = _config = def entry_point(pkg, script = None): def _get_annotation(requirement): def install_deps(**whl_library_kwargs):Is this a bug in the new pip.parse?
bzlmodhas lockfile mechanism and its own vendoring code. Does it not satisfy your needs?There are no plans to add vendoring in the same way it was done in
WORKSPACE.We have some internal custom bazel rules that
- uses
_config(specificallyextra_pip_argsfrom _config) from the old requirement.bzl generated bypip_parse. The newpip.parsegenerated requirement.bzl misses the whole_config. - use
_packagesfrom the old requirement.bzl generated bypip_parse. The newpip.parsegenerated requirement.bzl misses_packages
The python modules directory under runfiles also look different, eg: module
httplib2looks likeapp.runfiles/third_party_lib_httplib2/site-packages/when using WORKSPACE/pip_parse, but looks likeapp.runfiles/rules_python~~pip~third_party_lib_310_httplib2/site-packages/when using bzmod/pip.parseIs it possible to configure pip.parse to behave more like pip_parse to make the rules_python WORKSPACE to bzlmod migration easier?
- uses
It seems that you may be depending on implementation details within
rules_pythonand unfortunately some things just are not possible inbzlmod:- Having
extra_pip_argsmight be possible, but in general they can be different for each target platform, so supporting that might be tough. What is the use case here? - What is the use case for
_packages? - The layout is not controlled by us and is imposed by
bzlmoditself, so unfortunately that won't be possible.
You can go the other way though to first make
pip_parsecloser to whatpip.parseis. We tried to make it possible to do everything that you can do through the API exposed bypip_parse(i.e. the attributes and the publicly available constants inrequirements.bzl) to be possible inpip.parsebzlmod extension.Since there is nothing to do here, I will migrate this to a discussion.
- Having
- locked and limited conversation to collaborators
on Dec 16, 2024
We are currently using WORKSPACE and pip_parse_vendored following the examples in https://github.com/bazelbuild/rules_python/tree/main/examples/pip_parse_vendored
While migrating to MODULE.bazel, there are no similar pip.parse_vendored example. In particular, I am looking for the equivalent of https://github.com/bazelbuild/rules_python/blob/main/examples/pip_parse_vendored/BUILD.bazel#L24-L34
Looking at pip_parse generated @pip_deps_to_be_vendored//:requirements.bzl and pip.parse generated :requirements.bzl , I see the latter misses
_config = { ... }and_packages = [ ... ]Is that a bug?