using args makes it difficult to directly re-run tests

[kjw-enf]

bazel build //my:hello_test
cd bazel-bin/my/hello_test.runfiles
./hello_test # error
./hello_test ./hello_test.py # works

Also, if you try and embed the test inside some other rule, the args are lost/unavailable.

So... can the args be embedded as data into one of the wrappers instead?

[kjw-enf]

I cobbled this together. it's a bit hacky because it's my first time writing starlark:

def _os_environ(envs):
  # expand shell-like ${ENV} into python code +os.environ.get("ENV", "")+
  expanded = []
  for e in envs:
    expanded.append(e.replace("$$XML_OUTPUT_FILE", '" + os.environ.get("XML_OUTPUT_FILE", "") + "'))
  return expanded

def _wrapper_pytest_impl(ctx):
  # store pytest args in an expanded template of wrapper_pytest.py
  output = ctx.actions.declare_file(ctx.attr.name)
  _args = '\"' + "\", \"".join(_os_environ(ctx.attr.args) + [x.path for x in ctx.files.srcs]) + '\"'  # quoted
  ctx.actions.write(
    output = output,
    content = """# -*- coding: utf-8 -*-
import os
import sys
import pytest
# TODO: find/cd into runfiles tree for direct user execution?
args = [""" + _args + """]
if __name__ == "__main__":
  sys.exit(pytest.main(args + sys.argv[1:]))
""",
    is_executable = True,
  )
  return [DefaultInfo(
    files = depset([output]),
  )]

wrapper_pytest = rule(
  implementation = _wrapper_pytest_impl,
  attrs = {
    "srcs": attr.label_list(allow_files = True),
    "args": attr.string_list(),
    "output": attr.output(),
  },
)

and inside my pytest macro:

  wrapper_pytest(
    # store args in a generated file named/executed by native.py_test(main=)
    name = name + "-pytest.py",
    args = initial_args + args,
    srcs = srcs,
  )

  native.py_test(
    name = name,
    srcs = [
      name + "-pytest.py",
    ] + srcs,
    main = ":" + name + "-pytest.py",
    args = [],
    python_version = "PY3",
    srcs_version = "PY3",
    deps = deps + pytest_dependencies,
    data = [
      "//:.pylintrc",
      "//:pyproject.toml",
      "//:pytest.ini", # ymmv, this is how we standardized the ini file
    ] + data,
    **kwargs
  )

sample generated wrapper:

$ cat bazel-bin/experimental/sleep_test-pytest.py
# -*- coding: utf-8 -*-
import os
import sys
import pytest
# TODO: find/cd into runfiles tree for direct user execution?
args = ["--junitxml=" + os.environ.get("XML_OUTPUT_FILE", "") + "", "experimental/sleep.py"]
if __name__ == "__main__":
    sys.exit(pytest.main(args + sys.argv[1:]))

So far I've only tested one of our pytests with this code, but our presubmits are checking our whole code base wrt this change to every last pytest.

[kjw-enf]

lol, backtick for a code block doesn't prevent markdown. but it should make sense to you.

[caseyduquettesc]

Use triple backticks for code blocks, it looks much better. I updated your example.

[kjw-enf]

FYI, our codebase works with this change, but quoting is now strict. i.e. I had to edit our BUILD files like this:

As a side effect, shell is no longer involved in quoting, so py_test(args=[]) should now be strict ala python subprocess.Popen([cmd, args...]):

     args = [
-        "-k='test_ping'",
+        "-k=test_ping",
     ],

Bazel documentation says that args= is subject to $(location) and "Make variable" substitution. That is no longer true with this implementation, which is likely to be an issue for some users. i.e. the os.environ hack I have in the python replicating "Make variable" substitution.

So the concept works, but doesn't match py_test(args=[]) documentation/expectations.