Skip to content

Commit 4feb2a8

Browse files
gpsheadclaude
andcommitted
Add documentation for subprocess.run_pipeline()
Document the new run_pipeline() function, PipelineResult class, and PipelineError exception in the subprocess module documentation. Includes: - Function signature with stdin, stdout, stderr, capture_output, etc. - Note about shared stderr pipe and text mode caveat for interleaved multi-byte character sequences - Note that universal_newlines is not supported (use text=True) - Explanation that stdin connects to first process, stdout to last - Usage examples showing basic pipelines, multi-command pipelines, input handling, and error handling with check=True - PipelineResult attributes: commands, returncodes, returncode, stdout, stderr, and check_returncodes() method - PipelineError attributes: commands, returncodes, stdout, stderr, and failed list Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent e3a2fbe commit 4feb2a8

File tree

1 file changed

+179
-0
lines changed

1 file changed

+179
-0
lines changed

Doc/library/subprocess.rst

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,185 @@ underlying :class:`Popen` interface can be used directly.
264264
*stdout* and *stderr* attributes added
265265

266266

267+
.. function:: run_pipeline(*commands, stdin=None, input=None, \
268+
stdout=None, stderr=None, capture_output=False, \
269+
timeout=None, check=False, encoding=None, \
270+
errors=None, text=None, env=None, \
271+
**other_popen_kwargs)
272+
273+
Run a pipeline of commands connected via pipes, similar to shell pipelines.
274+
Wait for all commands to complete, then return a :class:`PipelineResult`
275+
instance.
276+
277+
Each positional argument should be a command (a list of strings, or a string
278+
if ``shell=True``) to execute. The standard output of each command is
279+
connected to the standard input of the next command in the pipeline.
280+
281+
This function requires at least two commands. For a single command, use
282+
:func:`run` instead.
283+
284+
If *capture_output* is true, the standard output of the final command and
285+
the standard error of all commands will be captured. All processes in the
286+
pipeline share a single stderr pipe, so their error output will be
287+
interleaved. The *stdout* and *stderr* arguments may not be supplied at
288+
the same time as *capture_output*.
289+
290+
A *timeout* may be specified in seconds. If the timeout expires, all
291+
child processes will be killed and waited for, and then a
292+
:exc:`TimeoutExpired` exception will be raised.
293+
294+
The *input* argument is passed to the first command's stdin. If used, it
295+
must be a byte sequence, or a string if *encoding* or *errors* is specified
296+
or *text* is true.
297+
298+
If *check* is true, and any process in the pipeline exits with a non-zero
299+
exit code, a :exc:`PipelineError` exception will be raised. This behavior
300+
is similar to the shell's ``pipefail`` option.
301+
302+
If *encoding* or *errors* are specified, or *text* is true, file objects
303+
are opened in text mode using the specified encoding and errors.
304+
305+
.. note::
306+
307+
When using ``text=True`` with ``capture_output=True`` or ``stderr=PIPE``,
308+
be aware that stderr output from multiple processes may be interleaved
309+
in ways that produce incomplete multi-byte character sequences. For
310+
reliable text decoding of stderr, consider capturing in binary mode
311+
and decoding manually with appropriate error handling, or use
312+
``errors='replace'`` or ``errors='backslashreplace'``.
313+
314+
If *stdin* is specified, it is connected to the first command's standard
315+
input. If *stdout* is specified, it is connected to the last command's
316+
standard output. When *stdout* is :data:`PIPE`, the output is available
317+
in the returned :class:`PipelineResult`'s :attr:`~PipelineResult.stdout`
318+
attribute. Other keyword arguments are passed to each :class:`Popen` call.
319+
320+
Unlike :func:`run`, this function does not accept *universal_newlines*.
321+
Use ``text=True`` instead.
322+
323+
Examples::
324+
325+
>>> import subprocess
326+
>>> # Equivalent to: echo "hello world" | tr a-z A-Z
327+
>>> result = subprocess.run_pipeline(
328+
... ['echo', 'hello world'],
329+
... ['tr', 'a-z', 'A-Z'],
330+
... capture_output=True, text=True
331+
... )
332+
>>> result.stdout
333+
'HELLO WORLD\n'
334+
>>> result.returncodes
335+
[0, 0]
336+
337+
>>> # Pipeline with three commands
338+
>>> result = subprocess.run_pipeline(
339+
... ['echo', 'one\ntwo\nthree'],
340+
... ['sort'],
341+
... ['head', '-n', '2'],
342+
... capture_output=True, text=True
343+
... )
344+
>>> result.stdout
345+
'one\nthree\n'
346+
347+
>>> # Using input parameter
348+
>>> result = subprocess.run_pipeline(
349+
... ['cat'],
350+
... ['wc', '-l'],
351+
... input='line1\nline2\nline3\n',
352+
... capture_output=True, text=True
353+
... )
354+
>>> result.stdout.strip()
355+
'3'
356+
357+
>>> # Error handling with check=True
358+
>>> subprocess.run_pipeline(
359+
... ['echo', 'hello'],
360+
... ['false'], # exits with status 1
361+
... check=True
362+
... )
363+
Traceback (most recent call last):
364+
...
365+
subprocess.PipelineError: Pipeline failed: command 1 ['false'] returned 1
366+
367+
.. versionadded:: next
368+
369+
370+
.. class:: PipelineResult
371+
372+
The return value from :func:`run_pipeline`, representing a pipeline of
373+
processes that have finished.
374+
375+
.. attribute:: commands
376+
377+
The list of commands used to launch the pipeline. Each command is a list
378+
of strings (or a string if ``shell=True`` was used).
379+
380+
.. attribute:: returncodes
381+
382+
List of exit status codes for each command in the pipeline. Typically,
383+
an exit status of 0 indicates that the command ran successfully.
384+
385+
A negative value ``-N`` indicates that the command was terminated by
386+
signal ``N`` (POSIX only).
387+
388+
.. attribute:: returncode
389+
390+
Exit status of the final command in the pipeline. This is a convenience
391+
property equivalent to ``returncodes[-1]``.
392+
393+
.. attribute:: stdout
394+
395+
Captured stdout from the final command in the pipeline. A bytes sequence,
396+
or a string if :func:`run_pipeline` was called with an encoding, errors,
397+
or ``text=True``. ``None`` if stdout was not captured.
398+
399+
.. attribute:: stderr
400+
401+
Captured stderr from all commands in the pipeline, combined. A bytes
402+
sequence, or a string if :func:`run_pipeline` was called with an
403+
encoding, errors, or ``text=True``. ``None`` if stderr was not captured.
404+
405+
.. method:: check_returncodes()
406+
407+
If any command's :attr:`returncode` is non-zero, raise a
408+
:exc:`PipelineError`.
409+
410+
.. versionadded:: next
411+
412+
413+
.. exception:: PipelineError
414+
415+
Subclass of :exc:`SubprocessError`, raised when a pipeline run by
416+
:func:`run_pipeline` (with ``check=True``) contains one or more commands
417+
that returned a non-zero exit status. This is similar to the shell's
418+
``pipefail`` behavior.
419+
420+
.. attribute:: commands
421+
422+
List of commands that were used in the pipeline.
423+
424+
.. attribute:: returncodes
425+
426+
List of exit status codes for each command in the pipeline.
427+
428+
.. attribute:: stdout
429+
430+
Output of the final command if it was captured. Otherwise, ``None``.
431+
432+
.. attribute:: stderr
433+
434+
Combined stderr output of all commands if it was captured.
435+
Otherwise, ``None``.
436+
437+
.. attribute:: failed
438+
439+
List of ``(index, command, returncode)`` tuples for each command
440+
that returned a non-zero exit status. The *index* is the position
441+
of the command in the pipeline (0-based).
442+
443+
.. versionadded:: next
444+
445+
267446
.. _frequently-used-arguments:
268447

269448
Frequently Used Arguments

0 commit comments

Comments
 (0)