Edit File: popen2.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>17.5. popen2 — Subprocesses with accessible I/O streams — Python v2.6.6 documentation</title> <link rel="stylesheet" href="../_static/default.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '../', VERSION: '2.6.6', COLLAPSE_MODINDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="../_static/jquery.js"></script> <script type="text/javascript" src="../_static/doctools.js"></script> <link rel="search" type="application/opensearchdescription+xml" title="Search within Python v2.6.6 documentation" href="../_static/opensearch.xml"/> <link rel="author" title="About these documents" href="../about.html" /> <link rel="copyright" title="Copyright" href="../copyright.html" /> <link rel="top" title="Python v2.6.6 documentation" href="../index.html" /> <link rel="up" title="17. Interprocess Communication and Networking" href="ipc.html" /> <link rel="next" title="17.6. asyncore — Asynchronous socket handler" href="asyncore.html" /> <link rel="prev" title="17.4. signal — Set handlers for asynchronous events" href="signal.html" /> <link rel="shortcut icon" type="image/png" href="../_static/py.png" /> </head> <body> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="../genindex.html" title="General Index" accesskey="I">index</a></li> <li class="right" > <a href="../modindex.html" title="Global Module Index" accesskey="M">modules</a> |</li> <li class="right" > <a href="asyncore.html" title="17.6. asyncore — Asynchronous socket handler" accesskey="N">next</a> |</li> <li class="right" > <a href="signal.html" title="17.4. signal — Set handlers for asynchronous events" accesskey="P">previous</a> |</li> <li><img src="../_static/py.png" alt="" style="vertical-align: middle; margin-top: -1px"/></li> <li><a href="../index.html">Python v2.6.6 documentation</a> »</li> <li><a href="index.html" >The Python Standard Library</a> »</li> <li><a href="ipc.html" accesskey="U">17. Interprocess Communication and Networking</a> »</li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="module-popen2"> <h1>17.5. <tt class="xref docutils literal"><span class="pre">popen2</span></tt> — Subprocesses with accessible I/O streams<a class="headerlink" href="#module-popen2" title="Permalink to this headline">¶</a></h1> <p class="deprecated"> <span class="versionmodified">Deprecated since version 2.6: </span>This module is obsolete. Use the <a title="Subprocess management." class="reference external" href="subprocess.html#module-subprocess"><tt class="xref docutils literal"><span class="pre">subprocess</span></tt></a> module. Check especially the <a class="reference external" href="subprocess.html#subprocess-replacements"><em>Replacing Older Functions with the subprocess Module</em></a> section.</p> <p>This module allows you to spawn processes and connect to their input/output/error pipes and obtain their return codes under Unix and Windows.</p> <p>The <a title="Subprocess management." class="reference external" href="subprocess.html#module-subprocess"><tt class="xref docutils literal"><span class="pre">subprocess</span></tt></a> module provides more powerful facilities for spawning new processes and retrieving their results. Using the <a title="Subprocess management." class="reference external" href="subprocess.html#module-subprocess"><tt class="xref docutils literal"><span class="pre">subprocess</span></tt></a> module is preferable to using the <tt class="xref docutils literal"><span class="pre">popen2</span></tt> module.</p> <p>The primary interface offered by this module is a trio of factory functions. For each of these, if <em>bufsize</em> is specified, it specifies the buffer size for the I/O pipes. <em>mode</em>, if provided, should be the string <tt class="docutils literal"><span class="pre">'b'</span></tt> or <tt class="docutils literal"><span class="pre">'t'</span></tt>; on Windows this is needed to determine whether the file objects should be opened in binary or text mode. The default value for <em>mode</em> is <tt class="docutils literal"><span class="pre">'t'</span></tt>.</p> <p>On Unix, <em>cmd</em> may be a sequence, in which case arguments will be passed directly to the program without shell intervention (as with <a title="os.spawnv" class="reference external" href="os.html#os.spawnv"><tt class="xref docutils literal"><span class="pre">os.spawnv()</span></tt></a>). If <em>cmd</em> is a string it will be passed to the shell (as with <a title="os.system" class="reference external" href="os.html#os.system"><tt class="xref docutils literal"><span class="pre">os.system()</span></tt></a>).</p> <p>The only way to retrieve the return codes for the child processes is by using the <tt class="xref docutils literal"><span class="pre">poll()</span></tt> or <tt class="xref docutils literal"><span class="pre">wait()</span></tt> methods on the <a title="popen2.Popen3" class="reference internal" href="#popen2.Popen3"><tt class="xref docutils literal"><span class="pre">Popen3</span></tt></a> and <a title="popen2.Popen4" class="reference internal" href="#popen2.Popen4"><tt class="xref docutils literal"><span class="pre">Popen4</span></tt></a> classes; these are only available on Unix. This information is not available when using the <a title="popen2.popen2" class="reference internal" href="#popen2.popen2"><tt class="xref docutils literal"><span class="pre">popen2()</span></tt></a>, <a title="popen2.popen3" class="reference internal" href="#popen2.popen3"><tt class="xref docutils literal"><span class="pre">popen3()</span></tt></a>, and <a title="popen2.popen4" class="reference internal" href="#popen2.popen4"><tt class="xref docutils literal"><span class="pre">popen4()</span></tt></a> functions, or the equivalent functions in the <a title="Miscellaneous operating system interfaces." class="reference external" href="os.html#module-os"><tt class="xref docutils literal"><span class="pre">os</span></tt></a> module. (Note that the tuples returned by the <a title="Miscellaneous operating system interfaces." class="reference external" href="os.html#module-os"><tt class="xref docutils literal"><span class="pre">os</span></tt></a> module’s functions are in a different order from the ones returned by the <tt class="xref docutils literal"><span class="pre">popen2</span></tt> module.)</p> <dl class="function"> <dt id="popen2.popen2"> <tt class="descclassname">popen2.</tt><tt class="descname">popen2</tt><big>(</big><em>cmd</em><span class="optional">[</span>, <em>bufsize</em><span class="optional">[</span>, <em>mode</em><span class="optional">]</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#popen2.popen2" title="Permalink to this definition">¶</a></dt> <dd>Executes <em>cmd</em> as a sub-process. Returns the file objects <tt class="docutils literal"><span class="pre">(child_stdout,</span> <span class="pre">child_stdin)</span></tt>.</dd></dl> <dl class="function"> <dt id="popen2.popen3"> <tt class="descclassname">popen2.</tt><tt class="descname">popen3</tt><big>(</big><em>cmd</em><span class="optional">[</span>, <em>bufsize</em><span class="optional">[</span>, <em>mode</em><span class="optional">]</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#popen2.popen3" title="Permalink to this definition">¶</a></dt> <dd>Executes <em>cmd</em> as a sub-process. Returns the file objects <tt class="docutils literal"><span class="pre">(child_stdout,</span> <span class="pre">child_stdin,</span> <span class="pre">child_stderr)</span></tt>.</dd></dl> <dl class="function"> <dt id="popen2.popen4"> <tt class="descclassname">popen2.</tt><tt class="descname">popen4</tt><big>(</big><em>cmd</em><span class="optional">[</span>, <em>bufsize</em><span class="optional">[</span>, <em>mode</em><span class="optional">]</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#popen2.popen4" title="Permalink to this definition">¶</a></dt> <dd><p>Executes <em>cmd</em> as a sub-process. Returns the file objects <tt class="docutils literal"><span class="pre">(child_stdout_and_stderr,</span> <span class="pre">child_stdin)</span></tt>.</p> <p class="versionadded"> <span class="versionmodified">New in version 2.0.</span></p> </dd></dl> <p>On Unix, a class defining the objects returned by the factory functions is also available. These are not used for the Windows implementation, and are not available on that platform.</p> <dl class="class"> <dt id="popen2.Popen3"> <em class="property">class </em><tt class="descclassname">popen2.</tt><tt class="descname">Popen3</tt><big>(</big><em>cmd</em><span class="optional">[</span>, <em>capturestderr</em><span class="optional">[</span>, <em>bufsize</em><span class="optional">]</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#popen2.Popen3" title="Permalink to this definition">¶</a></dt> <dd><p>This class represents a child process. Normally, <a title="popen2.Popen3" class="reference internal" href="#popen2.Popen3"><tt class="xref docutils literal"><span class="pre">Popen3</span></tt></a> instances are created using the <a title="popen2.popen2" class="reference internal" href="#popen2.popen2"><tt class="xref docutils literal"><span class="pre">popen2()</span></tt></a> and <a title="popen2.popen3" class="reference internal" href="#popen2.popen3"><tt class="xref docutils literal"><span class="pre">popen3()</span></tt></a> factory functions described above.</p> <p>If not using one of the helper functions to create <a title="popen2.Popen3" class="reference internal" href="#popen2.Popen3"><tt class="xref docutils literal"><span class="pre">Popen3</span></tt></a> objects, the parameter <em>cmd</em> is the shell command to execute in a sub-process. The <em>capturestderr</em> flag, if true, specifies that the object should capture standard error output of the child process. The default is false. If the <em>bufsize</em> parameter is specified, it specifies the size of the I/O buffers to/from the child process.</p> </dd></dl> <dl class="class"> <dt id="popen2.Popen4"> <em class="property">class </em><tt class="descclassname">popen2.</tt><tt class="descname">Popen4</tt><big>(</big><em>cmd</em><span class="optional">[</span>, <em>bufsize</em><span class="optional">]</span><big>)</big><a class="headerlink" href="#popen2.Popen4" title="Permalink to this definition">¶</a></dt> <dd><p>Similar to <a title="popen2.Popen3" class="reference internal" href="#popen2.Popen3"><tt class="xref docutils literal"><span class="pre">Popen3</span></tt></a>, but always captures standard error into the same file object as standard output. These are typically created using <a title="popen2.popen4" class="reference internal" href="#popen2.popen4"><tt class="xref docutils literal"><span class="pre">popen4()</span></tt></a>.</p> <p class="versionadded"> <span class="versionmodified">New in version 2.0.</span></p> </dd></dl> <div class="section" id="popen3-and-popen4-objects"> <span id="popen3-objects"></span><h2>17.5.1. Popen3 and Popen4 Objects<a class="headerlink" href="#popen3-and-popen4-objects" title="Permalink to this headline">¶</a></h2> <p>Instances of the <a title="popen2.Popen3" class="reference internal" href="#popen2.Popen3"><tt class="xref docutils literal"><span class="pre">Popen3</span></tt></a> and <a title="popen2.Popen4" class="reference internal" href="#popen2.Popen4"><tt class="xref docutils literal"><span class="pre">Popen4</span></tt></a> classes have the following methods:</p> <dl class="method"> <dt id="popen2.Popen3.poll"> <tt class="descclassname">Popen3.</tt><tt class="descname">poll</tt><big>(</big><big>)</big><a class="headerlink" href="#popen2.Popen3.poll" title="Permalink to this definition">¶</a></dt> <dd>Returns <tt class="docutils literal"><span class="pre">-1</span></tt> if child process hasn’t completed yet, or its status code (see <a title="popen2.Popen3.wait" class="reference internal" href="#popen2.Popen3.wait"><tt class="xref docutils literal"><span class="pre">wait()</span></tt></a>) otherwise.</dd></dl> <dl class="method"> <dt id="popen2.Popen3.wait"> <tt class="descclassname">Popen3.</tt><tt class="descname">wait</tt><big>(</big><big>)</big><a class="headerlink" href="#popen2.Popen3.wait" title="Permalink to this definition">¶</a></dt> <dd>Waits for and returns the status code of the child process. The status code encodes both the return code of the process and information about whether it exited using the <tt class="xref docutils literal"><span class="pre">exit()</span></tt> system call or died due to a signal. Functions to help interpret the status code are defined in the <a title="Miscellaneous operating system interfaces." class="reference external" href="os.html#module-os"><tt class="xref docutils literal"><span class="pre">os</span></tt></a> module; see section <a class="reference external" href="os.html#os-process"><em>Process Management</em></a> for the <tt class="xref docutils literal"><span class="pre">W*()</span></tt> family of functions.</dd></dl> <p>The following attributes are also available:</p> <dl class="attribute"> <dt id="popen2.Popen3.fromchild"> <tt class="descclassname">Popen3.</tt><tt class="descname">fromchild</tt><a class="headerlink" href="#popen2.Popen3.fromchild" title="Permalink to this definition">¶</a></dt> <dd>A file object that provides output from the child process. For <a title="popen2.Popen4" class="reference internal" href="#popen2.Popen4"><tt class="xref docutils literal"><span class="pre">Popen4</span></tt></a> instances, this will provide both the standard output and standard error streams.</dd></dl> <dl class="attribute"> <dt id="popen2.Popen3.tochild"> <tt class="descclassname">Popen3.</tt><tt class="descname">tochild</tt><a class="headerlink" href="#popen2.Popen3.tochild" title="Permalink to this definition">¶</a></dt> <dd>A file object that provides input to the child process.</dd></dl> <dl class="attribute"> <dt id="popen2.Popen3.childerr"> <tt class="descclassname">Popen3.</tt><tt class="descname">childerr</tt><a class="headerlink" href="#popen2.Popen3.childerr" title="Permalink to this definition">¶</a></dt> <dd>A file object that provides error output from the child process, if <em>capturestderr</em> was true for the constructor, otherwise <tt class="xref docutils literal"><span class="pre">None</span></tt>. This will always be <tt class="xref docutils literal"><span class="pre">None</span></tt> for <a title="popen2.Popen4" class="reference internal" href="#popen2.Popen4"><tt class="xref docutils literal"><span class="pre">Popen4</span></tt></a> instances.</dd></dl> <dl class="attribute"> <dt id="popen2.Popen3.pid"> <tt class="descclassname">Popen3.</tt><tt class="descname">pid</tt><a class="headerlink" href="#popen2.Popen3.pid" title="Permalink to this definition">¶</a></dt> <dd>The process ID of the child process.</dd></dl> </div> <div class="section" id="flow-control-issues"> <span id="popen2-flow-control"></span><h2>17.5.2. Flow Control Issues<a class="headerlink" href="#flow-control-issues" title="Permalink to this headline">¶</a></h2> <p>Any time you are working with any form of inter-process communication, control flow needs to be carefully thought out. This remains the case with the file objects provided by this module (or the <a title="Miscellaneous operating system interfaces." class="reference external" href="os.html#module-os"><tt class="xref docutils literal"><span class="pre">os</span></tt></a> module equivalents).</p> <p>When reading output from a child process that writes a lot of data to standard error while the parent is reading from the child’s standard output, a deadlock can occur. A similar situation can occur with other combinations of reads and writes. The essential factors are that more than <tt class="xref docutils literal"><span class="pre">_PC_PIPE_BUF</span></tt> bytes are being written by one process in a blocking fashion, while the other process is reading from the first process, also in a blocking fashion.</p> <p>There are several ways to deal with this situation.</p> <p>The simplest application change, in many cases, will be to follow this model in the parent process:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">popen2</span> <span class="n">r</span><span class="p">,</span> <span class="n">w</span><span class="p">,</span> <span class="n">e</span> <span class="o">=</span> <span class="n">popen2</span><span class="o">.</span><span class="n">popen3</span><span class="p">(</span><span class="s">'python slave.py'</span><span class="p">)</span> <span class="n">e</span><span class="o">.</span><span class="n">readlines</span><span class="p">()</span> <span class="n">r</span><span class="o">.</span><span class="n">readlines</span><span class="p">()</span> <span class="n">r</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="n">e</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="n">w</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> </pre></div> </div> <p>with code like this in the child:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">os</span> <span class="kn">import</span> <span class="nn">sys</span> <span class="c"># note that each of these print statements</span> <span class="c"># writes a single long string</span> <span class="k">print</span> <span class="o">>></span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">,</span> <span class="mi">400</span> <span class="o">*</span> <span class="s">'this is a test</span><span class="se">\n</span><span class="s">'</span> <span class="n">os</span><span class="o">.</span><span class="n">close</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="o">.</span><span class="n">fileno</span><span class="p">())</span> <span class="k">print</span> <span class="o">>></span><span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="p">,</span> <span class="mi">400</span> <span class="o">*</span> <span class="s">'this is another test</span><span class="se">\n</span><span class="s">'</span> </pre></div> </div> <p>In particular, note that <tt class="docutils literal"><span class="pre">sys.stderr</span></tt> must be closed after writing all data, or <tt class="xref docutils literal"><span class="pre">readlines()</span></tt> won’t return. Also note that <a title="os.close" class="reference external" href="os.html#os.close"><tt class="xref docutils literal"><span class="pre">os.close()</span></tt></a> must be used, as <tt class="docutils literal"><span class="pre">sys.stderr.close()</span></tt> won’t close <tt class="docutils literal"><span class="pre">stderr</span></tt> (otherwise assigning to <tt class="docutils literal"><span class="pre">sys.stderr</span></tt> will silently close it, so no further errors can be printed).</p> <p>Applications which need to support a more general approach should integrate I/O over pipes with their <tt class="xref docutils literal"><span class="pre">select()</span></tt> loops, or use separate threads to read each of the individual files provided by whichever <tt class="xref docutils literal"><span class="pre">popen*()</span></tt> function or <tt class="xref docutils literal"><span class="pre">Popen*</span></tt> class was used.</p> <div class="admonition-see-also admonition seealso"> <p class="first admonition-title">See also</p> <dl class="last docutils"> <dt>Module <a title="Subprocess management." class="reference external" href="subprocess.html#module-subprocess"><tt class="xref docutils literal"><span class="pre">subprocess</span></tt></a></dt> <dd>Module for spawning and managing subprocesses.</dd> </dl> </div> </div> </div> </div> </div> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <h3><a href="../contents.html">Table Of Contents</a></h3> <ul> <li><a class="reference external" href="#">17.5. <tt class="docutils literal"><span class="pre">popen2</span></tt> — Subprocesses with accessible I/O streams</a><ul> <li><a class="reference external" href="#popen3-and-popen4-objects">17.5.1. Popen3 and Popen4 Objects</a></li> <li><a class="reference external" href="#flow-control-issues">17.5.2. Flow Control Issues</a></li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="signal.html" title="previous chapter">17.4. <tt class="docutils literal"><span class="pre">signal</span></tt> — Set handlers for asynchronous events</a></p> <h4>Next topic</h4> <p class="topless"><a href="asyncore.html" title="next chapter">17.6. <tt class="docutils literal docutils literal docutils literal"><span class="pre">asyncore</span></tt> — Asynchronous socket handler</a></p> <h3>This Page</h3> <ul class="this-page-menu"> <li><a href="../bugs.html">Report a Bug</a></li> <li><a href="../_sources/library/popen2.txt" rel="nofollow">Show Source</a></li> </ul> <div id="searchbox" style="display: none"> <h3>Quick search</h3> <form class="search" action="../search.html" method="get"> <input type="text" name="q" size="18" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <script type="text/javascript">$('#searchbox').show(0);</script> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="../genindex.html" title="General Index" >index</a></li> <li class="right" > <a href="../modindex.html" title="Global Module Index" >modules</a> |</li> <li class="right" > <a href="asyncore.html" title="17.6. asyncore — Asynchronous socket handler" >next</a> |</li> <li class="right" > <a href="signal.html" title="17.4. signal — Set handlers for asynchronous events" >previous</a> |</li> <li><img src="../_static/py.png" alt="" style="vertical-align: middle; margin-top: -1px"/></li> <li><a href="../index.html">Python v2.6.6 documentation</a> »</li> <li><a href="index.html" >The Python Standard Library</a> »</li> <li><a href="ipc.html" >17. Interprocess Communication and Networking</a> »</li> </ul> </div> <div class="footer"> © <a href="../copyright.html">Copyright</a> 1990-2011, Python Software Foundation. <br /> The Python Software Foundation is a non-profit corporation. <a href="http://www.python.org/psf/donations/">Please donate.</a> <br /> Last updated on Jul 20, 2011. <a href="../bugs.html">Found a bug</a>? <br /> Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.6.6. </div> </body> </html>
Back to File Manager