scripts/release/util.py
author David Anderson <david.jc.anderson@gmail.com>
Fri, 13 Mar 2009 23:12:43 +0000
changeset 1846 ac30e04bcbba
parent 1835 3f30b7b14c57
child 1888 ef350db7f753
permissions -rw-r--r--
Redirect stdout/stderr through the logging system. This change also causes the uncaptured output of subprocesses to be routed to the logging system for recording in the session transcript.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     1
# Copyright 2009 the Melange authors.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     2
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     3
# Licensed under the Apache License, Version 2.0 (the "License");
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     4
# you may not use this file except in compliance with the License.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     5
# You may obtain a copy of the License at
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     6
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     7
#   http://www.apache.org/licenses/LICENSE-2.0
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     8
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     9
# Unless required by applicable law or agreed to in writing, software
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    10
# distributed under the License is distributed on an "AS IS" BASIS,
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    11
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    12
# See the License for the specific language governing permissions and
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    13
# limitations under the License.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    14
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    15
"""Various utilities.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    16
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    17
Current contents:
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    18
 - Text colorization using ANSI color codes
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    19
 - A class to construct and manage paths under a root path.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    20
 - A helper to manage running subprocesses, wrapping the subprocess module.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    21
"""
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    22
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    23
__authors__ = [
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    24
    # alphabetical order by last name, please
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    25
    '"David Anderson" <dave@natulte.net>',
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    26
    ]
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    27
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    28
import os.path
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    29
import re
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    30
try:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    31
  import cStringIO as StringIO
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    32
except ImportError:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    33
  import StringIO
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    34
import subprocess
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    35
import threading
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    36
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    37
import error
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    38
import log
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    39
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    40
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    41
class Error(error.Error):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    42
  pass
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    43
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    44
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    45
class SubprocessFailed(Error):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    46
  """A subprocess returned a non-zero error code."""
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    47
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    48
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    49
# The magic escape sequence understood by modern terminal emulators to
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    50
# configure fore/background colors and other basic text display
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    51
# settings.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    52
_ANSI_ESCAPE = '\x1b[%dm'
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    53
_ANSI_ESCAPE_RE = re.compile(r'\x1b\[\d+m')
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    54
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    55
1826
12de6d73a908 Followup to r2496. Fix obvious errors.
David Anderson <david.jc.anderson@gmail.com>
parents: 1825
diff changeset
    56
# Some internal non-color settings that we use.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    57
_RESET = 0  # Reset to terminal defaults.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    58
_BOLD = 1   # Brighter colors.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    59
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    60
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    61
# ANSI color codes.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    62
RED = 31
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    63
GREEN = 32
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    64
WHITE = 37
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    65
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    66
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    67
def _ansi_escape(code):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    68
  return _ANSI_ESCAPE % code
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    69
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    70
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    71
def colorize(text, color, bold=False):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    72
  """Colorize some text using ANSI color codes.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    73
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    74
  Note that while ANSI color codes look good in a terminal they look
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    75
  like noise in log files unless viewed in an ANSI color capable
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    76
  viewer (such as 'less -R').
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    77
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    78
  Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    79
    text: The text to colorize.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    80
    color: One of the color symbols from this module.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    81
    bold: If True, make the color brighter.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    82
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    83
  Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    84
    The input text string, appropriately sprinkled with color
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    85
    codes. Colors are reset to terminal defaults after the input
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    86
    text.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    87
  """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    88
  bold = _ansi_escape(_BOLD) if bold else ''
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    89
  return '%s%s%s%s' % (bold, _ansi_escape(color),
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    90
                       text, _ansi_escape(_RESET))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    91
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    92
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    93
def decolorize(text):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    94
  """Remove ANSI color codes from text."""
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    95
  return _ANSI_ESCAPE_RE.sub('', text)
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    96
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    97
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    98
class Paths(object):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    99
  """A helper to construct and check paths under a given root."""
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   100
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   101
  def __init__(self, root):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   102
    """Initializer.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   103
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   104
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   105
      root: The root of all paths this instance will consider.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   106
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   107
    self._root = os.path.abspath(
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   108
      os.path.expandvars(os.path.expanduser(root)))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   109
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   110
  def path(self, path=''):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   111
    """Construct and return a path under the path root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   112
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   113
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   114
      path: The desired path string relative to the root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   115
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   116
    Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   117
      The absolute path corresponding to the relative input path.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   118
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   119
    assert not os.path.isabs(path)
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   120
    return os.path.abspath(os.path.join(self._root, path))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   121
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   122
  def exists(self, path=''):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   123
    """Check for the existence of a path under the path root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   124
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   125
    Does not discriminate on the path type (ie. it could be a
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   126
    directory, a file, a symbolic link...), just checks for the
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   127
    existence of the path.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   128
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   129
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   130
      path: The path string relative to the root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   131
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   132
    Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   133
      True if the path exists, False otherwise.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   134
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   135
    return os.path.exists(self.path(path))
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   136
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   137
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   138
class _PipeAdapter(threading.Thread):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   139
  """A thread that connects one file-like object to another"""
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   140
  def __init__(self, pipe, logfile):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   141
    threading.Thread.__init__(self)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   142
    self.pipe, self.logfile = pipe, logfile
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   143
    self.setDaemon(True)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   144
    self.start()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   145
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   146
  def run(self):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   147
    try:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   148
      while True:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   149
        data = self.pipe.read(512)  # Small to retain interactivity
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   150
        if not data:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   151
          return
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   152
        self.logfile.write(data)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   153
    except (EOFError, OSError):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   154
      pass
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   155
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   156
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   157
def run(argv, cwd=None, capture=False, split_capture=True, stdin=None):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   158
  """Run the given command and optionally return its output.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   159
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   160
  Note that if you set capture=True, the command's output is
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   161
  buffered in memory. Output capture should only be used with
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   162
  commands that output small amounts of data. O(kB) is fine, O(MB)
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   163
  is starting to push it a little.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   164
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   165
  Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   166
    argv: A list containing the name of the program to run, followed
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   167
      by its argument vector.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   168
    cwd: Run the program from this directory.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   169
    capture: If True, capture the program's stdout stream. If False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   170
         stdout will output to sys.stdout.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   171
    split_capture: If True, return the captured output as a list of
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   172
           lines. Else, return as a single unaltered string.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   173
    stdin: The string to feed to the program's stdin stream.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   174
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   175
  Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   176
    If capture is True, a string containing the combined
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   177
    stdout/stderr output of the program. If capture is False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   178
    nothing is returned.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   179
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   180
  Raises:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   181
    SubprocessFailed: The subprocess exited with a non-zero exit
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   182
            code.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   183
  """
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   184
  log.debug(colorize('# ' + ' '.join(argv), WHITE, bold=True))
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   185
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   186
  process = subprocess.Popen(argv,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   187
                             shell=False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   188
                             cwd=cwd,
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   189
                             stdin=(subprocess.PIPE if stdin else None),
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   190
                             stdout=subprocess.PIPE,
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   191
                             stderr=subprocess.PIPE)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   192
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   193
  # Spin up threads to capture stdout and stderr. Depending on the
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   194
  # value of the capture parameter, stdout is pushed either into the
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   195
  # log or into a string for processing. stderr always goes
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   196
  # into the log.
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   197
  #
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   198
  # Threads are necessary because all of writing to stdin, reading
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   199
  # from stdout and reading from stderr must all happen
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   200
  # simultaneously. Otherwise, there is a risk that one of the pipes
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   201
  # will fill up, causing the subprocess and us to deadlock. So,
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   202
  # threads are used to keep the pipes safely flowing.
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   203
  stdout = StringIO.StringIO() if capture else log.FileLikeLogger()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   204
  out_adapter = _PipeAdapter(process.stdout, stdout)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   205
  err_adapter = _PipeAdapter(process.stderr, log.FileLikeLogger())
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   206
  if stdin:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   207
    process.stdin.write(stdin)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   208
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   209
  out_adapter.join()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   210
  err_adapter.join()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   211
  process.wait()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   212
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   213
  if process.returncode != 0:
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   214
    if capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   215
      raise SubprocessFailed('Process %s failed with output: %s' %
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   216
                             (argv[0], stdout.getvalue()))
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   217
    else:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   218
      raise SubprocessFailed('Process %s failed' % argv[0])
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   219
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   220
  if capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   221
    out = stdout.getvalue()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   222
    stdout.close()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   223
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   224
    if split_capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   225
      out = out.strip().split('\n')
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   226
    return out