scripts/release/util.py
author Sverre Rabbelier <sverre@rabbelier.nl>
Thu, 04 Jun 2009 22:26:24 +0200
changeset 2392 b53e1cdb0398
parent 1888 ef350db7f753
permissions -rw-r--r--
Make the maintenance page a hardcoded response Show the maintenance message on the requested page, instead of redirecting to a specific url which then displays a page with the maintenance notice.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     1
# Copyright 2009 the Melange authors.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     2
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     3
# Licensed under the Apache License, Version 2.0 (the "License");
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     4
# you may not use this file except in compliance with the License.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     5
# You may obtain a copy of the License at
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     6
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     7
#   http://www.apache.org/licenses/LICENSE-2.0
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     8
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     9
# Unless required by applicable law or agreed to in writing, software
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    10
# distributed under the License is distributed on an "AS IS" BASIS,
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    11
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    12
# See the License for the specific language governing permissions and
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    13
# limitations under the License.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    14
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    15
"""Various utilities.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    16
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    17
Current contents:
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    18
 - Text colorization using ANSI color codes
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    19
 - A class to construct and manage paths under a root path.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    20
 - A helper to manage running subprocesses, wrapping the subprocess module.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    21
"""
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    22
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    23
__authors__ = [
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    24
    # alphabetical order by last name, please
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    25
    '"David Anderson" <dave@natulte.net>',
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    26
    ]
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    27
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
    28
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    29
import os.path
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    30
import re
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    31
try:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    32
  import cStringIO as StringIO
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    33
except ImportError:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    34
  import StringIO
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    35
import subprocess
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    36
import threading
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    37
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    38
import error
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    39
import log
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    40
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    41
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    42
class Error(error.Error):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    43
  pass
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    44
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    45
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    46
class SubprocessFailed(Error):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    47
  """A subprocess returned a non-zero error code."""
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
    48
  pass
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    49
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    50
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    51
# The magic escape sequence understood by modern terminal emulators to
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    52
# configure fore/background colors and other basic text display
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    53
# settings.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    54
_ANSI_ESCAPE = '\x1b[%dm'
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    55
_ANSI_ESCAPE_RE = re.compile(r'\x1b\[\d+m')
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    56
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    57
1826
12de6d73a908 Followup to r2496. Fix obvious errors.
David Anderson <david.jc.anderson@gmail.com>
parents: 1825
diff changeset
    58
# Some internal non-color settings that we use.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    59
_RESET = 0  # Reset to terminal defaults.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    60
_BOLD = 1   # Brighter colors.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    61
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    62
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    63
# ANSI color codes.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    64
RED = 31
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    65
GREEN = 32
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    66
WHITE = 37
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    67
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    68
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    69
def _ansi_escape(code):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    70
  return _ANSI_ESCAPE % code
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    71
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    72
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    73
def colorize(text, color, bold=False):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    74
  """Colorize some text using ANSI color codes.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    75
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    76
  Note that while ANSI color codes look good in a terminal they look
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    77
  like noise in log files unless viewed in an ANSI color capable
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    78
  viewer (such as 'less -R').
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    79
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    80
  Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    81
    text: The text to colorize.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    82
    color: One of the color symbols from this module.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    83
    bold: If True, make the color brighter.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    84
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    85
  Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    86
    The input text string, appropriately sprinkled with color
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    87
    codes. Colors are reset to terminal defaults after the input
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    88
    text.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    89
  """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    90
  bold = _ansi_escape(_BOLD) if bold else ''
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    91
  return '%s%s%s%s' % (bold, _ansi_escape(color),
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    92
                       text, _ansi_escape(_RESET))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    93
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    94
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    95
def decolorize(text):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    96
  """Remove ANSI color codes from text."""
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    97
  return _ANSI_ESCAPE_RE.sub('', text)
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    98
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    99
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   100
class Paths(object):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   101
  """A helper to construct and check paths under a given root."""
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   102
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   103
  def __init__(self, root):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   104
    """Initializer.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   105
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   106
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   107
      root: The root of all paths this instance will consider.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   108
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   109
    self._root = os.path.abspath(
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   110
      os.path.expandvars(os.path.expanduser(root)))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   111
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   112
  def path(self, path=''):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   113
    """Construct and return a path under the path root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   114
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   115
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   116
      path: The desired path string relative to the root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   117
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   118
    Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   119
      The absolute path corresponding to the relative input path.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   120
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   121
    assert not os.path.isabs(path)
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   122
    return os.path.abspath(os.path.join(self._root, path))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   123
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   124
  def exists(self, path=''):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   125
    """Check for the existence of a path under the path root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   126
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   127
    Does not discriminate on the path type (ie. it could be a
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   128
    directory, a file, a symbolic link...), just checks for the
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   129
    existence of the path.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   130
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   131
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   132
      path: The path string relative to the root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   133
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   134
    Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   135
      True if the path exists, False otherwise.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   136
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   137
    return os.path.exists(self.path(path))
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   138
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   139
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   140
class _PipeAdapter(threading.Thread):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   141
  """A thread that connects one file-like object to another"""
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   142
  def __init__(self, pipe, logfile):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   143
    threading.Thread.__init__(self)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   144
    self.pipe, self.logfile = pipe, logfile
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   145
    self.setDaemon(True)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   146
    self.start()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   147
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   148
  def run(self):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   149
    try:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   150
      while True:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   151
        data = self.pipe.read(512)  # Small to retain interactivity
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   152
        if not data:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   153
          return
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   154
        self.logfile.write(data)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   155
    except (EOFError, OSError):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   156
      pass
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   157
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   158
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   159
def run(argv, cwd=None, capture=False, split_capture=True, stdin=None):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   160
  """Run the given command and optionally return its output.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   161
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   162
  Note that if you set capture=True, the command's output is
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   163
  buffered in memory. Output capture should only be used with
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   164
  commands that output small amounts of data. O(kB) is fine, O(MB)
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   165
  is starting to push it a little.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   166
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   167
  Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   168
    argv: A list containing the name of the program to run, followed
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   169
      by its argument vector.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   170
    cwd: Run the program from this directory.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   171
    capture: If True, capture the program's stdout stream. If False,
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   172
      stdout will output to sys.stdout.
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   173
    split_capture: If True, return the captured output as a list of
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   174
      lines. Else, return as a single unaltered string.
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   175
    stdin: The string to feed to the program's stdin stream.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   176
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   177
  Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   178
    If capture is True, a string containing the combined
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   179
    stdout/stderr output of the program. If capture is False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   180
    nothing is returned.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   181
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   182
  Raises:
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   183
    SubprocessFailed: The subprocess exited with a non-zero exit code.
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   184
  """
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   185
  log.debug(colorize('# ' + ' '.join(argv), WHITE, bold=True))
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   186
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   187
  process = subprocess.Popen(argv,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   188
                             shell=False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   189
                             cwd=cwd,
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   190
                             stdin=(subprocess.PIPE if stdin else None),
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   191
                             stdout=subprocess.PIPE,
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   192
                             stderr=subprocess.PIPE)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   193
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   194
  # Spin up threads to capture stdout and stderr. Depending on the
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   195
  # value of the capture parameter, stdout is pushed either into the
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   196
  # log or into a string for processing. stderr always goes
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   197
  # into the log.
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   198
  #
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   199
  # Threads are necessary because all of writing to stdin, reading
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   200
  # from stdout and reading from stderr must all happen
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   201
  # simultaneously. Otherwise, there is a risk that one of the pipes
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   202
  # will fill up, causing the subprocess and us to deadlock. So,
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   203
  # threads are used to keep the pipes safely flowing.
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   204
  stdout = StringIO.StringIO() if capture else log.FileLikeLogger()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   205
  out_adapter = _PipeAdapter(process.stdout, stdout)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   206
  err_adapter = _PipeAdapter(process.stderr, log.FileLikeLogger())
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   207
  
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   208
  if stdin:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   209
    process.stdin.write(stdin)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   210
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   211
  out_adapter.join()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   212
  err_adapter.join()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   213
  process.wait()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   214
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   215
  if process.returncode != 0:
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   216
    if capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   217
      raise SubprocessFailed('Process %s failed with output: %s' %
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   218
                             (argv[0], stdout.getvalue()))
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   219
    else:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   220
      raise SubprocessFailed('Process %s failed' % argv[0])
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   221
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   222
  if capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   223
    out = stdout.getvalue()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   224
    stdout.close()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   225
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   226
    if split_capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   227
      out = out.strip().split('\n')
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   228
    return out