scripts/release/util.py
author Mario Ferraro <fadinlight@gmail.com>
Sun, 15 Nov 2009 22:12:20 +0100
changeset 3093 d1be59b6b627
parent 1888 ef350db7f753
permissions -rw-r--r--
GMaps related JS changed to use new google namespace. Google is going to change permanently in the future the way to load its services, so better stay safe. Also this commit shows uses of the new melange.js module. Fixes Issue 634.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     1
# Copyright 2009 the Melange authors.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     2
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     3
# Licensed under the Apache License, Version 2.0 (the "License");
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     4
# you may not use this file except in compliance with the License.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     5
# You may obtain a copy of the License at
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     6
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     7
#   http://www.apache.org/licenses/LICENSE-2.0
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     8
#
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
     9
# Unless required by applicable law or agreed to in writing, software
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    10
# distributed under the License is distributed on an "AS IS" BASIS,
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    11
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    12
# See the License for the specific language governing permissions and
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    13
# limitations under the License.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    14
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    15
"""Various utilities.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    16
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    17
Current contents:
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    18
 - Text colorization using ANSI color codes
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    19
 - A class to construct and manage paths under a root path.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    20
 - A helper to manage running subprocesses, wrapping the subprocess module.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    21
"""
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    22
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    23
__authors__ = [
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    24
    # alphabetical order by last name, please
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    25
    '"David Anderson" <dave@natulte.net>',
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    26
    ]
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    27
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
    28
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    29
import os.path
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    30
import re
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    31
try:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    32
  import cStringIO as StringIO
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    33
except ImportError:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    34
  import StringIO
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    35
import subprocess
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    36
import threading
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    37
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    38
import error
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
    39
import log
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    40
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    41
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    42
class Error(error.Error):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    43
  pass
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    44
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    45
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
    46
class SubprocessFailed(Error):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    47
  """A subprocess returned a non-zero error code."""
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
    48
  pass
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    49
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    50
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    51
# The magic escape sequence understood by modern terminal emulators to
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    52
# configure fore/background colors and other basic text display
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    53
# settings.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    54
_ANSI_ESCAPE = '\x1b[%dm'
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    55
_ANSI_ESCAPE_RE = re.compile(r'\x1b\[\d+m')
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    56
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    57
1826
12de6d73a908 Followup to r2496. Fix obvious errors.
David Anderson <david.jc.anderson@gmail.com>
parents: 1825
diff changeset
    58
# Some internal non-color settings that we use.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    59
_RESET = 0  # Reset to terminal defaults.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    60
_BOLD = 1   # Brighter colors.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    61
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    62
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    63
# ANSI color codes.
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    64
RED = 31
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    65
GREEN = 32
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    66
WHITE = 37
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    67
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    68
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    69
def _ansi_escape(code):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    70
  return _ANSI_ESCAPE % code
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    71
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    72
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    73
def colorize(text, color, bold=False):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    74
  """Colorize some text using ANSI color codes.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    75
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    76
  Note that while ANSI color codes look good in a terminal they look
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    77
  like noise in log files unless viewed in an ANSI color capable
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    78
  viewer (such as 'less -R').
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    79
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    80
  Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    81
    text: The text to colorize.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    82
    color: One of the color symbols from this module.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    83
    bold: If True, make the color brighter.
1824
c54c304e3c0e Refactor ANSI colorization into a new utility module.
David Anderson <david.jc.anderson@gmail.com>
parents:
diff changeset
    84
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    85
  Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    86
    The input text string, appropriately sprinkled with color
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    87
    codes. Colors are reset to terminal defaults after the input
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    88
    text.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    89
  """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    90
  bold = _ansi_escape(_BOLD) if bold else ''
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    91
  return '%s%s%s%s' % (bold, _ansi_escape(color),
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    92
                       text, _ansi_escape(_RESET))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    93
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
    94
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    95
def decolorize(text):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    96
  """Remove ANSI color codes from text."""
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
    97
  return _ANSI_ESCAPE_RE.sub('', text)
1833
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    98
9df2e9a67081 Add a decolorize() helper to undo the effects of colorize().
David Anderson <david.jc.anderson@gmail.com>
parents: 1827
diff changeset
    99
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   100
class Paths(object):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   101
  """A helper to construct and check paths under a given root."""
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   102
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   103
  def __init__(self, root):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   104
    """Initializer.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   105
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   106
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   107
      root: The root of all paths this instance will consider.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   108
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   109
    self._root = os.path.abspath(
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   110
      os.path.expandvars(os.path.expanduser(root)))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   111
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   112
  def path(self, path=''):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   113
    """Construct and return a path under the path root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   114
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   115
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   116
      path: The desired path string relative to the root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   117
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   118
    Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   119
      The absolute path corresponding to the relative input path.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   120
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   121
    assert not os.path.isabs(path)
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   122
    return os.path.abspath(os.path.join(self._root, path))
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   123
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   124
  def exists(self, path=''):
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   125
    """Check for the existence of a path under the path root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   126
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   127
    Does not discriminate on the path type (ie. it could be a
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   128
    directory, a file, a symbolic link...), just checks for the
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   129
    existence of the path.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   130
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   131
    Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   132
      path: The path string relative to the root.
1825
a610a2df83d2 Move the Paths class to the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1824
diff changeset
   133
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   134
    Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   135
      True if the path exists, False otherwise.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   136
    """
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   137
    return os.path.exists(self.path(path))
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   138
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   139
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   140
class _PipeAdapter(threading.Thread):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   141
  """A thread that connects one file-like object to another"""
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   142
  def __init__(self, pipe, logfile):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   143
    threading.Thread.__init__(self)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   144
    self.pipe, self.logfile = pipe, logfile
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   145
    self.setDaemon(True)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   146
    self.start()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   147
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   148
  def run(self):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   149
    try:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   150
      while True:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   151
        data = self.pipe.read(512)  # Small to retain interactivity
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   152
        if not data:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   153
          return
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   154
        self.logfile.write(data)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   155
    except (EOFError, OSError):
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   156
      pass
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   157
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   158
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   159
def run(argv, cwd=None, capture=False, split_capture=True, stdin=None):
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   160
  """Run the given command and optionally return its output.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   161
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   162
  Note that if you set capture=True, the command's output is
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   163
  buffered in memory. Output capture should only be used with
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   164
  commands that output small amounts of data. O(kB) is fine, O(MB)
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   165
  is starting to push it a little.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   166
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   167
  Args:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   168
    argv: A list containing the name of the program to run, followed
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   169
      by its argument vector.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   170
    cwd: Run the program from this directory.
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   171
    capture: If True, capture the program's stdout stream. If False,
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   172
      stdout will output to sys.stdout.
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   173
    split_capture: If True, return the captured output as a list of
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   174
      lines. Else, return as a single unaltered string.
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   175
    stdin: The string to feed to the program's stdin stream.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   176
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   177
  Returns:
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   178
    If capture is True, a string containing the combined
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   179
    stdout/stderr output of the program. If capture is False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   180
    nothing is returned.
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   181
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   182
  Raises:
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   183
    SubprocessFailed: The subprocess exited with a non-zero exit code.
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   184
  """
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   185
  log.debug(colorize('# ' + ' '.join(argv), WHITE, bold=True))
1827
c03995a6a88e Move run() into the util module.
David Anderson <david.jc.anderson@gmail.com>
parents: 1826
diff changeset
   186
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   187
  process = subprocess.Popen(argv,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   188
                             shell=False,
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   189
                             cwd=cwd,
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   190
                             stdin=(subprocess.PIPE if stdin else None),
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   191
                             stdout=subprocess.PIPE,
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   192
                             stderr=subprocess.PIPE)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   193
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   194
  # Spin up threads to capture stdout and stderr. Depending on the
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   195
  # value of the capture parameter, stdout is pushed either into the
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   196
  # log or into a string for processing. stderr always goes
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   197
  # into the log.
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   198
  #
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   199
  # Threads are necessary because all of writing to stdin, reading
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   200
  # from stdout and reading from stderr must all happen
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   201
  # simultaneously. Otherwise, there is a risk that one of the pipes
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   202
  # will fill up, causing the subprocess and us to deadlock. So,
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   203
  # threads are used to keep the pipes safely flowing.
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   204
  stdout = StringIO.StringIO() if capture else log.FileLikeLogger()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   205
  out_adapter = _PipeAdapter(process.stdout, stdout)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   206
  err_adapter = _PipeAdapter(process.stderr, log.FileLikeLogger())
1888
ef350db7f753 Style fixes for release modules.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 1846
diff changeset
   207
  
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   208
  if stdin:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   209
    process.stdin.write(stdin)
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   210
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   211
  out_adapter.join()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   212
  err_adapter.join()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   213
  process.wait()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   214
1835
3f30b7b14c57 Fix indentation to match Melange/Google style.
David Anderson <david.jc.anderson@gmail.com>
parents: 1833
diff changeset
   215
  if process.returncode != 0:
1846
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   216
    if capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   217
      raise SubprocessFailed('Process %s failed with output: %s' %
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   218
                             (argv[0], stdout.getvalue()))
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   219
    else:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   220
      raise SubprocessFailed('Process %s failed' % argv[0])
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   221
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   222
  if capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   223
    out = stdout.getvalue()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   224
    stdout.close()
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   225
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   226
    if split_capture:
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   227
      out = out.strip().split('\n')
ac30e04bcbba Redirect stdout/stderr through the logging system.
David Anderson <david.jc.anderson@gmail.com>
parents: 1835
diff changeset
   228
    return out