thirdparty/google_appengine/google/appengine/api/labs/taskqueue/taskqueue_stub.py
author Pawel Solyga <Pawel.Solyga@gmail.com>
Sun, 06 Sep 2009 23:31:53 +0200
changeset 2864 2e0b0af889be
parent 2413 d0b7dac5325c
child 3031 7678f72140e6
permissions -rwxr-xr-x
Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     1
#!/usr/bin/env python
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     2
#
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     3
# Copyright 2007 Google Inc.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     4
#
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     5
# Licensed under the Apache License, Version 2.0 (the "License");
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     6
# you may not use this file except in compliance with the License.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     7
# You may obtain a copy of the License at
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     8
#
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
     9
#     http://www.apache.org/licenses/LICENSE-2.0
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    10
#
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    11
# Unless required by applicable law or agreed to in writing, software
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    12
# distributed under the License is distributed on an "AS IS" BASIS,
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    13
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    14
# See the License for the specific language governing permissions and
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    15
# limitations under the License.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    16
#
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    17
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    18
"""Stub version of the Task Queue API.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    19
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    20
This stub only stores tasks; it doesn't actually run them. It also validates
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    21
the tasks by checking their queue name against the queue.yaml.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    22
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    23
As well as implementing Task Queue API functions, the stub exposes various other
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    24
functions that are used by the dev_appserver's admin console to display the
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    25
application's queues and tasks.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    26
"""
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    27
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    28
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    29
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    30
import base64
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    31
import datetime
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    32
import os
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    33
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    34
import taskqueue_service_pb
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    35
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    36
from google.appengine.api import apiproxy_stub
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    37
from google.appengine.api import queueinfo
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    38
from google.appengine.api import urlfetch
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    39
from google.appengine.runtime import apiproxy_errors
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    40
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    41
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    42
DEFAULT_RATE = '5.00/s'
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    43
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    44
DEFAULT_BUCKET_SIZE = 5
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    45
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    46
MAX_ETA_DELTA_DAYS = 30
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    47
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    48
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    49
def _ParseQueueYaml(unused_self, root_path):
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    50
  """Loads the queue.yaml file and parses it.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    51
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    52
  Args:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    53
    unused_self: Allows this function to be bound to a class member. Not used.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    54
    root_path: Directory containing queue.yaml. Not used.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    55
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    56
  Returns:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    57
    None if queue.yaml doesn't exist, otherwise a queueinfo.QueueEntry object
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    58
    populaeted from the queue.yaml.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    59
  """
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    60
  if root_path is None:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    61
    return None
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    62
  for queueyaml in ('queue.yaml', 'queue.yml'):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    63
    try:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    64
      fh = open(os.path.join(root_path, queueyaml), 'r')
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    65
    except IOError:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    66
      continue
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    67
    try:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    68
      queue_info = queueinfo.LoadSingleQueue(fh)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    69
      return queue_info
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    70
    finally:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    71
      fh.close()
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    72
  return None
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    73
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    74
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    75
def _CompareTasksByEta(a, b):
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    76
  """Python sort comparator for tasks by estimated time of arrival (ETA).
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    77
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    78
  Args:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    79
    a: A taskqueue_service_pb.TaskQueueAddRequest.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    80
    b: A taskqueue_service_pb.TaskQueueAddRequest.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    81
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    82
  Returns:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    83
    Standard 1/0/-1 comparison result.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
    84
  """
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    85
  if a.eta_usec() > b.eta_usec():
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    86
    return 1
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    87
  if a.eta_usec() < b.eta_usec():
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    88
    return -1
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    89
  return 0
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    90
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    91
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    92
def _FormatEta(eta_usec):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    93
  """Formats a task ETA as a date string in UTC."""
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    94
  eta = datetime.datetime.fromtimestamp(eta_usec/1000000)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    95
  return eta.strftime('%Y/%m/%d %H:%M:%S')
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    96
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    97
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    98
def _EtaDelta(eta_usec):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
    99
  """Formats a task ETA as a relative time string."""
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   100
  eta = datetime.datetime.fromtimestamp(eta_usec/1000000)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   101
  now = datetime.datetime.utcnow()
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   102
  if eta > now:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   103
    return str(eta - now) + ' from now'
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   104
  else:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   105
    return str(now - eta) + ' ago'
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   106
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   107
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   108
class TaskQueueServiceStub(apiproxy_stub.APIProxyStub):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   109
  """Python only task queue service stub.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   110
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   111
  This stub does not attempt to automatically execute tasks.  Instead, it
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   112
  stores them for display on a console.  The user may manually execute the
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   113
  tasks from the console.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   114
  """
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   115
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   116
  queue_yaml_parser = _ParseQueueYaml
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   117
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   118
  def __init__(self, service_name='taskqueue', root_path=None):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   119
    """Constructor.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   120
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   121
    Args:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   122
      service_name: Service name expected for all calls.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   123
      root_path: Root path to the directory of the application which may contain
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   124
        a queue.yaml file. If None, then it's assumed no queue.yaml file is
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   125
        available.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   126
    """
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   127
    super(TaskQueueServiceStub, self).__init__(service_name)
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   128
    self._taskqueues = {}
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   129
    self._next_task_id = 1
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   130
    self._root_path = root_path
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   131
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   132
  def _Dynamic_Add(self, request, response):
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   133
    """Local implementation of the Add RPC in TaskQueueService.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   134
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   135
    Must adhere to the '_Dynamic_' naming convention for stubbing to work.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   136
    See taskqueue_service.proto for a full description of the RPC.
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   137
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   138
    Args:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   139
      request: A taskqueue_service_pb.TaskQueueAddRequest.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   140
      response: A taskqueue_service_pb.TaskQueueAddResponse.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   141
    """
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   142
    if request.eta_usec() < 0:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   143
      raise apiproxy_errors.ApplicationError(
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   144
          taskqueue_service_pb.TaskQueueServiceError.INVALID_ETA)
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   145
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   146
    eta = datetime.datetime.utcfromtimestamp(request.eta_usec() / 1e6)
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   147
    max_eta = (datetime.datetime.utcnow() +
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   148
               datetime.timedelta(days=MAX_ETA_DELTA_DAYS))
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   149
    if eta > max_eta:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   150
      raise apiproxy_errors.ApplicationError(
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   151
          taskqueue_service_pb.TaskQueueServiceError.INVALID_ETA)
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   152
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   153
    if not self._IsValidQueue(request.queue_name()):
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   154
      raise apiproxy_errors.ApplicationError(
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   155
          taskqueue_service_pb.TaskQueueServiceError.UNKNOWN_QUEUE)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   156
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   157
    if not request.task_name():
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   158
      request.set_task_name('task%d' % self._next_task_id)
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   159
      response.set_chosen_task_name(request.task_name())
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   160
      self._next_task_id += 1
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   161
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   162
    tasks = self._taskqueues.setdefault(request.queue_name(), [])
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   163
    for task in tasks:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   164
      if task.task_name() == request.task_name():
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   165
        raise apiproxy_errors.ApplicationError(
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   166
            taskqueue_service_pb.TaskQueueServiceError.TASK_ALREADY_EXISTS)
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   167
    tasks.append(request)
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   168
    tasks.sort(_CompareTasksByEta)
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   169
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   170
  def _IsValidQueue(self, queue_name):
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   171
    """Determines whether a queue is valid, i.e. tasks can be added to it.
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   172
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   173
    Valid queues are the 'default' queue, plus any queues in the queue.yaml
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   174
    file.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   175
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   176
    Args:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   177
      queue_name: the name of the queue to validate.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   178
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   179
    Returns:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   180
      True iff queue is valid.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   181
    """
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   182
    if queue_name == 'default':
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   183
      return True
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   184
    queue_info = self.queue_yaml_parser(self._root_path)
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   185
    if queue_info and queue_info.queue:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   186
      for entry in queue_info.queue:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   187
        if entry.name == queue_name:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   188
          return True
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   189
    return False
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   190
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   191
  def GetQueues(self):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   192
    """Gets all the applications's queues.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   193
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   194
    Returns:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   195
      A list of dictionaries, where each dictionary contains one queue's
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   196
      attributes. E.g.:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   197
        [{'name': 'some-queue',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   198
          'max_rate': '1/s',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   199
          'bucket_size': 5,
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   200
          'oldest_task': '2009/02/02 05:37:42',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   201
          'eta_delta': '0:00:06.342511 ago',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   202
          'tasks_in_queue': 12}, ...]
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   203
    """
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   204
    queues = []
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   205
    queue_info = self.queue_yaml_parser(self._root_path)
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   206
    has_default = False
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   207
    if queue_info and queue_info.queue:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   208
      for entry in queue_info.queue:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   209
        if entry.name == 'default':
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   210
          has_default = True
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   211
        queue = {}
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   212
        queues.append(queue)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   213
        queue['name'] = entry.name
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   214
        queue['max_rate'] = entry.rate
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   215
        if entry.bucket_size:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   216
          queue['bucket_size'] = entry.bucket_size
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   217
        else:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   218
          queue['bucket_size'] = DEFAULT_BUCKET_SIZE
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   219
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   220
        tasks = self._taskqueues.setdefault(entry.name, [])
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   221
        if tasks:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   222
          queue['oldest_task'] = _FormatEta(tasks[0].eta_usec())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   223
          queue['eta_delta'] = _EtaDelta(tasks[0].eta_usec())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   224
        else:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   225
          queue['oldest_task'] = ''
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   226
        queue['tasks_in_queue'] = len(tasks)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   227
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   228
    if not has_default:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   229
      queue = {}
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   230
      queues.append(queue)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   231
      queue['name'] = 'default'
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   232
      queue['max_rate'] = DEFAULT_RATE
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   233
      queue['bucket_size'] = DEFAULT_BUCKET_SIZE
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   234
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   235
      tasks = self._taskqueues.get('default', [])
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   236
      if tasks:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   237
        queue['oldest_task'] = _FormatEta(tasks[0].eta_usec())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   238
        queue['eta_delta'] = _EtaDelta(tasks[0].eta_usec())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   239
      else:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   240
        queue['oldest_task'] = ''
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   241
      queue['tasks_in_queue'] = len(tasks)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   242
    return queues
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   243
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   244
  def GetTasks(self, queue_name):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   245
    """Gets a queue's tasks.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   246
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   247
    Args:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   248
      queue_name: Queue's name to return tasks for.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   249
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   250
    Returns:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   251
      A list of dictionaries, where each dictionary contains one task's
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   252
      attributes. E.g.
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   253
        [{'name': 'task-123',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   254
          'url': '/update',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   255
          'method': 'GET',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   256
          'eta': '2009/02/02 05:37:42',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   257
          'eta_delta': '0:00:06.342511 ago',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   258
          'body': '',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   259
          'headers': {'X-AppEngine-QueueName': 'update-queue',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   260
                      'X-AppEngine-TaskName': 'task-123',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   261
                      'X-AppEngine-TaskRetryCount': '0',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   262
                      'X-AppEngine-Development-Payload': '1',
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   263
                      'Content-Length': 0,
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   264
                      'Content-Type': 'application/octet-streamn'}, ...]
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   265
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   266
    Raises:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   267
      ValueError: A task request contains an unknown HTTP method type.
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   268
    """
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   269
    tasks = self._taskqueues.get(queue_name, [])
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   270
    result_tasks = []
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   271
    for task_request in tasks:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   272
      task = {}
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   273
      result_tasks.append(task)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   274
      task['name'] = task_request.task_name()
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   275
      task['url'] = task_request.url()
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   276
      method = task_request.method()
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   277
      if method == taskqueue_service_pb.TaskQueueAddRequest.GET:
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   278
        task['method'] = 'GET'
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   279
      elif method == taskqueue_service_pb.TaskQueueAddRequest.POST:
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   280
        task['method'] = 'POST'
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   281
      elif method == taskqueue_service_pb.TaskQueueAddRequest.HEAD:
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   282
        task['method'] = 'HEAD'
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   283
      elif method == taskqueue_service_pb.TaskQueueAddRequest.PUT:
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   284
        task['method'] = 'PUT'
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   285
      elif method == taskqueue_service_pb.TaskQueueAddRequest.DELETE:
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   286
        task['method'] = 'DELETE'
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   287
      else:
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   288
        raise ValueError('Unexpected method: %d' % method)
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   289
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   290
      task['eta'] = _FormatEta(task_request.eta_usec())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   291
      task['eta_delta'] = _EtaDelta(task_request.eta_usec())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   292
      task['body'] = base64.b64encode(task_request.body())
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   293
      headers = urlfetch._CaselessDict()
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   294
      task['headers'] = headers
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   295
      for req_header in task_request.header_list():
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   296
        headers[req_header.key()] = req_header.value()
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   297
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   298
      headers['X-AppEngine-QueueName'] = queue_name
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   299
      headers['X-AppEngine-TaskName'] = task['name']
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   300
      headers['X-AppEngine-TaskRetryCount'] = '0'
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   301
      headers['X-AppEngine-Development-Payload'] = '1'
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   302
      headers['Content-Length'] = len(task['body'])
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   303
      headers['Content-Type'] = headers.get(
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   304
          'Content-Type', 'application/octet-stream')
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   305
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   306
    return result_tasks
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   307
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   308
  def DeleteTask(self, queue_name, task_name):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   309
    """Deletes a task from a queue.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   310
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   311
    Args:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   312
      queue_name: the name of the queue to delete the task from.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   313
      task_name: the name of the task to delete.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   314
    """
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   315
    tasks = self._taskqueues.get(queue_name, [])
2413
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   316
    for task in tasks:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   317
      if task.task_name() == task_name:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   318
        tasks.remove(task)
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   319
        return
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   320
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   321
  def FlushQueue(self, queue_name):
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   322
    """Removes all tasks from a queue.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   323
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   324
    Args:
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   325
      queue_name: the name of the queue to remove tasks from.
d0b7dac5325c Update GAE to 1.2.3 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff changeset
   326
    """
2864
2e0b0af889be Update Google App Engine from 1.2.3 to 1.2.5 in thirdparty folder.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2413
diff changeset
   327
    self._taskqueues[queue_name] = []