scripts/stats.py
author Mario Ferraro <fadinlight@gmail.com>
Sun, 15 Nov 2009 22:12:20 +0100
changeset 3093 d1be59b6b627
parent 3016 63625e7e0cac
permissions -rwxr-xr-x
GMaps related JS changed to use new google namespace. Google is going to change permanently in the future the way to load its services, so better stay safe. Also this commit shows uses of the new melange.js module. Fixes Issue 634.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     1
#!/usr/bin/python2.5
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     2
#
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     3
# Copyright 2009 the Melange authors.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     4
#
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     5
# Licensed under the Apache License, Version 2.0 (the "License");
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     6
# you may not use this file except in compliance with the License.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     7
# You may obtain a copy of the License at
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     8
#
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
     9
#   http://www.apache.org/licenses/LICENSE-2.0
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    10
#
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    11
# Unless required by applicable law or agreed to in writing, software
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    12
# distributed under the License is distributed on an "AS IS" BASIS,
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    13
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    14
# See the License for the specific language governing permissions and
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    15
# limitations under the License.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    16
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    17
"""Starts an interactive shell with statistic helpers.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    18
"""
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    19
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    20
__authors__ = [
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    21
  '"Sverre Rabbelier" <sverre@rabbelier.nl>',
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    22
]
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    23
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    24
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    25
import cPickle
2079
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
    26
import datetime
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    27
import operator
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    28
import sys
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
    29
import time
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    30
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    31
import interactive
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    32
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    33
2058
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    34
def dateFetch(queryGen, last=None, batchSize=100):
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    35
  """Iterator that yields an entity in batches.
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    36
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    37
  Args:
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    38
    queryGen: should return a Query object
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    39
    last: used to .filter() for last_modified_on
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    40
    batchSize: how many entities to retrieve in one datastore call
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    41
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    42
  Retrieved from http://tinyurl.com/d887ll (AppEngine cookbook).
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    43
  """
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    44
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    45
  from google.appengine.ext import db
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    46
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    47
   # AppEngine will not fetch more than 1000 results
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    48
  batchSize = min(batchSize,1000)
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    49
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    50
  query = None
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    51
  done = False
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    52
  count = 0
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    53
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    54
  while not done:
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    55
    print count
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    56
    query = queryGen()
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    57
    query.order('last_modified_on')
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    58
    if last:
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    59
      query.filter("last_modified_on > ", last)
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    60
    results = query.fetch(batchSize)
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    61
    for result in results:
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    62
      count += 1
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    63
      yield result
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    64
    if batchSize > len(results):
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    65
      done = True
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    66
    else:
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    67
      last = results[-1].last_modified_on
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    68
773b13d86309 Add dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2052
diff changeset
    69
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    70
def addKey(target, fieldname):
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    71
  """Adds the key of the specified field.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    72
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    73
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    74
  result = target.copy()
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    75
  result['%s_key' % fieldname] = target[fieldname].key().name()
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    76
  return result
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    77
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    78
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
    79
def getEntities(model):
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
    80
  """Returns all entities as dictionary keyed on their id_or_name property.
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
    81
  """
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
    82
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
    83
  def wrapped():
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
    84
    gen = lambda: model.all()
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
    85
    it = interactive.deepFetch(gen)
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
    86
2592
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
    87
    entities = [(i.key().id_or_name(), i) for i in it]
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
    88
    return dict(entities)
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
    89
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
    90
  return wrapped
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
    91
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
    92
2062
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
    93
def getProps(last=None):
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    94
  """Returns all proposals as a list of dictionaries.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    95
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    96
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    97
  key_order = [
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
    98
      'link_id', 'scope_path', 'title', 'abstract', 'content',
2062
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
    99
      'additional_info', '_mentor', 'possible_mentors', 'score',
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
   100
      'status', '_org', 'created_on', 'last_modified_on']
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   101
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   102
  from soc.models.student_proposal import StudentProposal
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   103
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   104
  gen = lambda: StudentProposal.all()
2062
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
   105
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
   106
  it = dateFetch(gen, last)
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   107
2062
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
   108
  proposals = [(i.key().name(), i.toDict(key_order)) for i in it]
2079
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   109
  if proposals:
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   110
    last = i.last_modified_on # last modified entity
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   111
  else:
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   112
    last = datetime.datetime.now()
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   113
2079
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   114
  return dict(proposals), last
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   115
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   116
2079
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   117
def orgStats(target, orgs):
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   118
  """Retrieves org stats.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   119
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   120
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   121
  from soc.logic import dicts
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   122
2079
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   123
  orgs = [(v.key(), v) for k, v in orgs.iteritems()]
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   124
  orgs = dict(orgs)
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   125
2062
9c739b37c367 Refactor getProps to use dateFetch
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2061
diff changeset
   126
  grouped = dicts.groupby(target.values(), '_org')
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   127
2079
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   128
  grouped = [(orgs[k], v) for k, v in grouped.iteritems()]
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   129
  popularity = [(k.link_id, len(v)) for k, v in grouped]
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   130
086c60aad3d8 Minor tweaks to orgStats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2063
diff changeset
   131
  return dict(grouped), dict(popularity)
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   132
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   133
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   134
def countStudentsWithProposals():
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   135
  """Retrieves number of Students who have submitted at least one Student Proposal.
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   136
  """
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   137
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   138
  proposals = getStudentProposals()
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   139
  students = {}
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   140
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   141
  for proposal_key in proposals.keys():
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   142
    students[proposals[proposal_key].scope_path] = True
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   143
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   144
  return len(students)
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   145
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   146
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   147
def printPopularity(popularity):
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   148
  """Prints the popularity for the specified proposals.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   149
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   150
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   151
  g = operator.itemgetter(1)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   152
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   153
  for item in sorted(popularity.iteritems(), key=g, reverse=True):
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   154
    print "%s: %d" % item
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   155
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   156
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   157
def saveValues(values, saver):
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   158
  """Saves the specified popularities.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   159
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   160
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   161
  import logging
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   162
  from google.appengine.ext import db
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   163
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   164
  from soc.models.organization import Organization
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   165
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   166
  def txn(key, value):
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   167
    org = Organization.get_by_key_name(key)
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   168
    saver(org, value)
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   169
    org.put()
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   170
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   171
  for key, value in sorted(values.iteritems()):
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   172
    print key
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   173
    db.run_in_transaction_custom_retries(10, txn, key, value)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   174
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   175
  print "done"
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   176
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   177
2090
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   178
def addFollower(follower, proposals, add_public=True, add_private=True):
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   179
  """Adds a user as follower to the specified proposals.
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   180
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   181
  Args:
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   182
    follower: the User to add as follower
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   183
    proposals: a list with the StudnetProposals that should be subscribed to
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   184
    add_public: whether the user is subscribed to public updates
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   185
    add_private: whether the user should be subscribed to private updates
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   186
  """
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   187
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   188
  from soc.models.review_follower import ReviewFollower
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   189
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   190
  result = []
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   191
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   192
  for i in proposals:
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   193
     properties = {
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   194
       'user': follower,
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   195
       'link_id': follower.link_id,
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   196
       'scope': i,
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   197
       'scope_path': i.key().name(),
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   198
       'key_name': '%s/%s' % (i.key().name(), follower.link_id),
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   199
       'subscribed_public': add_public,
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   200
       'subscribed_private': add_private,
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   201
     }
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   202
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   203
     entity = ReviewFollower(**properties)
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   204
     result.append(entity)
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   205
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   206
  return result
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   207
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   208
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   209
def convertProposals(org):
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   210
  """Convert all proposals for the specified organization.
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   211
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   212
  Args:
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   213
    org: the organization for which all proposals will be converted
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   214
  """
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   215
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   216
  from soc.logic.models.student_proposal import logic as proposal_logic
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   217
  from soc.logic.models.student_project import logic as project_logic
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   219
  proposals = proposal_logic.getProposalsToBeAcceptedForOrg(org)
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   220
2262
5b0576dcc107 Set batch size to 10
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2245
diff changeset
   221
  print "accepting %d proposals, with %d slots" % (len(proposals), org.slots)
5b0576dcc107 Set batch size to 10
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2245
diff changeset
   222
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   223
  for proposal in proposals:
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   224
    fields = {
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   225
        'link_id': 't%i' % (int(time.time()*100)),
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   226
        'scope_path': proposal.org.key().id_or_name(),
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   227
        'scope': proposal.org,
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   228
        'program': proposal.program,
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   229
        'student': proposal.scope,
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   230
        'title': proposal.title,
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   231
        'abstract': proposal.abstract,
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   232
        'mentor': proposal.mentor,
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   233
        }
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   234
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   235
    project = project_logic.updateOrCreateFromFields(fields, silent=True)
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   236
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   237
    fields = {
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   238
        'status':'accepted',
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   239
        }
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   240
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   241
    proposal_logic.updateEntityProperties(proposal, fields, silent=True)
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   242
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   243
  fields = {
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   244
      'status': ['new', 'pending'],
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   245
      'org': org,
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   246
      }
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   247
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   248
  querygen = lambda: proposal_logic.getQueryForFields(fields)
2262
5b0576dcc107 Set batch size to 10
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2245
diff changeset
   249
  proposals = [i for i in interactive.deepFetch(querygen, batchSize=10)]
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   250
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   251
  print "rejecting %d proposals" % len(proposals)
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   252
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   253
  fields = {
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   254
      'status': 'rejected',
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   255
      }
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   256
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   257
  for proposal in proposals:
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   258
    proposal_logic.updateEntityProperties(proposal, fields, silent=True)
2218
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   259
84b0ce492cf5 Add a script to convert proposals.
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2151
diff changeset
   260
2234
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   261
def startSpam():
2270
543f11d7b0e4 Fix missing dot in stats.py docstring.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2265
diff changeset
   262
  """Creates the job that is responsible for sending mails.
2265
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   263
  """
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   264
2234
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   265
  from soc.logic.models.job import logic as job_logic
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   266
  from soc.logic.models.priority_group import logic as priority_logic
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   267
  from soc.logic.models.program import logic as program_logic
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   268
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   269
  program_entity = program_logic.getFromKeyName('google/gsoc2009')
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   270
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   271
  priority_group = priority_logic.getGroup(priority_logic.EMAIL)
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   272
  job_fields = {
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   273
      'priority_group': priority_group,
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   274
      'task_name': 'setupStudentProposalMailing',
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   275
      'key_data': [program_entity.key()]}
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   276
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   277
  job_logic.updateOrCreateFromFields(job_fields)
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   278
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   279
2345
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   280
def startUniqueUserIdConversion():
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   281
  """Creates the job that is responsible for adding unique user ids.
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   282
  """
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   283
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   284
  from soc.logic.models.job import logic as job_logic
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   285
  from soc.logic.models.priority_group import logic as priority_logic
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   286
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   287
  priority_group = priority_logic.getGroup(priority_logic.CONVERT)
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   288
  job_fields = {
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   289
      'priority_group': priority_group,
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   290
      'task_name': 'setupUniqueUserIdAdder'}
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   291
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   292
  job_logic.updateOrCreateFromFields(job_fields)
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   293
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   294
2265
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   295
def reviveJobs(amount):
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   296
  """Sets jobs that are stuck in 'aborted' to waiting.
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   297
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   298
  Args:
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   299
    amount: the amount of jobs to revive
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   300
  """
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   301
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   302
  from soc.models.job import Job
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   303
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   304
  query = Job.all().filter('status', 'aborted')
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   305
  jobs = query.fetch(amount)
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   306
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   307
  if not jobs:
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   308
    print "no dead jobs"
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   309
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   310
  for job in jobs:
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   311
     job.status = 'waiting'
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   312
     job.put()
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   313
     print "restarted %d" % job.key().id()
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   314
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   315
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   316
def deidleJobs(amount):
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   317
  """Sets jobs that are stuck in 'started' to waiting.
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   318
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   319
  Args:
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   320
    amount: the amount of jobs to deidle
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   321
  """
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   322
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   323
  from soc.models.job import Job
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   324
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   325
  query = Job.all().filter('status', 'started')
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   326
  jobs = query.fetch(amount)
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   327
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   328
  if not jobs:
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   329
    print "no idle jobs"
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   330
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   331
  for job in jobs:
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   332
     job.status = 'waiting'
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   333
     job.put()
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   334
     print "restarted %d" % job.key().id()
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   335
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   336
2245
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   337
def deleteEntities(model, step_size=25):
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   338
  """Deletes all entities of the specified type
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   339
  """
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   340
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   341
  print "Deleting..."
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   342
  count = 0
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   343
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   344
  while True:
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   345
    entities = model.all().fetch(step_size)
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   346
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   347
    if not entities:
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   348
      break
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   349
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   350
    for entity in entities:
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   351
      entity.delete()
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   352
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   353
    count += step_size
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   354
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   355
    print "deleted %d entities" % count
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   356
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   357
  print "Done"
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   358
2592
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   359
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   360
def loadPickle(name):
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   361
  """Loads a pickle.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   362
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   363
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   364
  f = open(name + '.dat')
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   365
  return cPickle.load(f)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   366
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   367
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   368
def dumpPickle(target, name):
2061
c7ba473b091e Style fixes
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2060
diff changeset
   369
  """Dumps a pickle.
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   370
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   371
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   372
  f = open("%s.dat" % name, 'w')
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   373
  cPickle.dump(target, f)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   374
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   375
2592
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   376
def addOrganizationToSurveyRecords(survey_record_model):
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   377
  """Set Organization in SurveyRecords entities of a given model.
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   378
  """
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   379
  
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   380
  print "Fetching %s." % survey_record_model.__name__
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   381
  getSurveyRecord = getEntities(survey_record_model)
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   382
  survey_records = getSurveyRecord()
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   383
  survey_records_amount = len(survey_records)
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   384
  print "Fetched %d %s." % (survey_records_amount, survey_record_model.__name__)
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   385
  
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   386
  counter = 0
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   387
  
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   388
  for key in survey_records.keys():
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   389
    survey_records[key].org = survey_records[key].project.scope
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   390
    survey_records[key].put()
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   391
    
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   392
    counter += 1
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   393
    print str(counter) + '/' + str(survey_records_amount) + ' ' + str(key)
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   394
    
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   395
  print "Organization added to all %s." % survey_record_model.__name__
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   396
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   397
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   398
def setOrganizationInSurveyRecords():
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   399
  """Sets Organization property in ProjectSurveyRecords 
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   400
  and GradingProjectSurveyRecords entities.
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   401
  """
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   402
  from soc.models.project_survey_record import ProjectSurveyRecord
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   403
  from soc.models.grading_project_survey_record \
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   404
      import GradingProjectSurveyRecord
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   405
  
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   406
  addOrganizationToSurveyRecords(ProjectSurveyRecord)
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   407
  addOrganizationToSurveyRecords(GradingProjectSurveyRecord)
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   408
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   409
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   410
def exportStudentsWithProjects(csv_filename, scope_path_start=''):
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   411
  """Exports all Students who have a project assigned.
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   412
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   413
  Args:
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   414
    csv_filename: the name of the file where to save the CSV export
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   415
    scope_path_start: The string with which the scope_path of the project
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   416
      should start with. Can be used to select which sponsor, program or org
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   417
      the projects should belong to.
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   418
  """
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   419
  # TODO(Pawel.Solyga): Add additional Program parameter to this method 
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   420
  # so we export students from different programs
2592
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   421
  # TODO(Pawel.Solyga): Make it universal so it works with both GHOP 
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   422
  # and GSoC programs
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   423
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   424
  from soc.models.student_project import StudentProject
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   425
  from soc.models.student import Student
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   426
  from soc.models.organization import Organization
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   427
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   428
  # get all projects
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   429
  getStudentProjects = getEntities(StudentProject)
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   430
  student_projects = getStudentProjects()
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   431
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   432
  student_projects_amount = len(student_projects)
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   433
  print "Fetched %d Student Projects." % student_projects_amount
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   434
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   435
  print "Fetching Student entities from Student Projects."
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   436
  accepted_students = {}
2642
1bb33f56a3f4 Add new project status column to accepted students csv export.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2592
diff changeset
   437
  student_extra_data = {}
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   438
  counter = 0
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   439
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   440
  for student_project in student_projects.values():
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   441
    counter += 1
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   442
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   443
    if student_project.status == 'invalid' or not \
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   444
        student_project.scope_path.startswith(scope_path_start):
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   445
      # no need to export this project
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   446
      continue
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   447
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   448
    student_entity = student_project.student
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   449
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   450
    student_key = student_entity.key().id_or_name()
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   451
    accepted_students[student_key] = student_entity
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   452
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   453
    org_name = student_project.scope.name
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   454
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   455
    extra_data = {}
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   456
    extra_data['organization'] = org_name
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   457
    extra_data['project_status'] = student_project.status
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   458
    student_extra_data[student_key] = extra_data
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   459
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   460
    print '%s/%s %s (%s)' %(counter, student_projects_amount,
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   461
                            student_key, org_name)
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   462
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   463
  print "All Student entities fetched."
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   464
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   465
  students_key_order = ['link_id', 'given_name', 'surname', 
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   466
      'document_name', 'email', 'res_street', 'res_city', 'res_state',
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   467
      'res_country', 'res_postalcode', 'phone', 'shipping_street',
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   468
      'shipping_city', 'shipping_state', 'shipping_country',
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   469
      'shipping_postalcode', 'birth_date', 'tshirt_size', 'tshirt_style',
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   470
      'school_name', 'school_country', 'major', 'degree']
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   471
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   472
  print "Preparing Students data for export."
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   473
  students_data = []
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   474
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   475
  for student_key, student_entity in accepted_students.iteritems():
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   476
    # transform the Student into a set of dict entries
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   477
    prepared_data = student_entity.toDict(students_key_order)
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   478
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   479
    # add the additional fields
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   480
    extra_data = student_extra_data[student_key]
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   481
    prepared_data['organization'] = extra_data['organization']
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   482
    prepared_data['project_status'] = extra_data['project_status']
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   483
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   484
    # append the prepared data to the collected data
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   485
    students_data.append(prepared_data)
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   486
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   487
  # append the extra fields to the key_order
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   488
  students_key_order.append('organization')
2642
1bb33f56a3f4 Add new project status column to accepted students csv export.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2592
diff changeset
   489
  students_key_order.append('project_status')
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   490
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   491
  saveDataToCSV(csv_filename, students_data, students_key_order)
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   492
  print "Students with Projects exported to %s file." % csv_filename
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   493
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   494
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   495
def exportUniqueOrgAdminsAndMentors(csv_filename, scope_path_start=''):
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   496
  """Exports Org Admins and Mentors to a CSV file, one per User.
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   497
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   498
  Args:
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   499
    csv_filename: the name of the csv file to save
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   500
    scope_path_start: the start of the scope path of the roles to get could be
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   501
      google/gsoc2009 if you want to export all GSoC 2009 Org Admins and
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   502
      Mentors.
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   503
  """
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   504
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   505
  from soc.models.mentor import Mentor
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   506
  from soc.models.org_admin import OrgAdmin
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   507
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   508
  print 'Retrieving all Mentors'
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   509
  mentors = getEntities(Mentor)()
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   510
  all_mentors = mentors.values()
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   511
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   512
  print 'Retrieving all Org Admins'
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   513
  org_admins = getEntities(OrgAdmin)()
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   514
  all_org_admins = org_admins.values()
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   515
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   516
  print 'Combining the list of Mentors and Org Admins'
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   517
  unique_users = {}
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   518
  all_users = []
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   519
  all_users.extend(all_mentors)
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   520
  all_users.extend(all_org_admins)
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   521
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   522
  for user in all_users:
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   523
    if not user.scope_path.startswith(scope_path_start) or \
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   524
        user.status == 'invalid':
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   525
      # not the correct program or valid user
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   526
      continue
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   527
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   528
    unique_users[user.link_id] = user
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   529
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   530
  export_fields = ['link_id', 'given_name', 'surname', 
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   531
      'document_name', 'email', 'res_street', 'res_city', 'res_state',
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   532
      'res_country', 'res_postalcode', 'phone', 'shipping_street',
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   533
      'shipping_city', 'shipping_state', 'shipping_country',
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   534
      'shipping_postalcode', 'birth_date', 'tshirt_size', 'tshirt_style']
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   535
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   536
  print 'Preparing the data for export'
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   537
  data = [user.toDict(field_names=export_fields) for user in \
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   538
          unique_users.values()]
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   539
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   540
  print 'Exporting the data to CSV'
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   541
  saveDataToCSV(csv_filename, data, export_fields)
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   542
  print "Exported Org admins and Mentors (1 per User) to %s file." % csv_filename
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   543
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   544
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   545
def saveDataToCSV(csv_filename, data, key_order):
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   546
  """Saves data in order into CSV file.
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   547
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   548
  This is a helper function used for exporting CSV data.
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   549
  
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   550
  Args:
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   551
    csv_filename: The name of the file where to save the CSV data
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   552
    data: the data dict to write to CSV
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   553
    key_order: the order in which to export the data in data dict
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   554
  """
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   555
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   556
  import csv
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   557
  import StringIO
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   558
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   559
  from soc.logic import dicts
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   560
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   561
  file_handler = StringIO.StringIO()
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   562
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   563
  # ignore the extra data
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   564
  writer = csv.DictWriter(file_handler, key_order, extrasaction='ignore', dialect='excel')
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   565
  writer.writerow(dicts.identity(key_order))
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   566
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   567
  # encode the data to UTF-8 to ensure compatibiliy
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   568
  for row_dict in data:
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   569
    for key in row_dict.keys():
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   570
      value = row_dict[key]
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   571
      if isinstance(value, basestring):
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   572
        row_dict[key] = value.encode("utf-8")
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   573
      else:
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   574
        row_dict[key] = str(value)
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   575
    writer.writerow(row_dict)
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   576
2322
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   577
  csv_data = file_handler.getvalue()
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   578
  csv_file = open(csv_filename, 'w')
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   579
  csv_file.write(csv_data)
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   580
  csv_file.close()
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   581
98fe07a5542f Add acceptedStudentsCSVExport function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2274
diff changeset
   582
3015
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   583
def exportOrgsForGoogleCode(csv_filename, gc_project_prefix='', 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   584
                            scope_path_start=''):
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   585
  """Export all Organizations from given program as CSV.
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   586
      
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   587
  CSV file will contain 3 columns: organization name, organization google 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   588
  code project name, organization description.
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   589
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   590
  Args:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   591
    csv_filename: the name of the csv file to save
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   592
    gc_project_prefix: Google Code project prefix for example
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   593
      could be google-summer-of-code-2009- for GSoC 2009
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   594
    scope_path_start: the start of the scope path of the roles to get could be
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   595
      google/gsoc2009 if you want to export all GSoC 2009 Organizations.
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   596
  """
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   597
  from soc.models.organization import Organization
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   598
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   599
  print 'Retrieving all Organizations'
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   600
  orgs = getEntities(Organization)()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   601
  orgs_export = []
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   602
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   603
  print 'Preparing data for CSV export'
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   604
  for key in orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   605
    if not orgs[key].scope_path.startswith(scope_path_start):
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   606
      continue
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   607
    
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   608
    org_for_export = {} 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   609
    org_short_name = orgs[key].short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   610
    org_short_name = org_short_name.replace(' ','-').replace('.', '')
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   611
    org_short_name = org_short_name.replace('/','-').replace('!','').lower()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   612
    org_for_export['org_description'] = orgs[key].description
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   613
    org_for_export['org_name'] = orgs[key].name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   614
    org_for_export['google_code_project_name'] = gc_project_prefix + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   615
        org_short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   616
    orgs_export.append(org_for_export)
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   617
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   618
  export_fields = ['org_name', 'google_code_project_name', 'org_description']
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   619
  print 'Exporting the data to CSV'
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   620
  saveDataToCSV(csv_filename, orgs_export, export_fields)
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   621
  print "Exported Organizations for Google Code to %s file." % csv_filename
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   622
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   623
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   624
def exportRolesForGoogleCode(csv_filename, gc_project_prefix='', 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   625
                             scope_path_start=''):
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   626
  """Export all Students/Mentors/OrgAdmins Roles from given program as CSV.
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   627
      
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   628
  CSV file will contain 3 columns: google_account, organization google 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   629
  code project name and role (project member or project owner).
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   630
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   631
  Args:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   632
    csv_filename: the name of the csv file to save
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   633
    gc_project_prefix: Google Code project prefix for example
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   634
      could be google-summer-of-code-2009- for GSoC 2009
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   635
    scope_path_start: the start of the scope path of the roles to get could be
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   636
      google/gsoc2009 if you want to export all GSoC 2009 Organizations.
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   637
  """
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   638
  from soc.models.org_admin import OrgAdmin
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   639
  from soc.models.student_project import StudentProject
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   640
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   641
  # get all projects
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   642
  getStudentProjects = getEntities(StudentProject)
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   643
  student_projects = getStudentProjects()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   644
  org_admins = getEntities(OrgAdmin)()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   645
  all_org_admins = org_admins.values()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   646
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   647
  org_admins_by_orgs = {}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   648
  students_by_orgs = {}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   649
  mentors_by_orgs = {}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   650
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   651
  for org_admin in all_org_admins:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   652
      
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   653
    if not org_admin.scope_path.startswith(scope_path_start):
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   654
      continue
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   655
      
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   656
    org_short_name = org_admin.scope.short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   657
    org_short_name = org_short_name.replace(' ','-').replace('.', '')
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   658
    org_short_name = org_short_name.replace('/','-').replace('!','').lower()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   659
    if gc_project_prefix + org_short_name not in org_admins_by_orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   660
      org_admins_by_orgs[gc_project_prefix + org_short_name] = []
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   661
    
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   662
    org_admins_by_orgs[gc_project_prefix + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   663
        org_short_name].append(str(org_admin.user.account))
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   664
    print 'OrgAdmin ' + str(org_admin.user.account) + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   665
        ' for ' + gc_project_prefix + org_short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   666
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   667
  for student_project in student_projects.values():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   668
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   669
    if student_project.status != 'accepted' or not \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   670
      student_project.scope_path.startswith(scope_path_start):
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   671
      # no need to export this project
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   672
      continue
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   673
    
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   674
    student_entity = student_project.student
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   675
    mentor_entity = student_project.mentor
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   676
    org_short_name = student_project.scope.short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   677
    org_short_name = org_short_name.replace(' ','-').replace('.', '')
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   678
    org_short_name = org_short_name.replace('/','-').replace('!','').lower()
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   679
    if gc_project_prefix + org_short_name not in students_by_orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   680
      students_by_orgs[gc_project_prefix + org_short_name] = []
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   681
    students_by_orgs[gc_project_prefix + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   682
        org_short_name].append(str(student_entity.user.account))
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   683
    print 'Student ' + str(student_entity.user.account) + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   684
        ' for ' + gc_project_prefix + org_short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   685
    if gc_project_prefix + org_short_name not in mentors_by_orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   686
      mentors_by_orgs[gc_project_prefix + org_short_name] = []
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   687
    mentors_by_orgs[gc_project_prefix + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   688
        org_short_name].append(str(mentor_entity.user.account))
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   689
    print 'Mentor ' + str(mentor_entity.user.account) + \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   690
        ' for ' + gc_project_prefix + org_short_name
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   691
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   692
  roles_data = {}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   693
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   694
  # prepare org admins data
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   695
  for org_key in org_admins_by_orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   696
    for org_admin in org_admins_by_orgs[org_key]:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   697
      roles_data[org_admin + '|' + org_key] = \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   698
          {'role': 'project_owner', 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   699
           'google_code_project_name': org_key, 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   700
           'google_account': org_admin}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   701
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   702
  # prepare mentors data
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   703
  for org_key in mentors_by_orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   704
    for mentor in mentors_by_orgs[org_key]:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   705
      if mentor + '|' + org_key not in roles_data.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   706
        roles_data[mentor + '|' + org_key] = \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   707
            {'role': 'project_member', 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   708
             'google_code_project_name': org_key, 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   709
             'google_account': mentor}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   710
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   711
  # prepare students data
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   712
  for org_key in students_by_orgs.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   713
    for student in students_by_orgs[org_key]:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   714
      roles_data[student + '|' + org_key] = \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   715
          {'role': 'project_member', 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   716
           'google_code_project_name': org_key, 
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   717
           'google_account': student}
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   718
   
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   719
  data = []
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   720
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   721
  # add @gmail.com to all accounts that don't have @ in the account string
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   722
  # gmail.com is authorized domain for Google AppEngine that's why it's
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   723
  # missing
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   724
  for roles_key in roles_data.keys():
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   725
    if roles_data[roles_key]['google_account'].find('@') == -1:
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   726
      roles_data[roles_key]['google_account'] = \
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   727
        roles_data[roles_key]['google_account'] + '@gmail.com'
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   728
    data.append(roles_data[roles_key])
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   729
  
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   730
  export_fields = ['google_account', 'google_code_project_name', 'role']
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   731
  print 'Exporting the data to CSV'
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   732
  saveDataToCSV(csv_filename, data, export_fields)
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   733
  print "Exported Roles for Google Code to %s file." % csv_filename
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   734
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   735
3016
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   736
def entityToDict(entity, field_names=None):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   737
  """Returns a dict with all specified values of this entity.
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   738
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   739
  Args:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   740
    entity: entity that will be converted to dictionary
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   741
    field_names: the fields that should be included, defaults to
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   742
      all fields that are of a type that is in DICT_TYPES.
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   743
  """
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   744
  from google.appengine.ext import db
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   745
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   746
  DICT_TYPES = (db.StringProperty, db.IntegerProperty)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   747
  result = {}
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   748
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   749
  if not field_names:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   750
    props = entity.properties().iteritems()
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   751
    field_names = [k for k, v in props if isinstance(v, DICT_TYPES)]
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   752
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   753
  for key in field_names:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   754
    # Skip everything that is not valid
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   755
    if not hasattr(entity, key):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   756
      continue
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   757
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   758
    result[key] = getattr(entity, key)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   759
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   760
  if hasattr(entity, 'name'):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   761
    name_prop = getattr(entity, 'name')
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   762
    if callable(name_prop):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   763
      result['name'] = name_prop()
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   764
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   765
  return result
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   766
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   767
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   768
def surveyRecordCSVExport(csv_filename, survey_record_model, 
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   769
                          survey_model, survey_key):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   770
  """CSV export for Survey Records for selected survey type and given survey key.
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   771
  
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   772
  Args:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   773
    csv_filename: the name of the csv file to save
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   774
    survey_record_model: model of surver record that will be exported
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   775
    survey_model: model of the survey that wil be exported
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   776
    survey_key: key of the survey that records will be exported
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   777
  """
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   778
  from soc.models.project_survey import ProjectSurvey
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   779
  from soc.models.grading_project_survey import GradingProjectSurvey
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   780
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   781
  # fetch survey
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   782
  survey = survey_model.get(survey_key)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   783
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   784
  if not survey:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   785
    print "Survey of given model and key doesn't exist."
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   786
    return
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   787
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   788
  schema = eval(survey.survey_content.schema)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   789
  ordered_properties = survey.survey_content.orderedProperties()
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   790
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   791
  getSurveyRecords = getEntities(survey_record_model)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   792
  survey_records = getSurveyRecords()
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   793
  survey_records_amount = len(survey_records)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   794
  print "Fetched %d Survey Records." % survey_records_amount
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   795
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   796
  count = 0
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   797
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   798
  print "Preparing SurveyRecord data for export."
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   799
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   800
  sr_key = {}
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   801
  comments_properties = []
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   802
  for property_name in ordered_properties:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   803
    sr_key[property_name] = schema[property_name]['question']
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   804
    if schema[property_name]['has_comment']:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   805
      sr_key['comment_for_' + property_name] = 'None'
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   806
      comments_properties.append('comment_for_' + property_name)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   807
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   808
  for comment in comments_properties:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   809
    ordered_properties.append(comment)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   810
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   811
  survey_record_data = []
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   812
  for i in survey_records.keys():
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   813
    if str(survey_records[i].survey.key()) != survey_key:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   814
        continue
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   815
    data = entityToDict(survey_records[i], ordered_properties)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   816
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   817
    if (survey_model == GradingProjectSurvey) or \
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   818
        (survey_model == ProjectSurvey):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   819
      data['organization'] = survey_records[i].org.name
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   820
      data['project_title'] = survey_records[i].project.title
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   821
      data['user_link_id'] = survey_records[i].user.link_id
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   822
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   823
      if survey_model == GradingProjectSurvey:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   824
        data['project_grade'] = survey_records[i].grade
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   825
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   826
    survey_record_data.append(data)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   827
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   828
    count += 1
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   829
    print str(count) + '/' + str(survey_records_amount) + ' ' + str(i)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   830
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   831
  if (survey_model == GradingProjectSurvey) or (survey_model == ProjectSurvey):
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   832
    ordered_properties.append('organization')
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   833
    ordered_properties.append('project_title')
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   834
    ordered_properties.append('user_link_id')
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   835
    sr_key['organization'] = 'None'
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   836
    sr_key['project_title'] = 'None'
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   837
    sr_key['user_link_id'] = 'None'
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   838
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   839
    if survey_model == GradingProjectSurvey:
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   840
      ordered_properties.append('project_grade')
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   841
      sr_key['project_grade'] = 'None'
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   842
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   843
  survey_record_data.insert(0, sr_key)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   844
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   845
  saveDataToCSV(csv_filename, survey_record_data, ordered_properties)
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   846
  print "Survey Records exported to %s file." % csv_filename
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   847
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   848
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   849
def main(args):
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   850
  """Main routine.
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   851
  """
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   852
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   853
  interactive.setup()
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   854
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   855
  from soc.models.organization import Organization
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   856
  from soc.models.user import User
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   857
  from soc.models.student import Student
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   858
  from soc.models.mentor import Mentor
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   859
  from soc.models.org_admin import OrgAdmin
2245
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   860
  from soc.models.job import Job
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   861
  from soc.models.student_proposal import StudentProposal
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   862
  from soc.models.student_project import StudentProject
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   863
2151
f58515b0b2e1 Style fix in function names in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2148
diff changeset
   864
  def slotSaver(org, value):
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   865
    org.slots = value
2151
f58515b0b2e1 Style fix in function names in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2148
diff changeset
   866
  def popSaver(org, value):
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   867
    org.nr_applications = value
2151
f58515b0b2e1 Style fix in function names in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2148
diff changeset
   868
  def rawSaver(org, value):
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   869
    org.slots_calculated = value
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   870
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   871
  context = {
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   872
      'load': loadPickle,
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   873
      'dump': dumpPickle,
2052
a723a2509e21 Add some additional functions for stats to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2047
diff changeset
   874
      'orgStats': orgStats,
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   875
      'printPopularity': printPopularity,
2148
0345046ed7a5 Make it possible to save any value with saveValues
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2090
diff changeset
   876
      'saveValues': saveValues,
2245
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   877
      'getEntities': getEntities,
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   878
      'deleteEntities': deleteEntities,
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   879
      'getOrgs': getEntities(Organization),
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   880
      'getUsers': getEntities(User),
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   881
      'getStudents': getEntities(Student),
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   882
      'getMentors': getEntities(Mentor),
2063
319d9f05955a Fixed a typo in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2062
diff changeset
   883
      'getOrgAdmins': getEntities(OrgAdmin),
2274
1885cda38bb7 Add getStudentsProjects to context in stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2270
diff changeset
   884
      'getStudentProjects': getEntities(StudentProject),
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   885
      'getProps': getProps,
2060
45029d87be4a Add and use a getEntities method
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2058
diff changeset
   886
      'countStudentsWithProposals': countStudentsWithProposals,
2592
040359ccd20b Add setOrganizationInSurveyRecords function to stats.py script.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2345
diff changeset
   887
      'setOrganizationInSurveyRecords': setOrganizationInSurveyRecords,
2222
a91d55e9c9cd Various improvements to convertProposals
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2218
diff changeset
   888
      'convertProposals': convertProposals,
2090
a07ff1e1bd1f Add a script to subscribe someone to all updates
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2089
diff changeset
   889
      'addFollower': addFollower,
2089
b8a9691da6e7 Add the entities to the context too
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2079
diff changeset
   890
      'Organization': Organization,
2245
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   891
      'Job': Job,
2089
b8a9691da6e7 Add the entities to the context too
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2079
diff changeset
   892
      'User': User,
b8a9691da6e7 Add the entities to the context too
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2079
diff changeset
   893
      'Student': Student,
b8a9691da6e7 Add the entities to the context too
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2079
diff changeset
   894
      'Mentor': Mentor,
b8a9691da6e7 Add the entities to the context too
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2079
diff changeset
   895
      'OrgAdmin': OrgAdmin,
2245
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   896
      'StudentProject': StudentProject,
46e490606f3d Added deleteEntities methods to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2234
diff changeset
   897
      'StudentProposal': StudentProposal,
2151
f58515b0b2e1 Style fix in function names in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2148
diff changeset
   898
      'slotSaver': slotSaver,
f58515b0b2e1 Style fix in function names in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2148
diff changeset
   899
      'popSaver': popSaver,
f58515b0b2e1 Style fix in function names in stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2148
diff changeset
   900
      'rawSaver': rawSaver,
2234
69539a5ca63a Add a startSpam method to stats
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2222
diff changeset
   901
      'startSpam': startSpam,
2265
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   902
      'reviveJobs': reviveJobs,
bfaadb6ab559 Added reviveJobs and deidleJobs to stats.py
Sverre Rabbelier <srabbelier@gmail.com>
parents: 2262
diff changeset
   903
      'deidleJobs': deidleJobs,
2770
71a5a56cf29e Redone the acceptedStudentsExport functionality.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2769
diff changeset
   904
      'exportStudentsWithProjects': exportStudentsWithProjects,
2769
279d26439495 Added export for Org Admins and Mentors, but only one per User.
Lennard de Rijk <ljvderijk@gmail.com>
parents: 2642
diff changeset
   905
      'exportUniqueOrgAdminsAndMentors': exportUniqueOrgAdminsAndMentors,
3015
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   906
      'exportOrgsForGoogleCode': exportOrgsForGoogleCode,
ad05c8063e37 Add functions for CSV export of organizations and roles.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2770
diff changeset
   907
      'exportRolesForGoogleCode': exportRolesForGoogleCode,
2345
f78caf12f32d Add helper functions, model update and jobs for unique user ids.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 2322
diff changeset
   908
      'startUniqueUserIdConversion': startUniqueUserIdConversion,
3016
63625e7e0cac Add function for CSV export of survey records for a given survey.
Pawel Solyga <Pawel.Solyga@gmail.com>
parents: 3015
diff changeset
   909
      'surveyRecordCSVExport': surveyRecordCSVExport,
2047
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   910
  }
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   911
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   912
  interactive.remote(args, context)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   913
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   914
if __name__ == '__main__':
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   915
  if len(sys.argv) < 2:
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   916
    print "Usage: %s app_id [host]" % (sys.argv[0],)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   917
    sys.exit(1)
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   918
7e9656691c8e Added a stats module to scripts
Sverre Rabbelier <srabbelier@gmail.com>
parents:
diff changeset
   919
  main(sys.argv[1:])