author | Lennard de Rijk <ljvderijk@gmail.com> |
Mon, 13 Jul 2009 00:53:42 +0200 | |
changeset 2616 | 9321e100a7fb |
parent 151 | 6f8eb27752dc |
permissions | -rwxr-xr-x |
140
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1 |
#!/usr/bin/env python |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2 |
"""Universal feed parser |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
3 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
4 |
Handles RSS 0.9x, RSS 1.0, RSS 2.0, CDF, Atom 0.3, and Atom 1.0 feeds |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
5 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
6 |
Visit http://feedparser.org/ for the latest version |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
7 |
Visit http://feedparser.org/docs/ for the latest documentation |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
8 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
9 |
Required: Python 2.1 or later |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
10 |
Recommended: Python 2.3 or later |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
11 |
Recommended: CJKCodecs and iconv_codec <http://cjkpython.i18n.org/> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
12 |
""" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
13 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
14 |
__version__ = "4.1"# + "$Revision: 1.92 $"[11:15] + "-cvs" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
15 |
__license__ = """Copyright (c) 2002-2006, Mark Pilgrim, All rights reserved. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
16 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
17 |
Redistribution and use in source and binary forms, with or without modification, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
18 |
are permitted provided that the following conditions are met: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
19 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
20 |
* Redistributions of source code must retain the above copyright notice, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
21 |
this list of conditions and the following disclaimer. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
22 |
* Redistributions in binary form must reproduce the above copyright notice, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
23 |
this list of conditions and the following disclaimer in the documentation |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
24 |
and/or other materials provided with the distribution. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
25 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
26 |
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 'AS IS' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
27 |
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
28 |
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
29 |
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
30 |
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
31 |
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
32 |
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
33 |
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
34 |
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
35 |
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
36 |
POSSIBILITY OF SUCH DAMAGE.""" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
37 |
__author__ = "Mark Pilgrim <http://diveintomark.org/>" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
38 |
__contributors__ = ["Jason Diamond <http://injektilo.org/>", |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
39 |
"John Beimler <http://john.beimler.org/>", |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
40 |
"Fazal Majid <http://www.majid.info/mylos/weblog/>", |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
41 |
"Aaron Swartz <http://aaronsw.com/>", |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
42 |
"Kevin Marks <http://epeus.blogspot.com/>"] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
43 |
_debug = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
44 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
45 |
# HTTP "User-Agent" header to send to servers when downloading feeds. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
46 |
# If you are embedding feedparser in a larger application, you should |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
47 |
# change this to your application name and URL. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
48 |
USER_AGENT = "UniversalFeedParser/%s +http://feedparser.org/" % __version__ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
49 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
50 |
# HTTP "Accept" header to send to servers when downloading feeds. If you don't |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
51 |
# want to send an Accept header, set this to None. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
52 |
ACCEPT_HEADER = "application/atom+xml,application/rdf+xml,application/rss+xml,application/x-netcdf,application/xml;q=0.9,text/xml;q=0.2,*/*;q=0.1" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
53 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
54 |
# List of preferred XML parsers, by SAX driver name. These will be tried first, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
55 |
# but if they're not installed, Python will keep searching through its own list |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
56 |
# of pre-installed parsers until it finds one that supports everything we need. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
57 |
PREFERRED_XML_PARSERS = ["drv_libxml2"] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
58 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
59 |
# If you want feedparser to automatically run HTML markup through HTML Tidy, set |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
60 |
# this to 1. Requires mxTidy <http://www.egenix.com/files/python/mxTidy.html> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
61 |
# or utidylib <http://utidylib.berlios.de/>. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
62 |
TIDY_MARKUP = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
63 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
64 |
# List of Python interfaces for HTML Tidy, in order of preference. Only useful |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
65 |
# if TIDY_MARKUP = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
66 |
PREFERRED_TIDY_INTERFACES = ["uTidy", "mxTidy"] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
67 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
68 |
# ---------- required modules (should come with any Python distribution) ---------- |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
69 |
import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
70 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
71 |
from cStringIO import StringIO as _StringIO |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
72 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
73 |
from StringIO import StringIO as _StringIO |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
74 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
75 |
# ---------- optional modules (feedparser will work without these, but with reduced functionality) ---------- |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
76 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
77 |
# gzip is included with most Python distributions, but may not be available if you compiled your own |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
78 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
79 |
import gzip |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
80 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
81 |
gzip = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
82 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
83 |
import zlib |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
84 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
85 |
zlib = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
86 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
87 |
# If a real XML parser is available, feedparser will attempt to use it. feedparser has |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
88 |
# been tested with the built-in SAX parser, PyXML, and libxml2. On platforms where the |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
89 |
# Python distribution does not come with an XML parser (such as Mac OS X 10.2 and some |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
90 |
# versions of FreeBSD), feedparser will quietly fall back on regex-based parsing. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
91 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
92 |
import xml.sax |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
93 |
xml.sax.make_parser(PREFERRED_XML_PARSERS) # test for valid parsers |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
94 |
from xml.sax.saxutils import escape as _xmlescape |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
95 |
_XML_AVAILABLE = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
96 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
97 |
_XML_AVAILABLE = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
98 |
def _xmlescape(data): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
99 |
data = data.replace('&', '&') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
100 |
data = data.replace('>', '>') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
101 |
data = data.replace('<', '<') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
102 |
return data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
103 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
104 |
# base64 support for Atom feeds that contain embedded binary data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
105 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
106 |
import base64, binascii |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
107 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
108 |
base64 = binascii = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
109 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
110 |
# cjkcodecs and iconv_codec provide support for more character encodings. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
111 |
# Both are available from http://cjkpython.i18n.org/ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
112 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
113 |
import cjkcodecs.aliases |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
114 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
115 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
116 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
117 |
import iconv_codec |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
118 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
119 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
120 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
121 |
# chardet library auto-detects character encodings |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
122 |
# Download from http://chardet.feedparser.org/ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
123 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
124 |
import chardet |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
125 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
126 |
import chardet.constants |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
127 |
chardet.constants._debug = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
128 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
129 |
chardet = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
130 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
131 |
# ---------- don't touch these ---------- |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
132 |
class ThingsNobodyCaresAboutButMe(Exception): pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
133 |
class CharacterEncodingOverride(ThingsNobodyCaresAboutButMe): pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
134 |
class CharacterEncodingUnknown(ThingsNobodyCaresAboutButMe): pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
135 |
class NonXMLContentType(ThingsNobodyCaresAboutButMe): pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
136 |
class UndeclaredNamespace(Exception): pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
137 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
138 |
sgmllib.tagfind = re.compile('[a-zA-Z][-_.:a-zA-Z0-9]*') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
139 |
sgmllib.special = re.compile('<!') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
140 |
sgmllib.charref = re.compile('&#(x?[0-9A-Fa-f]+)[^0-9A-Fa-f]') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
141 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
142 |
SUPPORTED_VERSIONS = {'': 'unknown', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
143 |
'rss090': 'RSS 0.90', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
144 |
'rss091n': 'RSS 0.91 (Netscape)', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
145 |
'rss091u': 'RSS 0.91 (Userland)', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
146 |
'rss092': 'RSS 0.92', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
147 |
'rss093': 'RSS 0.93', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
148 |
'rss094': 'RSS 0.94', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
149 |
'rss20': 'RSS 2.0', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
150 |
'rss10': 'RSS 1.0', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
151 |
'rss': 'RSS (unknown version)', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
152 |
'atom01': 'Atom 0.1', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
153 |
'atom02': 'Atom 0.2', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
154 |
'atom03': 'Atom 0.3', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
155 |
'atom10': 'Atom 1.0', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
156 |
'atom': 'Atom (unknown version)', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
157 |
'cdf': 'CDF', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
158 |
'hotrss': 'Hot RSS' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
159 |
} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
160 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
161 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
162 |
UserDict = dict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
163 |
except NameError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
164 |
# Python 2.1 does not have dict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
165 |
from UserDict import UserDict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
166 |
def dict(aList): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
167 |
rc = {} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
168 |
for k, v in aList: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
169 |
rc[k] = v |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
170 |
return rc |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
171 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
172 |
class FeedParserDict(UserDict): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
173 |
keymap = {'channel': 'feed', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
174 |
'items': 'entries', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
175 |
'guid': 'id', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
176 |
'date': 'updated', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
177 |
'date_parsed': 'updated_parsed', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
178 |
'description': ['subtitle', 'summary'], |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
179 |
'url': ['href'], |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
180 |
'modified': 'updated', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
181 |
'modified_parsed': 'updated_parsed', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
182 |
'issued': 'published', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
183 |
'issued_parsed': 'published_parsed', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
184 |
'copyright': 'rights', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
185 |
'copyright_detail': 'rights_detail', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
186 |
'tagline': 'subtitle', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
187 |
'tagline_detail': 'subtitle_detail'} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
188 |
def __getitem__(self, key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
189 |
if key == 'category': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
190 |
return UserDict.__getitem__(self, 'tags')[0]['term'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
191 |
if key == 'categories': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
192 |
return [(tag['scheme'], tag['term']) for tag in UserDict.__getitem__(self, 'tags')] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
193 |
realkey = self.keymap.get(key, key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
194 |
if type(realkey) == types.ListType: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
195 |
for k in realkey: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
196 |
if UserDict.has_key(self, k): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
197 |
return UserDict.__getitem__(self, k) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
198 |
if UserDict.has_key(self, key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
199 |
return UserDict.__getitem__(self, key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
200 |
return UserDict.__getitem__(self, realkey) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
201 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
202 |
def __setitem__(self, key, value): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
203 |
for k in self.keymap.keys(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
204 |
if key == k: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
205 |
key = self.keymap[k] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
206 |
if type(key) == types.ListType: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
207 |
key = key[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
208 |
return UserDict.__setitem__(self, key, value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
209 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
210 |
def get(self, key, default=None): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
211 |
if self.has_key(key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
212 |
return self[key] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
213 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
214 |
return default |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
215 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
216 |
def setdefault(self, key, value): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
217 |
if not self.has_key(key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
218 |
self[key] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
219 |
return self[key] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
220 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
221 |
def has_key(self, key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
222 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
223 |
return hasattr(self, key) or UserDict.has_key(self, key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
224 |
except AttributeError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
225 |
return False |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
226 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
227 |
def __getattr__(self, key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
228 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
229 |
return self.__dict__[key] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
230 |
except KeyError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
231 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
232 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
233 |
assert not key.startswith('_') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
234 |
return self.__getitem__(key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
235 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
236 |
raise AttributeError, "object has no attribute '%s'" % key |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
237 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
238 |
def __setattr__(self, key, value): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
239 |
if key.startswith('_') or key == 'data': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
240 |
self.__dict__[key] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
241 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
242 |
return self.__setitem__(key, value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
243 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
244 |
def __contains__(self, key): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
245 |
return self.has_key(key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
246 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
247 |
def zopeCompatibilityHack(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
248 |
global FeedParserDict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
249 |
del FeedParserDict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
250 |
def FeedParserDict(aDict=None): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
251 |
rc = {} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
252 |
if aDict: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
253 |
rc.update(aDict) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
254 |
return rc |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
255 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
256 |
_ebcdic_to_ascii_map = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
257 |
def _ebcdic_to_ascii(s): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
258 |
global _ebcdic_to_ascii_map |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
259 |
if not _ebcdic_to_ascii_map: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
260 |
emap = ( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
261 |
0,1,2,3,156,9,134,127,151,141,142,11,12,13,14,15, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
262 |
16,17,18,19,157,133,8,135,24,25,146,143,28,29,30,31, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
263 |
128,129,130,131,132,10,23,27,136,137,138,139,140,5,6,7, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
264 |
144,145,22,147,148,149,150,4,152,153,154,155,20,21,158,26, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
265 |
32,160,161,162,163,164,165,166,167,168,91,46,60,40,43,33, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
266 |
38,169,170,171,172,173,174,175,176,177,93,36,42,41,59,94, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
267 |
45,47,178,179,180,181,182,183,184,185,124,44,37,95,62,63, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
268 |
186,187,188,189,190,191,192,193,194,96,58,35,64,39,61,34, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
269 |
195,97,98,99,100,101,102,103,104,105,196,197,198,199,200,201, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
270 |
202,106,107,108,109,110,111,112,113,114,203,204,205,206,207,208, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
271 |
209,126,115,116,117,118,119,120,121,122,210,211,212,213,214,215, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
272 |
216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
273 |
123,65,66,67,68,69,70,71,72,73,232,233,234,235,236,237, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
274 |
125,74,75,76,77,78,79,80,81,82,238,239,240,241,242,243, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
275 |
92,159,83,84,85,86,87,88,89,90,244,245,246,247,248,249, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
276 |
48,49,50,51,52,53,54,55,56,57,250,251,252,253,254,255 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
277 |
) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
278 |
import string |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
279 |
_ebcdic_to_ascii_map = string.maketrans( \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
280 |
''.join(map(chr, range(256))), ''.join(map(chr, emap))) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
281 |
return s.translate(_ebcdic_to_ascii_map) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
282 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
283 |
_urifixer = re.compile('^([A-Za-z][A-Za-z0-9+-.]*://)(/*)(.*?)') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
284 |
def _urljoin(base, uri): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
285 |
uri = _urifixer.sub(r'\1\3', uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
286 |
return urlparse.urljoin(base, uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
287 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
288 |
class _FeedParserMixin: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
289 |
namespaces = {'': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
290 |
'http://backend.userland.com/rss': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
291 |
'http://blogs.law.harvard.edu/tech/rss': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
292 |
'http://purl.org/rss/1.0/': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
293 |
'http://my.netscape.com/rdf/simple/0.9/': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
294 |
'http://example.com/newformat#': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
295 |
'http://example.com/necho': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
296 |
'http://purl.org/echo/': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
297 |
'uri/of/echo/namespace#': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
298 |
'http://purl.org/pie/': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
299 |
'http://purl.org/atom/ns#': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
300 |
'http://www.w3.org/2005/Atom': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
301 |
'http://purl.org/rss/1.0/modules/rss091#': '', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
302 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
303 |
'http://webns.net/mvcb/': 'admin', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
304 |
'http://purl.org/rss/1.0/modules/aggregation/': 'ag', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
305 |
'http://purl.org/rss/1.0/modules/annotate/': 'annotate', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
306 |
'http://media.tangent.org/rss/1.0/': 'audio', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
307 |
'http://backend.userland.com/blogChannelModule': 'blogChannel', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
308 |
'http://web.resource.org/cc/': 'cc', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
309 |
'http://backend.userland.com/creativeCommonsRssModule': 'creativeCommons', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
310 |
'http://purl.org/rss/1.0/modules/company': 'co', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
311 |
'http://purl.org/rss/1.0/modules/content/': 'content', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
312 |
'http://my.theinfo.org/changed/1.0/rss/': 'cp', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
313 |
'http://purl.org/dc/elements/1.1/': 'dc', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
314 |
'http://purl.org/dc/terms/': 'dcterms', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
315 |
'http://purl.org/rss/1.0/modules/email/': 'email', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
316 |
'http://purl.org/rss/1.0/modules/event/': 'ev', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
317 |
'http://rssnamespace.org/feedburner/ext/1.0': 'feedburner', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
318 |
'http://freshmeat.net/rss/fm/': 'fm', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
319 |
'http://xmlns.com/foaf/0.1/': 'foaf', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
320 |
'http://www.w3.org/2003/01/geo/wgs84_pos#': 'geo', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
321 |
'http://postneo.com/icbm/': 'icbm', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
322 |
'http://purl.org/rss/1.0/modules/image/': 'image', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
323 |
'http://www.itunes.com/DTDs/PodCast-1.0.dtd': 'itunes', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
324 |
'http://example.com/DTDs/PodCast-1.0.dtd': 'itunes', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
325 |
'http://purl.org/rss/1.0/modules/link/': 'l', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
326 |
'http://search.yahoo.com/mrss': 'media', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
327 |
'http://madskills.com/public/xml/rss/module/pingback/': 'pingback', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
328 |
'http://prismstandard.org/namespaces/1.2/basic/': 'prism', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
329 |
'http://www.w3.org/1999/02/22-rdf-syntax-ns#': 'rdf', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
330 |
'http://www.w3.org/2000/01/rdf-schema#': 'rdfs', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
331 |
'http://purl.org/rss/1.0/modules/reference/': 'ref', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
332 |
'http://purl.org/rss/1.0/modules/richequiv/': 'reqv', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
333 |
'http://purl.org/rss/1.0/modules/search/': 'search', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
334 |
'http://purl.org/rss/1.0/modules/slash/': 'slash', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
335 |
'http://schemas.xmlsoap.org/soap/envelope/': 'soap', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
336 |
'http://purl.org/rss/1.0/modules/servicestatus/': 'ss', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
337 |
'http://hacks.benhammersley.com/rss/streaming/': 'str', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
338 |
'http://purl.org/rss/1.0/modules/subscription/': 'sub', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
339 |
'http://purl.org/rss/1.0/modules/syndication/': 'sy', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
340 |
'http://purl.org/rss/1.0/modules/taxonomy/': 'taxo', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
341 |
'http://purl.org/rss/1.0/modules/threading/': 'thr', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
342 |
'http://purl.org/rss/1.0/modules/textinput/': 'ti', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
343 |
'http://madskills.com/public/xml/rss/module/trackback/':'trackback', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
344 |
'http://wellformedweb.org/commentAPI/': 'wfw', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
345 |
'http://purl.org/rss/1.0/modules/wiki/': 'wiki', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
346 |
'http://www.w3.org/1999/xhtml': 'xhtml', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
347 |
'http://www.w3.org/XML/1998/namespace': 'xml', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
348 |
'http://schemas.pocketsoap.com/rss/myDescModule/': 'szf' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
349 |
} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
350 |
_matchnamespaces = {} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
351 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
352 |
can_be_relative_uri = ['link', 'id', 'wfw_comment', 'wfw_commentrss', 'docs', 'url', 'href', 'comments', 'license', 'icon', 'logo'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
353 |
can_contain_relative_uris = ['content', 'title', 'summary', 'info', 'tagline', 'subtitle', 'copyright', 'rights', 'description'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
354 |
can_contain_dangerous_markup = ['content', 'title', 'summary', 'info', 'tagline', 'subtitle', 'copyright', 'rights', 'description'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
355 |
html_types = ['text/html', 'application/xhtml+xml'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
356 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
357 |
def __init__(self, baseuri=None, baselang=None, encoding='utf-8'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
358 |
if _debug: sys.stderr.write('initializing FeedParser\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
359 |
if not self._matchnamespaces: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
360 |
for k, v in self.namespaces.items(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
361 |
self._matchnamespaces[k.lower()] = v |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
362 |
self.feeddata = FeedParserDict() # feed-level data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
363 |
self.encoding = encoding # character encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
364 |
self.entries = [] # list of entry-level data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
365 |
self.version = '' # feed type/version, see SUPPORTED_VERSIONS |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
366 |
self.namespacesInUse = {} # dictionary of namespaces defined by the feed |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
367 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
368 |
# the following are used internally to track state; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
369 |
# this is really out of control and should be refactored |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
370 |
self.infeed = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
371 |
self.inentry = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
372 |
self.incontent = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
373 |
self.intextinput = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
374 |
self.inimage = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
375 |
self.inauthor = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
376 |
self.incontributor = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
377 |
self.inpublisher = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
378 |
self.insource = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
379 |
self.sourcedata = FeedParserDict() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
380 |
self.contentparams = FeedParserDict() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
381 |
self._summaryKey = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
382 |
self.namespacemap = {} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
383 |
self.elementstack = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
384 |
self.basestack = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
385 |
self.langstack = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
386 |
self.baseuri = baseuri or '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
387 |
self.lang = baselang or None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
388 |
if baselang: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
389 |
self.feeddata['language'] = baselang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
390 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
391 |
def unknown_starttag(self, tag, attrs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
392 |
if _debug: sys.stderr.write('start %s with %s\n' % (tag, attrs)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
393 |
# normalize attrs |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
394 |
attrs = [(k.lower(), v) for k, v in attrs] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
395 |
attrs = [(k, k in ('rel', 'type') and v.lower() or v) for k, v in attrs] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
396 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
397 |
# track xml:base and xml:lang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
398 |
attrsD = dict(attrs) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
399 |
baseuri = attrsD.get('xml:base', attrsD.get('base')) or self.baseuri |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
400 |
self.baseuri = _urljoin(self.baseuri, baseuri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
401 |
lang = attrsD.get('xml:lang', attrsD.get('lang')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
402 |
if lang == '': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
403 |
# xml:lang could be explicitly set to '', we need to capture that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
404 |
lang = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
405 |
elif lang is None: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
406 |
# if no xml:lang is specified, use parent lang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
407 |
lang = self.lang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
408 |
if lang: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
409 |
if tag in ('feed', 'rss', 'rdf:RDF'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
410 |
self.feeddata['language'] = lang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
411 |
self.lang = lang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
412 |
self.basestack.append(self.baseuri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
413 |
self.langstack.append(lang) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
414 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
415 |
# track namespaces |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
416 |
for prefix, uri in attrs: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
417 |
if prefix.startswith('xmlns:'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
418 |
self.trackNamespace(prefix[6:], uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
419 |
elif prefix == 'xmlns': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
420 |
self.trackNamespace(None, uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
421 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
422 |
# track inline content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
423 |
if self.incontent and self.contentparams.has_key('type') and not self.contentparams.get('type', 'xml').endswith('xml'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
424 |
# element declared itself as escaped markup, but it isn't really |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
425 |
self.contentparams['type'] = 'application/xhtml+xml' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
426 |
if self.incontent and self.contentparams.get('type') == 'application/xhtml+xml': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
427 |
# Note: probably shouldn't simply recreate localname here, but |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
428 |
# our namespace handling isn't actually 100% correct in cases where |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
429 |
# the feed redefines the default namespace (which is actually |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
430 |
# the usual case for inline content, thanks Sam), so here we |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
431 |
# cheat and just reconstruct the element based on localname |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
432 |
# because that compensates for the bugs in our namespace handling. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
433 |
# This will horribly munge inline content with non-empty qnames, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
434 |
# but nobody actually does that, so I'm not fixing it. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
435 |
tag = tag.split(':')[-1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
436 |
return self.handle_data('<%s%s>' % (tag, ''.join([' %s="%s"' % t for t in attrs])), escape=0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
437 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
438 |
# match namespaces |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
439 |
if tag.find(':') <> -1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
440 |
prefix, suffix = tag.split(':', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
441 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
442 |
prefix, suffix = '', tag |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
443 |
prefix = self.namespacemap.get(prefix, prefix) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
444 |
if prefix: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
445 |
prefix = prefix + '_' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
446 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
447 |
# special hack for better tracking of empty textinput/image elements in illformed feeds |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
448 |
if (not prefix) and tag not in ('title', 'link', 'description', 'name'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
449 |
self.intextinput = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
450 |
if (not prefix) and tag not in ('title', 'link', 'description', 'url', 'href', 'width', 'height'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
451 |
self.inimage = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
452 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
453 |
# call special handler (if defined) or default handler |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
454 |
methodname = '_start_' + prefix + suffix |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
455 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
456 |
method = getattr(self, methodname) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
457 |
return method(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
458 |
except AttributeError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
459 |
return self.push(prefix + suffix, 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
460 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
461 |
def unknown_endtag(self, tag): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
462 |
if _debug: sys.stderr.write('end %s\n' % tag) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
463 |
# match namespaces |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
464 |
if tag.find(':') <> -1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
465 |
prefix, suffix = tag.split(':', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
466 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
467 |
prefix, suffix = '', tag |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
468 |
prefix = self.namespacemap.get(prefix, prefix) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
469 |
if prefix: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
470 |
prefix = prefix + '_' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
471 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
472 |
# call special handler (if defined) or default handler |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
473 |
methodname = '_end_' + prefix + suffix |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
474 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
475 |
method = getattr(self, methodname) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
476 |
method() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
477 |
except AttributeError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
478 |
self.pop(prefix + suffix) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
479 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
480 |
# track inline content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
481 |
if self.incontent and self.contentparams.has_key('type') and not self.contentparams.get('type', 'xml').endswith('xml'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
482 |
# element declared itself as escaped markup, but it isn't really |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
483 |
self.contentparams['type'] = 'application/xhtml+xml' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
484 |
if self.incontent and self.contentparams.get('type') == 'application/xhtml+xml': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
485 |
tag = tag.split(':')[-1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
486 |
self.handle_data('</%s>' % tag, escape=0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
487 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
488 |
# track xml:base and xml:lang going out of scope |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
489 |
if self.basestack: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
490 |
self.basestack.pop() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
491 |
if self.basestack and self.basestack[-1]: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
492 |
self.baseuri = self.basestack[-1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
493 |
if self.langstack: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
494 |
self.langstack.pop() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
495 |
if self.langstack: # and (self.langstack[-1] is not None): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
496 |
self.lang = self.langstack[-1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
497 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
498 |
def handle_charref(self, ref): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
499 |
# called for each character reference, e.g. for ' ', ref will be '160' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
500 |
if not self.elementstack: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
501 |
ref = ref.lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
502 |
if ref in ('34', '38', '39', '60', '62', 'x22', 'x26', 'x27', 'x3c', 'x3e'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
503 |
text = '&#%s;' % ref |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
504 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
505 |
if ref[0] == 'x': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
506 |
c = int(ref[1:], 16) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
507 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
508 |
c = int(ref) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
509 |
text = unichr(c).encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
510 |
self.elementstack[-1][2].append(text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
511 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
512 |
def handle_entityref(self, ref): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
513 |
# called for each entity reference, e.g. for '©', ref will be 'copy' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
514 |
if not self.elementstack: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
515 |
if _debug: sys.stderr.write('entering handle_entityref with %s\n' % ref) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
516 |
if ref in ('lt', 'gt', 'quot', 'amp', 'apos'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
517 |
text = '&%s;' % ref |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
518 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
519 |
# entity resolution graciously donated by Aaron Swartz |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
520 |
def name2cp(k): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
521 |
import htmlentitydefs |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
522 |
if hasattr(htmlentitydefs, 'name2codepoint'): # requires Python 2.3 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
523 |
return htmlentitydefs.name2codepoint[k] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
524 |
k = htmlentitydefs.entitydefs[k] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
525 |
if k.startswith('&#') and k.endswith(';'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
526 |
return int(k[2:-1]) # not in latin-1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
527 |
return ord(k) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
528 |
try: name2cp(ref) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
529 |
except KeyError: text = '&%s;' % ref |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
530 |
else: text = unichr(name2cp(ref)).encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
531 |
self.elementstack[-1][2].append(text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
532 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
533 |
def handle_data(self, text, escape=1): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
534 |
# called for each block of plain text, i.e. outside of any tag and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
535 |
# not containing any character or entity references |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
536 |
if not self.elementstack: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
537 |
if escape and self.contentparams.get('type') == 'application/xhtml+xml': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
538 |
text = _xmlescape(text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
539 |
self.elementstack[-1][2].append(text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
540 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
541 |
def handle_comment(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
542 |
# called for each comment, e.g. <!-- insert message here --> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
543 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
544 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
545 |
def handle_pi(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
546 |
# called for each processing instruction, e.g. <?instruction> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
547 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
548 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
549 |
def handle_decl(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
550 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
551 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
552 |
def parse_declaration(self, i): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
553 |
# override internal declaration handler to handle CDATA blocks |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
554 |
if _debug: sys.stderr.write('entering parse_declaration\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
555 |
if self.rawdata[i:i+9] == '<![CDATA[': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
556 |
k = self.rawdata.find(']]>', i) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
557 |
if k == -1: k = len(self.rawdata) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
558 |
self.handle_data(_xmlescape(self.rawdata[i+9:k]), 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
559 |
return k+3 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
560 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
561 |
k = self.rawdata.find('>', i) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
562 |
return k+1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
563 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
564 |
def mapContentType(self, contentType): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
565 |
contentType = contentType.lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
566 |
if contentType == 'text': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
567 |
contentType = 'text/plain' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
568 |
elif contentType == 'html': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
569 |
contentType = 'text/html' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
570 |
elif contentType == 'xhtml': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
571 |
contentType = 'application/xhtml+xml' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
572 |
return contentType |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
573 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
574 |
def trackNamespace(self, prefix, uri): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
575 |
loweruri = uri.lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
576 |
if (prefix, loweruri) == (None, 'http://my.netscape.com/rdf/simple/0.9/') and not self.version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
577 |
self.version = 'rss090' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
578 |
if loweruri == 'http://purl.org/rss/1.0/' and not self.version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
579 |
self.version = 'rss10' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
580 |
if loweruri == 'http://www.w3.org/2005/atom' and not self.version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
581 |
self.version = 'atom10' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
582 |
if loweruri.find('backend.userland.com/rss') <> -1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
583 |
# match any backend.userland.com namespace |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
584 |
uri = 'http://backend.userland.com/rss' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
585 |
loweruri = uri |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
586 |
if self._matchnamespaces.has_key(loweruri): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
587 |
self.namespacemap[prefix] = self._matchnamespaces[loweruri] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
588 |
self.namespacesInUse[self._matchnamespaces[loweruri]] = uri |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
589 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
590 |
self.namespacesInUse[prefix or ''] = uri |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
591 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
592 |
def resolveURI(self, uri): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
593 |
return _urljoin(self.baseuri or '', uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
594 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
595 |
def decodeEntities(self, element, data): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
596 |
return data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
597 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
598 |
def push(self, element, expectingText): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
599 |
self.elementstack.append([element, expectingText, []]) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
600 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
601 |
def pop(self, element, stripWhitespace=1): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
602 |
if not self.elementstack: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
603 |
if self.elementstack[-1][0] != element: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
604 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
605 |
element, expectingText, pieces = self.elementstack.pop() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
606 |
output = ''.join(pieces) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
607 |
if stripWhitespace: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
608 |
output = output.strip() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
609 |
if not expectingText: return output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
610 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
611 |
# decode base64 content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
612 |
if base64 and self.contentparams.get('base64', 0): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
613 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
614 |
output = base64.decodestring(output) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
615 |
except binascii.Error: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
616 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
617 |
except binascii.Incomplete: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
618 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
619 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
620 |
# resolve relative URIs |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
621 |
if (element in self.can_be_relative_uri) and output: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
622 |
output = self.resolveURI(output) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
623 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
624 |
# decode entities within embedded markup |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
625 |
if not self.contentparams.get('base64', 0): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
626 |
output = self.decodeEntities(element, output) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
627 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
628 |
# remove temporary cruft from contentparams |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
629 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
630 |
del self.contentparams['mode'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
631 |
except KeyError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
632 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
633 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
634 |
del self.contentparams['base64'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
635 |
except KeyError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
636 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
637 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
638 |
# resolve relative URIs within embedded markup |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
639 |
if self.mapContentType(self.contentparams.get('type', 'text/html')) in self.html_types: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
640 |
if element in self.can_contain_relative_uris: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
641 |
output = _resolveRelativeURIs(output, self.baseuri, self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
642 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
643 |
# sanitize embedded markup |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
644 |
if self.mapContentType(self.contentparams.get('type', 'text/html')) in self.html_types: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
645 |
if element in self.can_contain_dangerous_markup: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
646 |
output = _sanitizeHTML(output, self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
647 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
648 |
if self.encoding and type(output) != type(u''): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
649 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
650 |
output = unicode(output, self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
651 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
652 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
653 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
654 |
# categories/tags/keywords/whatever are handled in _end_category |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
655 |
if element == 'category': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
656 |
return output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
657 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
658 |
# store output in appropriate place(s) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
659 |
if self.inentry and not self.insource: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
660 |
if element == 'content': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
661 |
self.entries[-1].setdefault(element, []) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
662 |
contentparams = copy.deepcopy(self.contentparams) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
663 |
contentparams['value'] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
664 |
self.entries[-1][element].append(contentparams) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
665 |
elif element == 'link': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
666 |
self.entries[-1][element] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
667 |
if output: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
668 |
self.entries[-1]['links'][-1]['href'] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
669 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
670 |
if element == 'description': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
671 |
element = 'summary' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
672 |
self.entries[-1][element] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
673 |
if self.incontent: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
674 |
contentparams = copy.deepcopy(self.contentparams) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
675 |
contentparams['value'] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
676 |
self.entries[-1][element + '_detail'] = contentparams |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
677 |
elif (self.infeed or self.insource) and (not self.intextinput) and (not self.inimage): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
678 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
679 |
if element == 'description': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
680 |
element = 'subtitle' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
681 |
context[element] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
682 |
if element == 'link': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
683 |
context['links'][-1]['href'] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
684 |
elif self.incontent: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
685 |
contentparams = copy.deepcopy(self.contentparams) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
686 |
contentparams['value'] = output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
687 |
context[element + '_detail'] = contentparams |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
688 |
return output |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
689 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
690 |
def pushContent(self, tag, attrsD, defaultContentType, expectingText): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
691 |
self.incontent += 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
692 |
self.contentparams = FeedParserDict({ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
693 |
'type': self.mapContentType(attrsD.get('type', defaultContentType)), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
694 |
'language': self.lang, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
695 |
'base': self.baseuri}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
696 |
self.contentparams['base64'] = self._isBase64(attrsD, self.contentparams) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
697 |
self.push(tag, expectingText) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
698 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
699 |
def popContent(self, tag): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
700 |
value = self.pop(tag) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
701 |
self.incontent -= 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
702 |
self.contentparams.clear() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
703 |
return value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
704 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
705 |
def _mapToStandardPrefix(self, name): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
706 |
colonpos = name.find(':') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
707 |
if colonpos <> -1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
708 |
prefix = name[:colonpos] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
709 |
suffix = name[colonpos+1:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
710 |
prefix = self.namespacemap.get(prefix, prefix) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
711 |
name = prefix + ':' + suffix |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
712 |
return name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
713 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
714 |
def _getAttribute(self, attrsD, name): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
715 |
return attrsD.get(self._mapToStandardPrefix(name)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
716 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
717 |
def _isBase64(self, attrsD, contentparams): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
718 |
if attrsD.get('mode', '') == 'base64': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
719 |
return 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
720 |
if self.contentparams['type'].startswith('text/'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
721 |
return 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
722 |
if self.contentparams['type'].endswith('+xml'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
723 |
return 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
724 |
if self.contentparams['type'].endswith('/xml'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
725 |
return 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
726 |
return 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
727 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
728 |
def _itsAnHrefDamnIt(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
729 |
href = attrsD.get('url', attrsD.get('uri', attrsD.get('href', None))) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
730 |
if href: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
731 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
732 |
del attrsD['url'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
733 |
except KeyError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
734 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
735 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
736 |
del attrsD['uri'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
737 |
except KeyError: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
738 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
739 |
attrsD['href'] = href |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
740 |
return attrsD |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
741 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
742 |
def _save(self, key, value): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
743 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
744 |
context.setdefault(key, value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
745 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
746 |
def _start_rss(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
747 |
versionmap = {'0.91': 'rss091u', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
748 |
'0.92': 'rss092', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
749 |
'0.93': 'rss093', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
750 |
'0.94': 'rss094'} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
751 |
if not self.version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
752 |
attr_version = attrsD.get('version', '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
753 |
version = versionmap.get(attr_version) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
754 |
if version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
755 |
self.version = version |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
756 |
elif attr_version.startswith('2.'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
757 |
self.version = 'rss20' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
758 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
759 |
self.version = 'rss' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
760 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
761 |
def _start_dlhottitles(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
762 |
self.version = 'hotrss' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
763 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
764 |
def _start_channel(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
765 |
self.infeed = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
766 |
self._cdf_common(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
767 |
_start_feedinfo = _start_channel |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
768 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
769 |
def _cdf_common(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
770 |
if attrsD.has_key('lastmod'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
771 |
self._start_modified({}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
772 |
self.elementstack[-1][-1] = attrsD['lastmod'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
773 |
self._end_modified() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
774 |
if attrsD.has_key('href'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
775 |
self._start_link({}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
776 |
self.elementstack[-1][-1] = attrsD['href'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
777 |
self._end_link() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
778 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
779 |
def _start_feed(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
780 |
self.infeed = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
781 |
versionmap = {'0.1': 'atom01', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
782 |
'0.2': 'atom02', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
783 |
'0.3': 'atom03'} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
784 |
if not self.version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
785 |
attr_version = attrsD.get('version') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
786 |
version = versionmap.get(attr_version) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
787 |
if version: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
788 |
self.version = version |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
789 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
790 |
self.version = 'atom' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
791 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
792 |
def _end_channel(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
793 |
self.infeed = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
794 |
_end_feed = _end_channel |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
795 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
796 |
def _start_image(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
797 |
self.inimage = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
798 |
self.push('image', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
799 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
800 |
context.setdefault('image', FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
801 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
802 |
def _end_image(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
803 |
self.pop('image') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
804 |
self.inimage = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
805 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
806 |
def _start_textinput(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
807 |
self.intextinput = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
808 |
self.push('textinput', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
809 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
810 |
context.setdefault('textinput', FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
811 |
_start_textInput = _start_textinput |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
812 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
813 |
def _end_textinput(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
814 |
self.pop('textinput') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
815 |
self.intextinput = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
816 |
_end_textInput = _end_textinput |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
817 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
818 |
def _start_author(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
819 |
self.inauthor = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
820 |
self.push('author', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
821 |
_start_managingeditor = _start_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
822 |
_start_dc_author = _start_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
823 |
_start_dc_creator = _start_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
824 |
_start_itunes_author = _start_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
825 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
826 |
def _end_author(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
827 |
self.pop('author') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
828 |
self.inauthor = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
829 |
self._sync_author_detail() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
830 |
_end_managingeditor = _end_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
831 |
_end_dc_author = _end_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
832 |
_end_dc_creator = _end_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
833 |
_end_itunes_author = _end_author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
834 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
835 |
def _start_itunes_owner(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
836 |
self.inpublisher = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
837 |
self.push('publisher', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
838 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
839 |
def _end_itunes_owner(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
840 |
self.pop('publisher') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
841 |
self.inpublisher = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
842 |
self._sync_author_detail('publisher') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
843 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
844 |
def _start_contributor(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
845 |
self.incontributor = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
846 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
847 |
context.setdefault('contributors', []) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
848 |
context['contributors'].append(FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
849 |
self.push('contributor', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
850 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
851 |
def _end_contributor(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
852 |
self.pop('contributor') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
853 |
self.incontributor = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
854 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
855 |
def _start_dc_contributor(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
856 |
self.incontributor = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
857 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
858 |
context.setdefault('contributors', []) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
859 |
context['contributors'].append(FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
860 |
self.push('name', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
861 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
862 |
def _end_dc_contributor(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
863 |
self._end_name() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
864 |
self.incontributor = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
865 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
866 |
def _start_name(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
867 |
self.push('name', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
868 |
_start_itunes_name = _start_name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
869 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
870 |
def _end_name(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
871 |
value = self.pop('name') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
872 |
if self.inpublisher: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
873 |
self._save_author('name', value, 'publisher') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
874 |
elif self.inauthor: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
875 |
self._save_author('name', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
876 |
elif self.incontributor: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
877 |
self._save_contributor('name', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
878 |
elif self.intextinput: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
879 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
880 |
context['textinput']['name'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
881 |
_end_itunes_name = _end_name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
882 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
883 |
def _start_width(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
884 |
self.push('width', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
885 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
886 |
def _end_width(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
887 |
value = self.pop('width') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
888 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
889 |
value = int(value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
890 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
891 |
value = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
892 |
if self.inimage: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
893 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
894 |
context['image']['width'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
895 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
896 |
def _start_height(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
897 |
self.push('height', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
898 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
899 |
def _end_height(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
900 |
value = self.pop('height') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
901 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
902 |
value = int(value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
903 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
904 |
value = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
905 |
if self.inimage: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
906 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
907 |
context['image']['height'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
908 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
909 |
def _start_url(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
910 |
self.push('href', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
911 |
_start_homepage = _start_url |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
912 |
_start_uri = _start_url |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
913 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
914 |
def _end_url(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
915 |
value = self.pop('href') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
916 |
if self.inauthor: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
917 |
self._save_author('href', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
918 |
elif self.incontributor: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
919 |
self._save_contributor('href', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
920 |
elif self.inimage: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
921 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
922 |
context['image']['href'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
923 |
elif self.intextinput: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
924 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
925 |
context['textinput']['link'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
926 |
_end_homepage = _end_url |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
927 |
_end_uri = _end_url |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
928 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
929 |
def _start_email(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
930 |
self.push('email', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
931 |
_start_itunes_email = _start_email |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
932 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
933 |
def _end_email(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
934 |
value = self.pop('email') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
935 |
if self.inpublisher: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
936 |
self._save_author('email', value, 'publisher') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
937 |
elif self.inauthor: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
938 |
self._save_author('email', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
939 |
elif self.incontributor: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
940 |
self._save_contributor('email', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
941 |
_end_itunes_email = _end_email |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
942 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
943 |
def _getContext(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
944 |
if self.insource: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
945 |
context = self.sourcedata |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
946 |
elif self.inentry: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
947 |
context = self.entries[-1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
948 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
949 |
context = self.feeddata |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
950 |
return context |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
951 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
952 |
def _save_author(self, key, value, prefix='author'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
953 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
954 |
context.setdefault(prefix + '_detail', FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
955 |
context[prefix + '_detail'][key] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
956 |
self._sync_author_detail() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
957 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
958 |
def _save_contributor(self, key, value): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
959 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
960 |
context.setdefault('contributors', [FeedParserDict()]) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
961 |
context['contributors'][-1][key] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
962 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
963 |
def _sync_author_detail(self, key='author'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
964 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
965 |
detail = context.get('%s_detail' % key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
966 |
if detail: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
967 |
name = detail.get('name') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
968 |
email = detail.get('email') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
969 |
if name and email: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
970 |
context[key] = '%s (%s)' % (name, email) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
971 |
elif name: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
972 |
context[key] = name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
973 |
elif email: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
974 |
context[key] = email |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
975 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
976 |
author = context.get(key) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
977 |
if not author: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
978 |
emailmatch = re.search(r'''(([a-zA-Z0-9\_\-\.\+]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?))''', author) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
979 |
if not emailmatch: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
980 |
email = emailmatch.group(0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
981 |
# probably a better way to do the following, but it passes all the tests |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
982 |
author = author.replace(email, '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
983 |
author = author.replace('()', '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
984 |
author = author.strip() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
985 |
if author and (author[0] == '('): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
986 |
author = author[1:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
987 |
if author and (author[-1] == ')'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
988 |
author = author[:-1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
989 |
author = author.strip() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
990 |
context.setdefault('%s_detail' % key, FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
991 |
context['%s_detail' % key]['name'] = author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
992 |
context['%s_detail' % key]['email'] = email |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
993 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
994 |
def _start_subtitle(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
995 |
self.pushContent('subtitle', attrsD, 'text/plain', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
996 |
_start_tagline = _start_subtitle |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
997 |
_start_itunes_subtitle = _start_subtitle |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
998 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
999 |
def _end_subtitle(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1000 |
self.popContent('subtitle') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1001 |
_end_tagline = _end_subtitle |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1002 |
_end_itunes_subtitle = _end_subtitle |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1003 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1004 |
def _start_rights(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1005 |
self.pushContent('rights', attrsD, 'text/plain', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1006 |
_start_dc_rights = _start_rights |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1007 |
_start_copyright = _start_rights |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1008 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1009 |
def _end_rights(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1010 |
self.popContent('rights') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1011 |
_end_dc_rights = _end_rights |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1012 |
_end_copyright = _end_rights |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1013 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1014 |
def _start_item(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1015 |
self.entries.append(FeedParserDict()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1016 |
self.push('item', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1017 |
self.inentry = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1018 |
self.guidislink = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1019 |
id = self._getAttribute(attrsD, 'rdf:about') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1020 |
if id: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1021 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1022 |
context['id'] = id |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1023 |
self._cdf_common(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1024 |
_start_entry = _start_item |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1025 |
_start_product = _start_item |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1026 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1027 |
def _end_item(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1028 |
self.pop('item') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1029 |
self.inentry = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1030 |
_end_entry = _end_item |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1031 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1032 |
def _start_dc_language(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1033 |
self.push('language', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1034 |
_start_language = _start_dc_language |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1035 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1036 |
def _end_dc_language(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1037 |
self.lang = self.pop('language') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1038 |
_end_language = _end_dc_language |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1039 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1040 |
def _start_dc_publisher(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1041 |
self.push('publisher', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1042 |
_start_webmaster = _start_dc_publisher |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1043 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1044 |
def _end_dc_publisher(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1045 |
self.pop('publisher') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1046 |
self._sync_author_detail('publisher') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1047 |
_end_webmaster = _end_dc_publisher |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1048 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1049 |
def _start_published(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1050 |
self.push('published', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1051 |
_start_dcterms_issued = _start_published |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1052 |
_start_issued = _start_published |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1053 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1054 |
def _end_published(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1055 |
value = self.pop('published') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1056 |
self._save('published_parsed', _parse_date(value)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1057 |
_end_dcterms_issued = _end_published |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1058 |
_end_issued = _end_published |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1059 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1060 |
def _start_updated(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1061 |
self.push('updated', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1062 |
_start_modified = _start_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1063 |
_start_dcterms_modified = _start_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1064 |
_start_pubdate = _start_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1065 |
_start_dc_date = _start_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1066 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1067 |
def _end_updated(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1068 |
value = self.pop('updated') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1069 |
parsed_value = _parse_date(value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1070 |
self._save('updated_parsed', parsed_value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1071 |
_end_modified = _end_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1072 |
_end_dcterms_modified = _end_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1073 |
_end_pubdate = _end_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1074 |
_end_dc_date = _end_updated |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1075 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1076 |
def _start_created(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1077 |
self.push('created', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1078 |
_start_dcterms_created = _start_created |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1079 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1080 |
def _end_created(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1081 |
value = self.pop('created') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1082 |
self._save('created_parsed', _parse_date(value)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1083 |
_end_dcterms_created = _end_created |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1084 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1085 |
def _start_expirationdate(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1086 |
self.push('expired', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1087 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1088 |
def _end_expirationdate(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1089 |
self._save('expired_parsed', _parse_date(self.pop('expired'))) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1090 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1091 |
def _start_cc_license(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1092 |
self.push('license', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1093 |
value = self._getAttribute(attrsD, 'rdf:resource') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1094 |
if value: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1095 |
self.elementstack[-1][2].append(value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1096 |
self.pop('license') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1097 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1098 |
def _start_creativecommons_license(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1099 |
self.push('license', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1100 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1101 |
def _end_creativecommons_license(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1102 |
self.pop('license') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1103 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1104 |
def _addTag(self, term, scheme, label): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1105 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1106 |
tags = context.setdefault('tags', []) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1107 |
if (not term) and (not scheme) and (not label): return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1108 |
value = FeedParserDict({'term': term, 'scheme': scheme, 'label': label}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1109 |
if value not in tags: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1110 |
tags.append(FeedParserDict({'term': term, 'scheme': scheme, 'label': label})) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1111 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1112 |
def _start_category(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1113 |
if _debug: sys.stderr.write('entering _start_category with %s\n' % repr(attrsD)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1114 |
term = attrsD.get('term') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1115 |
scheme = attrsD.get('scheme', attrsD.get('domain')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1116 |
label = attrsD.get('label') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1117 |
self._addTag(term, scheme, label) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1118 |
self.push('category', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1119 |
_start_dc_subject = _start_category |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1120 |
_start_keywords = _start_category |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1121 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1122 |
def _end_itunes_keywords(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1123 |
for term in self.pop('itunes_keywords').split(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1124 |
self._addTag(term, 'http://www.itunes.com/', None) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1125 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1126 |
def _start_itunes_category(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1127 |
self._addTag(attrsD.get('text'), 'http://www.itunes.com/', None) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1128 |
self.push('category', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1129 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1130 |
def _end_category(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1131 |
value = self.pop('category') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1132 |
if not value: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1133 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1134 |
tags = context['tags'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1135 |
if value and len(tags) and not tags[-1]['term']: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1136 |
tags[-1]['term'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1137 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1138 |
self._addTag(value, None, None) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1139 |
_end_dc_subject = _end_category |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1140 |
_end_keywords = _end_category |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1141 |
_end_itunes_category = _end_category |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1142 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1143 |
def _start_cloud(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1144 |
self._getContext()['cloud'] = FeedParserDict(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1145 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1146 |
def _start_link(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1147 |
attrsD.setdefault('rel', 'alternate') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1148 |
attrsD.setdefault('type', 'text/html') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1149 |
attrsD = self._itsAnHrefDamnIt(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1150 |
if attrsD.has_key('href'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1151 |
attrsD['href'] = self.resolveURI(attrsD['href']) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1152 |
expectingText = self.infeed or self.inentry or self.insource |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1153 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1154 |
context.setdefault('links', []) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1155 |
context['links'].append(FeedParserDict(attrsD)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1156 |
if attrsD['rel'] == 'enclosure': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1157 |
self._start_enclosure(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1158 |
if attrsD.has_key('href'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1159 |
expectingText = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1160 |
if (attrsD.get('rel') == 'alternate') and (self.mapContentType(attrsD.get('type')) in self.html_types): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1161 |
context['link'] = attrsD['href'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1162 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1163 |
self.push('link', expectingText) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1164 |
_start_producturl = _start_link |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1165 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1166 |
def _end_link(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1167 |
value = self.pop('link') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1168 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1169 |
if self.intextinput: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1170 |
context['textinput']['link'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1171 |
if self.inimage: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1172 |
context['image']['link'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1173 |
_end_producturl = _end_link |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1174 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1175 |
def _start_guid(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1176 |
self.guidislink = (attrsD.get('ispermalink', 'true') == 'true') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1177 |
self.push('id', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1178 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1179 |
def _end_guid(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1180 |
value = self.pop('id') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1181 |
self._save('guidislink', self.guidislink and not self._getContext().has_key('link')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1182 |
if self.guidislink: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1183 |
# guid acts as link, but only if 'ispermalink' is not present or is 'true', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1184 |
# and only if the item doesn't already have a link element |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1185 |
self._save('link', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1186 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1187 |
def _start_title(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1188 |
self.pushContent('title', attrsD, 'text/plain', self.infeed or self.inentry or self.insource) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1189 |
_start_dc_title = _start_title |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1190 |
_start_media_title = _start_title |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1191 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1192 |
def _end_title(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1193 |
value = self.popContent('title') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1194 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1195 |
if self.intextinput: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1196 |
context['textinput']['title'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1197 |
elif self.inimage: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1198 |
context['image']['title'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1199 |
_end_dc_title = _end_title |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1200 |
_end_media_title = _end_title |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1201 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1202 |
def _start_description(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1203 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1204 |
if context.has_key('summary'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1205 |
self._summaryKey = 'content' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1206 |
self._start_content(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1207 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1208 |
self.pushContent('description', attrsD, 'text/html', self.infeed or self.inentry or self.insource) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1209 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1210 |
def _start_abstract(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1211 |
self.pushContent('description', attrsD, 'text/plain', self.infeed or self.inentry or self.insource) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1212 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1213 |
def _end_description(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1214 |
if self._summaryKey == 'content': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1215 |
self._end_content() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1216 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1217 |
value = self.popContent('description') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1218 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1219 |
if self.intextinput: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1220 |
context['textinput']['description'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1221 |
elif self.inimage: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1222 |
context['image']['description'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1223 |
self._summaryKey = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1224 |
_end_abstract = _end_description |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1225 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1226 |
def _start_info(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1227 |
self.pushContent('info', attrsD, 'text/plain', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1228 |
_start_feedburner_browserfriendly = _start_info |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1229 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1230 |
def _end_info(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1231 |
self.popContent('info') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1232 |
_end_feedburner_browserfriendly = _end_info |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1233 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1234 |
def _start_generator(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1235 |
if attrsD: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1236 |
attrsD = self._itsAnHrefDamnIt(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1237 |
if attrsD.has_key('href'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1238 |
attrsD['href'] = self.resolveURI(attrsD['href']) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1239 |
self._getContext()['generator_detail'] = FeedParserDict(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1240 |
self.push('generator', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1241 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1242 |
def _end_generator(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1243 |
value = self.pop('generator') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1244 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1245 |
if context.has_key('generator_detail'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1246 |
context['generator_detail']['name'] = value |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1247 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1248 |
def _start_admin_generatoragent(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1249 |
self.push('generator', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1250 |
value = self._getAttribute(attrsD, 'rdf:resource') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1251 |
if value: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1252 |
self.elementstack[-1][2].append(value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1253 |
self.pop('generator') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1254 |
self._getContext()['generator_detail'] = FeedParserDict({'href': value}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1255 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1256 |
def _start_admin_errorreportsto(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1257 |
self.push('errorreportsto', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1258 |
value = self._getAttribute(attrsD, 'rdf:resource') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1259 |
if value: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1260 |
self.elementstack[-1][2].append(value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1261 |
self.pop('errorreportsto') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1262 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1263 |
def _start_summary(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1264 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1265 |
if context.has_key('summary'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1266 |
self._summaryKey = 'content' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1267 |
self._start_content(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1268 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1269 |
self._summaryKey = 'summary' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1270 |
self.pushContent(self._summaryKey, attrsD, 'text/plain', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1271 |
_start_itunes_summary = _start_summary |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1272 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1273 |
def _end_summary(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1274 |
if self._summaryKey == 'content': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1275 |
self._end_content() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1276 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1277 |
self.popContent(self._summaryKey or 'summary') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1278 |
self._summaryKey = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1279 |
_end_itunes_summary = _end_summary |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1280 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1281 |
def _start_enclosure(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1282 |
attrsD = self._itsAnHrefDamnIt(attrsD) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1283 |
self._getContext().setdefault('enclosures', []).append(FeedParserDict(attrsD)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1284 |
href = attrsD.get('href') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1285 |
if href: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1286 |
context = self._getContext() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1287 |
if not context.get('id'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1288 |
context['id'] = href |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1289 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1290 |
def _start_source(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1291 |
self.insource = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1292 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1293 |
def _end_source(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1294 |
self.insource = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1295 |
self._getContext()['source'] = copy.deepcopy(self.sourcedata) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1296 |
self.sourcedata.clear() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1297 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1298 |
def _start_content(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1299 |
self.pushContent('content', attrsD, 'text/plain', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1300 |
src = attrsD.get('src') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1301 |
if src: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1302 |
self.contentparams['src'] = src |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1303 |
self.push('content', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1304 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1305 |
def _start_prodlink(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1306 |
self.pushContent('content', attrsD, 'text/html', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1307 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1308 |
def _start_body(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1309 |
self.pushContent('content', attrsD, 'application/xhtml+xml', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1310 |
_start_xhtml_body = _start_body |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1311 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1312 |
def _start_content_encoded(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1313 |
self.pushContent('content', attrsD, 'text/html', 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1314 |
_start_fullitem = _start_content_encoded |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1315 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1316 |
def _end_content(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1317 |
copyToDescription = self.mapContentType(self.contentparams.get('type')) in (['text/plain'] + self.html_types) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1318 |
value = self.popContent('content') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1319 |
if copyToDescription: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1320 |
self._save('description', value) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1321 |
_end_body = _end_content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1322 |
_end_xhtml_body = _end_content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1323 |
_end_content_encoded = _end_content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1324 |
_end_fullitem = _end_content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1325 |
_end_prodlink = _end_content |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1326 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1327 |
def _start_itunes_image(self, attrsD): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1328 |
self.push('itunes_image', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1329 |
self._getContext()['image'] = FeedParserDict({'href': attrsD.get('href')}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1330 |
_start_itunes_link = _start_itunes_image |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1331 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1332 |
def _end_itunes_block(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1333 |
value = self.pop('itunes_block', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1334 |
self._getContext()['itunes_block'] = (value == 'yes') and 1 or 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1335 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1336 |
def _end_itunes_explicit(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1337 |
value = self.pop('itunes_explicit', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1338 |
self._getContext()['itunes_explicit'] = (value == 'yes') and 1 or 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1339 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1340 |
if _XML_AVAILABLE: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1341 |
class _StrictFeedParser(_FeedParserMixin, xml.sax.handler.ContentHandler): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1342 |
def __init__(self, baseuri, baselang, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1343 |
if _debug: sys.stderr.write('trying StrictFeedParser\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1344 |
xml.sax.handler.ContentHandler.__init__(self) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1345 |
_FeedParserMixin.__init__(self, baseuri, baselang, encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1346 |
self.bozo = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1347 |
self.exc = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1348 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1349 |
def startPrefixMapping(self, prefix, uri): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1350 |
self.trackNamespace(prefix, uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1351 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1352 |
def startElementNS(self, name, qname, attrs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1353 |
namespace, localname = name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1354 |
lowernamespace = str(namespace or '').lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1355 |
if lowernamespace.find('backend.userland.com/rss') <> -1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1356 |
# match any backend.userland.com namespace |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1357 |
namespace = 'http://backend.userland.com/rss' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1358 |
lowernamespace = namespace |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1359 |
if qname and qname.find(':') > 0: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1360 |
givenprefix = qname.split(':')[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1361 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1362 |
givenprefix = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1363 |
prefix = self._matchnamespaces.get(lowernamespace, givenprefix) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1364 |
if givenprefix and (prefix == None or (prefix == '' and lowernamespace == '')) and not self.namespacesInUse.has_key(givenprefix): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1365 |
raise UndeclaredNamespace, "'%s' is not associated with a namespace" % givenprefix |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1366 |
if prefix: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1367 |
localname = prefix + ':' + localname |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1368 |
localname = str(localname).lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1369 |
if _debug: sys.stderr.write('startElementNS: qname = %s, namespace = %s, givenprefix = %s, prefix = %s, attrs = %s, localname = %s\n' % (qname, namespace, givenprefix, prefix, attrs.items(), localname)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1370 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1371 |
# qname implementation is horribly broken in Python 2.1 (it |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1372 |
# doesn't report any), and slightly broken in Python 2.2 (it |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1373 |
# doesn't report the xml: namespace). So we match up namespaces |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1374 |
# with a known list first, and then possibly override them with |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1375 |
# the qnames the SAX parser gives us (if indeed it gives us any |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1376 |
# at all). Thanks to MatejC for helping me test this and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1377 |
# tirelessly telling me that it didn't work yet. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1378 |
attrsD = {} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1379 |
for (namespace, attrlocalname), attrvalue in attrs._attrs.items(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1380 |
lowernamespace = (namespace or '').lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1381 |
prefix = self._matchnamespaces.get(lowernamespace, '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1382 |
if prefix: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1383 |
attrlocalname = prefix + ':' + attrlocalname |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1384 |
attrsD[str(attrlocalname).lower()] = attrvalue |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1385 |
for qname in attrs.getQNames(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1386 |
attrsD[str(qname).lower()] = attrs.getValueByQName(qname) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1387 |
self.unknown_starttag(localname, attrsD.items()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1388 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1389 |
def characters(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1390 |
self.handle_data(text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1391 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1392 |
def endElementNS(self, name, qname): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1393 |
namespace, localname = name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1394 |
lowernamespace = str(namespace or '').lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1395 |
if qname and qname.find(':') > 0: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1396 |
givenprefix = qname.split(':')[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1397 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1398 |
givenprefix = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1399 |
prefix = self._matchnamespaces.get(lowernamespace, givenprefix) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1400 |
if prefix: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1401 |
localname = prefix + ':' + localname |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1402 |
localname = str(localname).lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1403 |
self.unknown_endtag(localname) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1404 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1405 |
def error(self, exc): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1406 |
self.bozo = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1407 |
self.exc = exc |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1408 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1409 |
def fatalError(self, exc): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1410 |
self.error(exc) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1411 |
raise exc |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1412 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1413 |
class _BaseHTMLProcessor(sgmllib.SGMLParser): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1414 |
elements_no_end_tag = ['area', 'base', 'basefont', 'br', 'col', 'frame', 'hr', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1415 |
'img', 'input', 'isindex', 'link', 'meta', 'param'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1416 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1417 |
def __init__(self, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1418 |
self.encoding = encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1419 |
if _debug: sys.stderr.write('entering BaseHTMLProcessor, encoding=%s\n' % self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1420 |
sgmllib.SGMLParser.__init__(self) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1421 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1422 |
def reset(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1423 |
self.pieces = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1424 |
sgmllib.SGMLParser.reset(self) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1425 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1426 |
def _shorttag_replace(self, match): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1427 |
tag = match.group(1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1428 |
if tag in self.elements_no_end_tag: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1429 |
return '<' + tag + ' />' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1430 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1431 |
return '<' + tag + '></' + tag + '>' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1432 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1433 |
def feed(self, data): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1434 |
data = re.compile(r'<!((?!DOCTYPE|--|\[))', re.IGNORECASE).sub(r'<!\1', data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1435 |
#data = re.sub(r'<(\S+?)\s*?/>', self._shorttag_replace, data) # bug [ 1399464 ] Bad regexp for _shorttag_replace |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1436 |
data = re.sub(r'<([^<\s]+?)\s*/>', self._shorttag_replace, data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1437 |
data = data.replace(''', "'") |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1438 |
data = data.replace('"', '"') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1439 |
if self.encoding and type(data) == type(u''): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1440 |
data = data.encode(self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1441 |
sgmllib.SGMLParser.feed(self, data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1442 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1443 |
def normalize_attrs(self, attrs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1444 |
# utility method to be called by descendants |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1445 |
attrs = [(k.lower(), v) for k, v in attrs] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1446 |
attrs = [(k, k in ('rel', 'type') and v.lower() or v) for k, v in attrs] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1447 |
return attrs |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1448 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1449 |
def unknown_starttag(self, tag, attrs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1450 |
# called for each start tag |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1451 |
# attrs is a list of (attr, value) tuples |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1452 |
# e.g. for <pre class='screen'>, tag='pre', attrs=[('class', 'screen')] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1453 |
if _debug: sys.stderr.write('_BaseHTMLProcessor, unknown_starttag, tag=%s\n' % tag) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1454 |
uattrs = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1455 |
# thanks to Kevin Marks for this breathtaking hack to deal with (valid) high-bit attribute values in UTF-8 feeds |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1456 |
for key, value in attrs: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1457 |
if type(value) != type(u''): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1458 |
value = unicode(value, self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1459 |
uattrs.append((unicode(key, self.encoding), value)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1460 |
strattrs = u''.join([u' %s="%s"' % (key, value) for key, value in uattrs]).encode(self.encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1461 |
if tag in self.elements_no_end_tag: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1462 |
self.pieces.append('<%(tag)s%(strattrs)s />' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1463 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1464 |
self.pieces.append('<%(tag)s%(strattrs)s>' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1465 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1466 |
def unknown_endtag(self, tag): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1467 |
# called for each end tag, e.g. for </pre>, tag will be 'pre' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1468 |
# Reconstruct the original end tag. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1469 |
if tag not in self.elements_no_end_tag: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1470 |
self.pieces.append("</%(tag)s>" % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1471 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1472 |
def handle_charref(self, ref): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1473 |
# called for each character reference, e.g. for ' ', ref will be '160' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1474 |
# Reconstruct the original character reference. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1475 |
self.pieces.append('&#%(ref)s;' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1476 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1477 |
def handle_entityref(self, ref): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1478 |
# called for each entity reference, e.g. for '©', ref will be 'copy' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1479 |
# Reconstruct the original entity reference. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1480 |
self.pieces.append('&%(ref)s;' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1481 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1482 |
def handle_data(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1483 |
# called for each block of plain text, i.e. outside of any tag and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1484 |
# not containing any character or entity references |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1485 |
# Store the original text verbatim. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1486 |
if _debug: sys.stderr.write('_BaseHTMLProcessor, handle_text, text=%s\n' % text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1487 |
self.pieces.append(text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1488 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1489 |
def handle_comment(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1490 |
# called for each HTML comment, e.g. <!-- insert Javascript code here --> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1491 |
# Reconstruct the original comment. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1492 |
self.pieces.append('<!--%(text)s-->' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1493 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1494 |
def handle_pi(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1495 |
# called for each processing instruction, e.g. <?instruction> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1496 |
# Reconstruct original processing instruction. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1497 |
self.pieces.append('<?%(text)s>' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1498 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1499 |
def handle_decl(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1500 |
# called for the DOCTYPE, if present, e.g. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1501 |
# <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1502 |
# "http://www.w3.org/TR/html4/loose.dtd"> |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1503 |
# Reconstruct original DOCTYPE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1504 |
self.pieces.append('<!%(text)s>' % locals()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1505 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1506 |
_new_declname_match = re.compile(r'[a-zA-Z][-_.a-zA-Z0-9:]*\s*').match |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1507 |
def _scan_name(self, i, declstartpos): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1508 |
rawdata = self.rawdata |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1509 |
n = len(rawdata) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1510 |
if i == n: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1511 |
return None, -1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1512 |
m = self._new_declname_match(rawdata, i) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1513 |
if m: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1514 |
s = m.group() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1515 |
name = s.strip() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1516 |
if (i + len(s)) == n: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1517 |
return None, -1 # end of buffer |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1518 |
return name.lower(), m.end() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1519 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1520 |
self.handle_data(rawdata) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1521 |
# self.updatepos(declstartpos, i) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1522 |
return None, -1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1523 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1524 |
def output(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1525 |
'''Return processed HTML as a single string''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1526 |
return ''.join([str(p) for p in self.pieces]) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1527 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1528 |
class _LooseFeedParser(_FeedParserMixin, _BaseHTMLProcessor): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1529 |
def __init__(self, baseuri, baselang, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1530 |
sgmllib.SGMLParser.__init__(self) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1531 |
_FeedParserMixin.__init__(self, baseuri, baselang, encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1532 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1533 |
def decodeEntities(self, element, data): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1534 |
data = data.replace('<', '<') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1535 |
data = data.replace('<', '<') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1536 |
data = data.replace('>', '>') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1537 |
data = data.replace('>', '>') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1538 |
data = data.replace('&', '&') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1539 |
data = data.replace('&', '&') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1540 |
data = data.replace('"', '"') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1541 |
data = data.replace('"', '"') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1542 |
data = data.replace(''', ''') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1543 |
data = data.replace(''', ''') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1544 |
if self.contentparams.has_key('type') and not self.contentparams.get('type', 'xml').endswith('xml'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1545 |
data = data.replace('<', '<') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1546 |
data = data.replace('>', '>') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1547 |
data = data.replace('&', '&') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1548 |
data = data.replace('"', '"') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1549 |
data = data.replace(''', "'") |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1550 |
return data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1551 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1552 |
class _RelativeURIResolver(_BaseHTMLProcessor): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1553 |
relative_uris = [('a', 'href'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1554 |
('applet', 'codebase'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1555 |
('area', 'href'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1556 |
('blockquote', 'cite'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1557 |
('body', 'background'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1558 |
('del', 'cite'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1559 |
('form', 'action'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1560 |
('frame', 'longdesc'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1561 |
('frame', 'src'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1562 |
('iframe', 'longdesc'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1563 |
('iframe', 'src'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1564 |
('head', 'profile'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1565 |
('img', 'longdesc'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1566 |
('img', 'src'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1567 |
('img', 'usemap'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1568 |
('input', 'src'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1569 |
('input', 'usemap'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1570 |
('ins', 'cite'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1571 |
('link', 'href'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1572 |
('object', 'classid'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1573 |
('object', 'codebase'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1574 |
('object', 'data'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1575 |
('object', 'usemap'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1576 |
('q', 'cite'), |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1577 |
('script', 'src')] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1578 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1579 |
def __init__(self, baseuri, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1580 |
_BaseHTMLProcessor.__init__(self, encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1581 |
self.baseuri = baseuri |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1582 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1583 |
def resolveURI(self, uri): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1584 |
return _urljoin(self.baseuri, uri) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1585 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1586 |
def unknown_starttag(self, tag, attrs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1587 |
attrs = self.normalize_attrs(attrs) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1588 |
attrs = [(key, ((tag, key) in self.relative_uris) and self.resolveURI(value) or value) for key, value in attrs] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1589 |
_BaseHTMLProcessor.unknown_starttag(self, tag, attrs) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1590 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1591 |
def _resolveRelativeURIs(htmlSource, baseURI, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1592 |
if _debug: sys.stderr.write('entering _resolveRelativeURIs\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1593 |
p = _RelativeURIResolver(baseURI, encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1594 |
p.feed(htmlSource) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1595 |
return p.output() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1596 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1597 |
class _HTMLSanitizer(_BaseHTMLProcessor): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1598 |
acceptable_elements = ['a', 'abbr', 'acronym', 'address', 'area', 'b', 'big', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1599 |
'blockquote', 'br', 'button', 'caption', 'center', 'cite', 'code', 'col', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1600 |
'colgroup', 'dd', 'del', 'dfn', 'dir', 'div', 'dl', 'dt', 'em', 'fieldset', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1601 |
'font', 'form', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'hr', 'i', 'img', 'input', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1602 |
'ins', 'kbd', 'label', 'legend', 'li', 'map', 'menu', 'ol', 'optgroup', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1603 |
'option', 'p', 'pre', 'q', 's', 'samp', 'select', 'small', 'span', 'strike', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1604 |
'strong', 'sub', 'sup', 'table', 'tbody', 'td', 'textarea', 'tfoot', 'th', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1605 |
'thead', 'tr', 'tt', 'u', 'ul', 'var'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1606 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1607 |
acceptable_attributes = ['abbr', 'accept', 'accept-charset', 'accesskey', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1608 |
'action', 'align', 'alt', 'axis', 'border', 'cellpadding', 'cellspacing', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1609 |
'char', 'charoff', 'charset', 'checked', 'cite', 'class', 'clear', 'cols', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1610 |
'colspan', 'color', 'compact', 'coords', 'datetime', 'dir', 'disabled', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1611 |
'enctype', 'for', 'frame', 'headers', 'height', 'href', 'hreflang', 'hspace', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1612 |
'id', 'ismap', 'label', 'lang', 'longdesc', 'maxlength', 'media', 'method', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1613 |
'multiple', 'name', 'nohref', 'noshade', 'nowrap', 'prompt', 'readonly', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1614 |
'rel', 'rev', 'rows', 'rowspan', 'rules', 'scope', 'selected', 'shape', 'size', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1615 |
'span', 'src', 'start', 'summary', 'tabindex', 'target', 'title', 'type', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1616 |
'usemap', 'valign', 'value', 'vspace', 'width'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1617 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1618 |
unacceptable_elements_with_end_tag = ['script', 'applet'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1619 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1620 |
def reset(self): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1621 |
_BaseHTMLProcessor.reset(self) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1622 |
self.unacceptablestack = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1623 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1624 |
def unknown_starttag(self, tag, attrs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1625 |
if not tag in self.acceptable_elements: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1626 |
if tag in self.unacceptable_elements_with_end_tag: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1627 |
self.unacceptablestack += 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1628 |
return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1629 |
attrs = self.normalize_attrs(attrs) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1630 |
attrs = [(key, value) for key, value in attrs if key in self.acceptable_attributes] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1631 |
_BaseHTMLProcessor.unknown_starttag(self, tag, attrs) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1632 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1633 |
def unknown_endtag(self, tag): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1634 |
if not tag in self.acceptable_elements: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1635 |
if tag in self.unacceptable_elements_with_end_tag: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1636 |
self.unacceptablestack -= 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1637 |
return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1638 |
_BaseHTMLProcessor.unknown_endtag(self, tag) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1639 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1640 |
def handle_pi(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1641 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1642 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1643 |
def handle_decl(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1644 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1645 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1646 |
def handle_data(self, text): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1647 |
if not self.unacceptablestack: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1648 |
_BaseHTMLProcessor.handle_data(self, text) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1649 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1650 |
def _sanitizeHTML(htmlSource, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1651 |
p = _HTMLSanitizer(encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1652 |
p.feed(htmlSource) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1653 |
data = p.output() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1654 |
if TIDY_MARKUP: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1655 |
# loop through list of preferred Tidy interfaces looking for one that's installed, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1656 |
# then set up a common _tidy function to wrap the interface-specific API. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1657 |
_tidy = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1658 |
for tidy_interface in PREFERRED_TIDY_INTERFACES: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1659 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1660 |
if tidy_interface == "uTidy": |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1661 |
from tidy import parseString as _utidy |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1662 |
def _tidy(data, **kwargs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1663 |
return str(_utidy(data, **kwargs)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1664 |
break |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1665 |
elif tidy_interface == "mxTidy": |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1666 |
from mx.Tidy import Tidy as _mxtidy |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1667 |
def _tidy(data, **kwargs): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1668 |
nerrors, nwarnings, data, errordata = _mxtidy.tidy(data, **kwargs) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1669 |
return data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1670 |
break |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1671 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1672 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1673 |
if _tidy: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1674 |
utf8 = type(data) == type(u'') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1675 |
if utf8: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1676 |
data = data.encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1677 |
data = _tidy(data, output_xhtml=1, numeric_entities=1, wrap=0, char_encoding="utf8") |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1678 |
if utf8: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1679 |
data = unicode(data, 'utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1680 |
if data.count('<body'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1681 |
data = data.split('<body', 1)[1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1682 |
if data.count('>'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1683 |
data = data.split('>', 1)[1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1684 |
if data.count('</body'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1685 |
data = data.split('</body', 1)[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1686 |
data = data.strip().replace('\r\n', '\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1687 |
return data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1688 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1689 |
class _FeedURLHandler(urllib2.HTTPDigestAuthHandler, urllib2.HTTPRedirectHandler, urllib2.HTTPDefaultErrorHandler): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1690 |
def http_error_default(self, req, fp, code, msg, headers): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1691 |
if ((code / 100) == 3) and (code != 304): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1692 |
return self.http_error_302(req, fp, code, msg, headers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1693 |
infourl = urllib.addinfourl(fp, headers, req.get_full_url()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1694 |
infourl.status = code |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1695 |
return infourl |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1696 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1697 |
def http_error_302(self, req, fp, code, msg, headers): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1698 |
if headers.dict.has_key('location'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1699 |
infourl = urllib2.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1700 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1701 |
infourl = urllib.addinfourl(fp, headers, req.get_full_url()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1702 |
if not hasattr(infourl, 'status'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1703 |
infourl.status = code |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1704 |
return infourl |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1705 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1706 |
def http_error_301(self, req, fp, code, msg, headers): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1707 |
if headers.dict.has_key('location'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1708 |
infourl = urllib2.HTTPRedirectHandler.http_error_301(self, req, fp, code, msg, headers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1709 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1710 |
infourl = urllib.addinfourl(fp, headers, req.get_full_url()) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1711 |
if not hasattr(infourl, 'status'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1712 |
infourl.status = code |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1713 |
return infourl |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1714 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1715 |
http_error_300 = http_error_302 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1716 |
http_error_303 = http_error_302 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1717 |
http_error_307 = http_error_302 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1718 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1719 |
def http_error_401(self, req, fp, code, msg, headers): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1720 |
# Check if |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1721 |
# - server requires digest auth, AND |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1722 |
# - we tried (unsuccessfully) with basic auth, AND |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1723 |
# - we're using Python 2.3.3 or later (digest auth is irreparably broken in earlier versions) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1724 |
# If all conditions hold, parse authentication information |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1725 |
# out of the Authorization header we sent the first time |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1726 |
# (for the username and password) and the WWW-Authenticate |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1727 |
# header the server sent back (for the realm) and retry |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1728 |
# the request with the appropriate digest auth headers instead. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1729 |
# This evil genius hack has been brought to you by Aaron Swartz. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1730 |
host = urlparse.urlparse(req.get_full_url())[1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1731 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1732 |
assert sys.version.split()[0] >= '2.3.3' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1733 |
assert base64 != None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1734 |
user, passw = base64.decodestring(req.headers['Authorization'].split(' ')[1]).split(':') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1735 |
realm = re.findall('realm="([^"]*)"', headers['WWW-Authenticate'])[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1736 |
self.add_password(realm, host, user, passw) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1737 |
retry = self.http_error_auth_reqed('www-authenticate', host, req, headers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1738 |
self.reset_retry_count() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1739 |
return retry |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1740 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1741 |
return self.http_error_default(req, fp, code, msg, headers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1742 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1743 |
def _open_resource(url_file_stream_or_string, etag, modified, agent, referrer, handlers): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1744 |
"""URL, filename, or string --> stream |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1745 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1746 |
This function lets you define parsers that take any input source |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1747 |
(URL, pathname to local or network file, or actual data as a string) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1748 |
and deal with it in a uniform manner. Returned object is guaranteed |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1749 |
to have all the basic stdio read methods (read, readline, readlines). |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1750 |
Just .close() the object when you're done with it. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1751 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1752 |
If the etag argument is supplied, it will be used as the value of an |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1753 |
If-None-Match request header. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1754 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1755 |
If the modified argument is supplied, it must be a tuple of 9 integers |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1756 |
as returned by gmtime() in the standard Python time module. This MUST |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1757 |
be in GMT (Greenwich Mean Time). The formatted date/time will be used |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1758 |
as the value of an If-Modified-Since request header. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1759 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1760 |
If the agent argument is supplied, it will be used as the value of a |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1761 |
User-Agent request header. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1762 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1763 |
If the referrer argument is supplied, it will be used as the value of a |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1764 |
Referer[sic] request header. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1765 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1766 |
If handlers is supplied, it is a list of handlers used to build a |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1767 |
urllib2 opener. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1768 |
""" |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1769 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1770 |
if hasattr(url_file_stream_or_string, 'read'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1771 |
return url_file_stream_or_string |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1772 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1773 |
if url_file_stream_or_string == '-': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1774 |
return sys.stdin |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1775 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1776 |
if urlparse.urlparse(url_file_stream_or_string)[0] in ('http', 'https', 'ftp'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1777 |
if not agent: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1778 |
agent = USER_AGENT |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1779 |
# test for inline user:password for basic auth |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1780 |
auth = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1781 |
if base64: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1782 |
urltype, rest = urllib.splittype(url_file_stream_or_string) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1783 |
realhost, rest = urllib.splithost(rest) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1784 |
if realhost: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1785 |
user_passwd, realhost = urllib.splituser(realhost) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1786 |
if user_passwd: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1787 |
url_file_stream_or_string = '%s://%s%s' % (urltype, realhost, rest) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1788 |
auth = base64.encodestring(user_passwd).strip() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1789 |
# try to open with urllib2 (to use optional headers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1790 |
request = urllib2.Request(url_file_stream_or_string) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1791 |
request.add_header('User-Agent', agent) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1792 |
if etag: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1793 |
request.add_header('If-None-Match', etag) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1794 |
if modified: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1795 |
# format into an RFC 1123-compliant timestamp. We can't use |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1796 |
# time.strftime() since the %a and %b directives can be affected |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1797 |
# by the current locale, but RFC 2616 states that dates must be |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1798 |
# in English. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1799 |
short_weekdays = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1800 |
months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1801 |
request.add_header('If-Modified-Since', '%s, %02d %s %04d %02d:%02d:%02d GMT' % (short_weekdays[modified[6]], modified[2], months[modified[1] - 1], modified[0], modified[3], modified[4], modified[5])) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1802 |
if referrer: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1803 |
request.add_header('Referer', referrer) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1804 |
if gzip and zlib: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1805 |
request.add_header('Accept-encoding', 'gzip, deflate') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1806 |
elif gzip: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1807 |
request.add_header('Accept-encoding', 'gzip') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1808 |
elif zlib: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1809 |
request.add_header('Accept-encoding', 'deflate') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1810 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1811 |
request.add_header('Accept-encoding', '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1812 |
if auth: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1813 |
request.add_header('Authorization', 'Basic %s' % auth) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1814 |
if ACCEPT_HEADER: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1815 |
request.add_header('Accept', ACCEPT_HEADER) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1816 |
request.add_header('A-IM', 'feed') # RFC 3229 support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1817 |
opener = apply(urllib2.build_opener, tuple([_FeedURLHandler()] + handlers)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1818 |
opener.addheaders = [] # RMK - must clear so we only send our custom User-Agent |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1819 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1820 |
return opener.open(request) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1821 |
finally: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1822 |
opener.close() # JohnD |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1823 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1824 |
# try to open with native open function (if url_file_stream_or_string is a filename) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1825 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1826 |
return open(url_file_stream_or_string) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1827 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1828 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1829 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1830 |
# treat url_file_stream_or_string as string |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1831 |
return _StringIO(str(url_file_stream_or_string)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1832 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1833 |
_date_handlers = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1834 |
def registerDateHandler(func): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1835 |
'''Register a date handler function (takes string, returns 9-tuple date in GMT)''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1836 |
_date_handlers.insert(0, func) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1837 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1838 |
# ISO-8601 date parsing routines written by Fazal Majid. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1839 |
# The ISO 8601 standard is very convoluted and irregular - a full ISO 8601 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1840 |
# parser is beyond the scope of feedparser and would be a worthwhile addition |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1841 |
# to the Python library. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1842 |
# A single regular expression cannot parse ISO 8601 date formats into groups |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1843 |
# as the standard is highly irregular (for instance is 030104 2003-01-04 or |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1844 |
# 0301-04-01), so we use templates instead. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1845 |
# Please note the order in templates is significant because we need a |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1846 |
# greedy match. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1847 |
_iso8601_tmpl = ['YYYY-?MM-?DD', 'YYYY-MM', 'YYYY-?OOO', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1848 |
'YY-?MM-?DD', 'YY-?OOO', 'YYYY', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1849 |
'-YY-?MM', '-OOO', '-YY', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1850 |
'--MM-?DD', '--MM', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1851 |
'---DD', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1852 |
'CC', ''] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1853 |
_iso8601_re = [ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1854 |
tmpl.replace( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1855 |
'YYYY', r'(?P<year>\d{4})').replace( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1856 |
'YY', r'(?P<year>\d\d)').replace( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1857 |
'MM', r'(?P<month>[01]\d)').replace( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1858 |
'DD', r'(?P<day>[0123]\d)').replace( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1859 |
'OOO', r'(?P<ordinal>[0123]\d\d)').replace( |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1860 |
'CC', r'(?P<century>\d\d$)') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1861 |
+ r'(T?(?P<hour>\d{2}):(?P<minute>\d{2})' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1862 |
+ r'(:(?P<second>\d{2}))?' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1863 |
+ r'(?P<tz>[+-](?P<tzhour>\d{2})(:(?P<tzmin>\d{2}))?|Z)?)?' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1864 |
for tmpl in _iso8601_tmpl] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1865 |
del tmpl |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1866 |
_iso8601_matches = [re.compile(regex).match for regex in _iso8601_re] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1867 |
del regex |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1868 |
def _parse_date_iso8601(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1869 |
'''Parse a variety of ISO-8601-compatible formats like 20040105''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1870 |
m = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1871 |
for _iso8601_match in _iso8601_matches: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1872 |
m = _iso8601_match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1873 |
if m: break |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1874 |
if not m: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1875 |
if m.span() == (0, 0): return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1876 |
params = m.groupdict() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1877 |
ordinal = params.get('ordinal', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1878 |
if ordinal: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1879 |
ordinal = int(ordinal) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1880 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1881 |
ordinal = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1882 |
year = params.get('year', '--') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1883 |
if not year or year == '--': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1884 |
year = time.gmtime()[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1885 |
elif len(year) == 2: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1886 |
# ISO 8601 assumes current century, i.e. 93 -> 2093, NOT 1993 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1887 |
year = 100 * int(time.gmtime()[0] / 100) + int(year) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1888 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1889 |
year = int(year) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1890 |
month = params.get('month', '-') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1891 |
if not month or month == '-': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1892 |
# ordinals are NOT normalized by mktime, we simulate them |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1893 |
# by setting month=1, day=ordinal |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1894 |
if ordinal: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1895 |
month = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1896 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1897 |
month = time.gmtime()[1] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1898 |
month = int(month) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1899 |
day = params.get('day', 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1900 |
if not day: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1901 |
# see above |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1902 |
if ordinal: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1903 |
day = ordinal |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1904 |
elif params.get('century', 0) or \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1905 |
params.get('year', 0) or params.get('month', 0): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1906 |
day = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1907 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1908 |
day = time.gmtime()[2] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1909 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1910 |
day = int(day) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1911 |
# special case of the century - is the first year of the 21st century |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1912 |
# 2000 or 2001 ? The debate goes on... |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1913 |
if 'century' in params.keys(): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1914 |
year = (int(params['century']) - 1) * 100 + 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1915 |
# in ISO 8601 most fields are optional |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1916 |
for field in ['hour', 'minute', 'second', 'tzhour', 'tzmin']: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1917 |
if not params.get(field, None): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1918 |
params[field] = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1919 |
hour = int(params.get('hour', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1920 |
minute = int(params.get('minute', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1921 |
second = int(params.get('second', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1922 |
# weekday is normalized by mktime(), we can ignore it |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1923 |
weekday = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1924 |
# daylight savings is complex, but not needed for feedparser's purposes |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1925 |
# as time zones, if specified, include mention of whether it is active |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1926 |
# (e.g. PST vs. PDT, CET). Using -1 is implementation-dependent and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1927 |
# and most implementations have DST bugs |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1928 |
daylight_savings_flag = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1929 |
tm = [year, month, day, hour, minute, second, weekday, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1930 |
ordinal, daylight_savings_flag] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1931 |
# ISO 8601 time zone adjustments |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1932 |
tz = params.get('tz') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1933 |
if tz and tz != 'Z': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1934 |
if tz[0] == '-': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1935 |
tm[3] += int(params.get('tzhour', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1936 |
tm[4] += int(params.get('tzmin', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1937 |
elif tz[0] == '+': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1938 |
tm[3] -= int(params.get('tzhour', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1939 |
tm[4] -= int(params.get('tzmin', 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1940 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1941 |
return None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1942 |
# Python's time.mktime() is a wrapper around the ANSI C mktime(3c) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1943 |
# which is guaranteed to normalize d/m/y/h/m/s. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1944 |
# Many implementations have bugs, but we'll pretend they don't. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1945 |
return time.localtime(time.mktime(tm)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1946 |
registerDateHandler(_parse_date_iso8601) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1947 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1948 |
# 8-bit date handling routines written by ytrewq1. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1949 |
_korean_year = u'\ub144' # b3e2 in euc-kr |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1950 |
_korean_month = u'\uc6d4' # bff9 in euc-kr |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1951 |
_korean_day = u'\uc77c' # c0cf in euc-kr |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1952 |
_korean_am = u'\uc624\uc804' # bfc0 c0fc in euc-kr |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1953 |
_korean_pm = u'\uc624\ud6c4' # bfc0 c8c4 in euc-kr |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1954 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1955 |
_korean_onblog_date_re = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1956 |
re.compile('(\d{4})%s\s+(\d{2})%s\s+(\d{2})%s\s+(\d{2}):(\d{2}):(\d{2})' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1957 |
(_korean_year, _korean_month, _korean_day)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1958 |
_korean_nate_date_re = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1959 |
re.compile(u'(\d{4})-(\d{2})-(\d{2})\s+(%s|%s)\s+(\d{,2}):(\d{,2}):(\d{,2})' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1960 |
(_korean_am, _korean_pm)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1961 |
def _parse_date_onblog(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1962 |
'''Parse a string according to the OnBlog 8-bit date format''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1963 |
m = _korean_onblog_date_re.match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1964 |
if not m: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1965 |
w3dtfdate = '%(year)s-%(month)s-%(day)sT%(hour)s:%(minute)s:%(second)s%(zonediff)s' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1966 |
{'year': m.group(1), 'month': m.group(2), 'day': m.group(3),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1967 |
'hour': m.group(4), 'minute': m.group(5), 'second': m.group(6),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1968 |
'zonediff': '+09:00'} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1969 |
if _debug: sys.stderr.write('OnBlog date parsed as: %s\n' % w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1970 |
return _parse_date_w3dtf(w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1971 |
registerDateHandler(_parse_date_onblog) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1972 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1973 |
def _parse_date_nate(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1974 |
'''Parse a string according to the Nate 8-bit date format''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1975 |
m = _korean_nate_date_re.match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1976 |
if not m: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1977 |
hour = int(m.group(5)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1978 |
ampm = m.group(4) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1979 |
if (ampm == _korean_pm): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1980 |
hour += 12 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1981 |
hour = str(hour) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1982 |
if len(hour) == 1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1983 |
hour = '0' + hour |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1984 |
w3dtfdate = '%(year)s-%(month)s-%(day)sT%(hour)s:%(minute)s:%(second)s%(zonediff)s' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1985 |
{'year': m.group(1), 'month': m.group(2), 'day': m.group(3),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1986 |
'hour': hour, 'minute': m.group(6), 'second': m.group(7),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1987 |
'zonediff': '+09:00'} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1988 |
if _debug: sys.stderr.write('Nate date parsed as: %s\n' % w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1989 |
return _parse_date_w3dtf(w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1990 |
registerDateHandler(_parse_date_nate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1991 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1992 |
_mssql_date_re = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1993 |
re.compile('(\d{4})-(\d{2})-(\d{2})\s+(\d{2}):(\d{2}):(\d{2})(\.\d+)?') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1994 |
def _parse_date_mssql(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1995 |
'''Parse a string according to the MS SQL date format''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1996 |
m = _mssql_date_re.match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1997 |
if not m: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1998 |
w3dtfdate = '%(year)s-%(month)s-%(day)sT%(hour)s:%(minute)s:%(second)s%(zonediff)s' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
1999 |
{'year': m.group(1), 'month': m.group(2), 'day': m.group(3),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2000 |
'hour': m.group(4), 'minute': m.group(5), 'second': m.group(6),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2001 |
'zonediff': '+09:00'} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2002 |
if _debug: sys.stderr.write('MS SQL date parsed as: %s\n' % w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2003 |
return _parse_date_w3dtf(w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2004 |
registerDateHandler(_parse_date_mssql) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2005 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2006 |
# Unicode strings for Greek date strings |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2007 |
_greek_months = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2008 |
{ \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2009 |
u'\u0399\u03b1\u03bd': u'Jan', # c9e1ed in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2010 |
u'\u03a6\u03b5\u03b2': u'Feb', # d6e5e2 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2011 |
u'\u039c\u03ac\u03ce': u'Mar', # ccdcfe in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2012 |
u'\u039c\u03b1\u03ce': u'Mar', # cce1fe in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2013 |
u'\u0391\u03c0\u03c1': u'Apr', # c1f0f1 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2014 |
u'\u039c\u03ac\u03b9': u'May', # ccdce9 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2015 |
u'\u039c\u03b1\u03ca': u'May', # cce1fa in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2016 |
u'\u039c\u03b1\u03b9': u'May', # cce1e9 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2017 |
u'\u0399\u03bf\u03cd\u03bd': u'Jun', # c9effded in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2018 |
u'\u0399\u03bf\u03bd': u'Jun', # c9efed in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2019 |
u'\u0399\u03bf\u03cd\u03bb': u'Jul', # c9effdeb in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2020 |
u'\u0399\u03bf\u03bb': u'Jul', # c9f9eb in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2021 |
u'\u0391\u03cd\u03b3': u'Aug', # c1fde3 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2022 |
u'\u0391\u03c5\u03b3': u'Aug', # c1f5e3 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2023 |
u'\u03a3\u03b5\u03c0': u'Sep', # d3e5f0 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2024 |
u'\u039f\u03ba\u03c4': u'Oct', # cfeaf4 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2025 |
u'\u039d\u03bf\u03ad': u'Nov', # cdefdd in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2026 |
u'\u039d\u03bf\u03b5': u'Nov', # cdefe5 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2027 |
u'\u0394\u03b5\u03ba': u'Dec', # c4e5ea in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2028 |
} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2029 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2030 |
_greek_wdays = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2031 |
{ \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2032 |
u'\u039a\u03c5\u03c1': u'Sun', # caf5f1 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2033 |
u'\u0394\u03b5\u03c5': u'Mon', # c4e5f5 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2034 |
u'\u03a4\u03c1\u03b9': u'Tue', # d4f1e9 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2035 |
u'\u03a4\u03b5\u03c4': u'Wed', # d4e5f4 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2036 |
u'\u03a0\u03b5\u03bc': u'Thu', # d0e5ec in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2037 |
u'\u03a0\u03b1\u03c1': u'Fri', # d0e1f1 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2038 |
u'\u03a3\u03b1\u03b2': u'Sat', # d3e1e2 in iso-8859-7 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2039 |
} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2040 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2041 |
_greek_date_format_re = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2042 |
re.compile(u'([^,]+),\s+(\d{2})\s+([^\s]+)\s+(\d{4})\s+(\d{2}):(\d{2}):(\d{2})\s+([^\s]+)') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2043 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2044 |
def _parse_date_greek(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2045 |
'''Parse a string according to a Greek 8-bit date format.''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2046 |
m = _greek_date_format_re.match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2047 |
if not m: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2048 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2049 |
wday = _greek_wdays[m.group(1)] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2050 |
month = _greek_months[m.group(3)] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2051 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2052 |
return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2053 |
rfc822date = '%(wday)s, %(day)s %(month)s %(year)s %(hour)s:%(minute)s:%(second)s %(zonediff)s' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2054 |
{'wday': wday, 'day': m.group(2), 'month': month, 'year': m.group(4),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2055 |
'hour': m.group(5), 'minute': m.group(6), 'second': m.group(7),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2056 |
'zonediff': m.group(8)} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2057 |
if _debug: sys.stderr.write('Greek date parsed as: %s\n' % rfc822date) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2058 |
return _parse_date_rfc822(rfc822date) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2059 |
registerDateHandler(_parse_date_greek) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2060 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2061 |
# Unicode strings for Hungarian date strings |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2062 |
_hungarian_months = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2063 |
{ \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2064 |
u'janu\u00e1r': u'01', # e1 in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2065 |
u'febru\u00e1ri': u'02', # e1 in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2066 |
u'm\u00e1rcius': u'03', # e1 in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2067 |
u'\u00e1prilis': u'04', # e1 in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2068 |
u'm\u00e1ujus': u'05', # e1 in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2069 |
u'j\u00fanius': u'06', # fa in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2070 |
u'j\u00falius': u'07', # fa in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2071 |
u'augusztus': u'08', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2072 |
u'szeptember': u'09', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2073 |
u'okt\u00f3ber': u'10', # f3 in iso-8859-2 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2074 |
u'november': u'11', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2075 |
u'december': u'12', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2076 |
} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2077 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2078 |
_hungarian_date_format_re = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2079 |
re.compile(u'(\d{4})-([^-]+)-(\d{,2})T(\d{,2}):(\d{2})((\+|-)(\d{,2}:\d{2}))') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2080 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2081 |
def _parse_date_hungarian(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2082 |
'''Parse a string according to a Hungarian 8-bit date format.''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2083 |
m = _hungarian_date_format_re.match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2084 |
if not m: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2085 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2086 |
month = _hungarian_months[m.group(2)] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2087 |
day = m.group(3) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2088 |
if len(day) == 1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2089 |
day = '0' + day |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2090 |
hour = m.group(4) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2091 |
if len(hour) == 1: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2092 |
hour = '0' + hour |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2093 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2094 |
return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2095 |
w3dtfdate = '%(year)s-%(month)s-%(day)sT%(hour)s:%(minute)s%(zonediff)s' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2096 |
{'year': m.group(1), 'month': month, 'day': day,\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2097 |
'hour': hour, 'minute': m.group(5),\ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2098 |
'zonediff': m.group(6)} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2099 |
if _debug: sys.stderr.write('Hungarian date parsed as: %s\n' % w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2100 |
return _parse_date_w3dtf(w3dtfdate) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2101 |
registerDateHandler(_parse_date_hungarian) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2102 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2103 |
# W3DTF-style date parsing adapted from PyXML xml.utils.iso8601, written by |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2104 |
# Drake and licensed under the Python license. Removed all range checking |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2105 |
# for month, day, hour, minute, and second, since mktime will normalize |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2106 |
# these later |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2107 |
def _parse_date_w3dtf(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2108 |
def __extract_date(m): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2109 |
year = int(m.group('year')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2110 |
if year < 100: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2111 |
year = 100 * int(time.gmtime()[0] / 100) + int(year) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2112 |
if year < 1000: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2113 |
return 0, 0, 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2114 |
julian = m.group('julian') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2115 |
if julian: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2116 |
julian = int(julian) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2117 |
month = julian / 30 + 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2118 |
day = julian % 30 + 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2119 |
jday = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2120 |
while jday != julian: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2121 |
t = time.mktime((year, month, day, 0, 0, 0, 0, 0, 0)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2122 |
jday = time.gmtime(t)[-2] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2123 |
diff = abs(jday - julian) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2124 |
if jday > julian: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2125 |
if diff < day: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2126 |
day = day - diff |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2127 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2128 |
month = month - 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2129 |
day = 31 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2130 |
elif jday < julian: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2131 |
if day + diff < 28: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2132 |
day = day + diff |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2133 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2134 |
month = month + 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2135 |
return year, month, day |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2136 |
month = m.group('month') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2137 |
day = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2138 |
if month is None: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2139 |
month = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2140 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2141 |
month = int(month) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2142 |
day = m.group('day') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2143 |
if day: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2144 |
day = int(day) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2145 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2146 |
day = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2147 |
return year, month, day |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2148 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2149 |
def __extract_time(m): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2150 |
if not m: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2151 |
return 0, 0, 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2152 |
hours = m.group('hours') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2153 |
if not hours: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2154 |
return 0, 0, 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2155 |
hours = int(hours) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2156 |
minutes = int(m.group('minutes')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2157 |
seconds = m.group('seconds') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2158 |
if seconds: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2159 |
seconds = int(seconds) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2160 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2161 |
seconds = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2162 |
return hours, minutes, seconds |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2163 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2164 |
def __extract_tzd(m): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2165 |
'''Return the Time Zone Designator as an offset in seconds from UTC.''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2166 |
if not m: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2167 |
return 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2168 |
tzd = m.group('tzd') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2169 |
if not tzd: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2170 |
return 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2171 |
if tzd == 'Z': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2172 |
return 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2173 |
hours = int(m.group('tzdhours')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2174 |
minutes = m.group('tzdminutes') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2175 |
if minutes: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2176 |
minutes = int(minutes) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2177 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2178 |
minutes = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2179 |
offset = (hours*60 + minutes) * 60 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2180 |
if tzd[0] == '+': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2181 |
return -offset |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2182 |
return offset |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2183 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2184 |
__date_re = ('(?P<year>\d\d\d\d)' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2185 |
'(?:(?P<dsep>-|)' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2186 |
'(?:(?P<julian>\d\d\d)' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2187 |
'|(?P<month>\d\d)(?:(?P=dsep)(?P<day>\d\d))?))?') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2188 |
__tzd_re = '(?P<tzd>[-+](?P<tzdhours>\d\d)(?::?(?P<tzdminutes>\d\d))|Z)' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2189 |
__tzd_rx = re.compile(__tzd_re) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2190 |
__time_re = ('(?P<hours>\d\d)(?P<tsep>:|)(?P<minutes>\d\d)' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2191 |
'(?:(?P=tsep)(?P<seconds>\d\d(?:[.,]\d+)?))?' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2192 |
+ __tzd_re) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2193 |
__datetime_re = '%s(?:T%s)?' % (__date_re, __time_re) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2194 |
__datetime_rx = re.compile(__datetime_re) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2195 |
m = __datetime_rx.match(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2196 |
if (m is None) or (m.group() != dateString): return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2197 |
gmt = __extract_date(m) + __extract_time(m) + (0, 0, 0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2198 |
if gmt[0] == 0: return |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2199 |
return time.gmtime(time.mktime(gmt) + __extract_tzd(m) - time.timezone) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2200 |
registerDateHandler(_parse_date_w3dtf) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2201 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2202 |
def _parse_date_rfc822(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2203 |
'''Parse an RFC822, RFC1123, RFC2822, or asctime-style date''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2204 |
data = dateString.split() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2205 |
if data[0][-1] in (',', '.') or data[0].lower() in rfc822._daynames: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2206 |
del data[0] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2207 |
if len(data) == 4: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2208 |
s = data[3] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2209 |
i = s.find('+') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2210 |
if i > 0: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2211 |
data[3:] = [s[:i], s[i+1:]] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2212 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2213 |
data.append('') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2214 |
dateString = " ".join(data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2215 |
if len(data) < 5: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2216 |
dateString += ' 00:00:00 GMT' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2217 |
tm = rfc822.parsedate_tz(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2218 |
if tm: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2219 |
return time.gmtime(rfc822.mktime_tz(tm)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2220 |
# rfc822.py defines several time zones, but we define some extra ones. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2221 |
# 'ET' is equivalent to 'EST', etc. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2222 |
_additional_timezones = {'AT': -400, 'ET': -500, 'CT': -600, 'MT': -700, 'PT': -800} |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2223 |
rfc822._timezones.update(_additional_timezones) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2224 |
registerDateHandler(_parse_date_rfc822) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2225 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2226 |
def _parse_date(dateString): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2227 |
'''Parses a variety of date formats into a 9-tuple in GMT''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2228 |
for handler in _date_handlers: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2229 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2230 |
date9tuple = handler(dateString) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2231 |
if not date9tuple: continue |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2232 |
if len(date9tuple) != 9: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2233 |
if _debug: sys.stderr.write('date handler function must return 9-tuple\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2234 |
raise ValueError |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2235 |
map(int, date9tuple) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2236 |
return date9tuple |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2237 |
except Exception, e: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2238 |
if _debug: sys.stderr.write('%s raised %s\n' % (handler.__name__, repr(e))) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2239 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2240 |
return None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2241 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2242 |
def _getCharacterEncoding(http_headers, xml_data): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2243 |
'''Get the character encoding of the XML document |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2244 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2245 |
http_headers is a dictionary |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2246 |
xml_data is a raw string (not Unicode) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2247 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2248 |
This is so much trickier than it sounds, it's not even funny. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2249 |
According to RFC 3023 ('XML Media Types'), if the HTTP Content-Type |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2250 |
is application/xml, application/*+xml, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2251 |
application/xml-external-parsed-entity, or application/xml-dtd, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2252 |
the encoding given in the charset parameter of the HTTP Content-Type |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2253 |
takes precedence over the encoding given in the XML prefix within the |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2254 |
document, and defaults to 'utf-8' if neither are specified. But, if |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2255 |
the HTTP Content-Type is text/xml, text/*+xml, or |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2256 |
text/xml-external-parsed-entity, the encoding given in the XML prefix |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2257 |
within the document is ALWAYS IGNORED and only the encoding given in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2258 |
the charset parameter of the HTTP Content-Type header should be |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2259 |
respected, and it defaults to 'us-ascii' if not specified. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2260 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2261 |
Furthermore, discussion on the atom-syntax mailing list with the |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2262 |
author of RFC 3023 leads me to the conclusion that any document |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2263 |
served with a Content-Type of text/* and no charset parameter |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2264 |
must be treated as us-ascii. (We now do this.) And also that it |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2265 |
must always be flagged as non-well-formed. (We now do this too.) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2266 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2267 |
If Content-Type is unspecified (input was local file or non-HTTP source) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2268 |
or unrecognized (server just got it totally wrong), then go by the |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2269 |
encoding given in the XML prefix of the document and default to |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2270 |
'iso-8859-1' as per the HTTP specification (RFC 2616). |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2271 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2272 |
Then, assuming we didn't find a character encoding in the HTTP headers |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2273 |
(and the HTTP Content-type allowed us to look in the body), we need |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2274 |
to sniff the first few bytes of the XML data and try to determine |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2275 |
whether the encoding is ASCII-compatible. Section F of the XML |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2276 |
specification shows the way here: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2277 |
http://www.w3.org/TR/REC-xml/#sec-guessing-no-ext-info |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2278 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2279 |
If the sniffed encoding is not ASCII-compatible, we need to make it |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2280 |
ASCII compatible so that we can sniff further into the XML declaration |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2281 |
to find the encoding attribute, which will tell us the true encoding. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2282 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2283 |
Of course, none of this guarantees that we will be able to parse the |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2284 |
feed in the declared character encoding (assuming it was declared |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2285 |
correctly, which many are not). CJKCodecs and iconv_codec help a lot; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2286 |
you should definitely install them if you can. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2287 |
http://cjkpython.i18n.org/ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2288 |
''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2289 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2290 |
def _parseHTTPContentType(content_type): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2291 |
'''takes HTTP Content-Type header and returns (content type, charset) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2292 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2293 |
If no charset is specified, returns (content type, '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2294 |
If no content type is specified, returns ('', '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2295 |
Both return parameters are guaranteed to be lowercase strings |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2296 |
''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2297 |
content_type = content_type or '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2298 |
content_type, params = cgi.parse_header(content_type) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2299 |
return content_type, params.get('charset', '').replace("'", '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2300 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2301 |
sniffed_xml_encoding = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2302 |
xml_encoding = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2303 |
true_encoding = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2304 |
http_content_type, http_encoding = _parseHTTPContentType(http_headers.get('content-type')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2305 |
# Must sniff for non-ASCII-compatible character encodings before |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2306 |
# searching for XML declaration. This heuristic is defined in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2307 |
# section F of the XML specification: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2308 |
# http://www.w3.org/TR/REC-xml/#sec-guessing-no-ext-info |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2309 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2310 |
if xml_data[:4] == '\x4c\x6f\xa7\x94': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2311 |
# EBCDIC |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2312 |
xml_data = _ebcdic_to_ascii(xml_data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2313 |
elif xml_data[:4] == '\x00\x3c\x00\x3f': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2314 |
# UTF-16BE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2315 |
sniffed_xml_encoding = 'utf-16be' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2316 |
xml_data = unicode(xml_data, 'utf-16be').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2317 |
elif (len(xml_data) >= 4) and (xml_data[:2] == '\xfe\xff') and (xml_data[2:4] != '\x00\x00'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2318 |
# UTF-16BE with BOM |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2319 |
sniffed_xml_encoding = 'utf-16be' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2320 |
xml_data = unicode(xml_data[2:], 'utf-16be').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2321 |
elif xml_data[:4] == '\x3c\x00\x3f\x00': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2322 |
# UTF-16LE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2323 |
sniffed_xml_encoding = 'utf-16le' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2324 |
xml_data = unicode(xml_data, 'utf-16le').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2325 |
elif (len(xml_data) >= 4) and (xml_data[:2] == '\xff\xfe') and (xml_data[2:4] != '\x00\x00'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2326 |
# UTF-16LE with BOM |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2327 |
sniffed_xml_encoding = 'utf-16le' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2328 |
xml_data = unicode(xml_data[2:], 'utf-16le').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2329 |
elif xml_data[:4] == '\x00\x00\x00\x3c': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2330 |
# UTF-32BE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2331 |
sniffed_xml_encoding = 'utf-32be' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2332 |
xml_data = unicode(xml_data, 'utf-32be').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2333 |
elif xml_data[:4] == '\x3c\x00\x00\x00': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2334 |
# UTF-32LE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2335 |
sniffed_xml_encoding = 'utf-32le' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2336 |
xml_data = unicode(xml_data, 'utf-32le').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2337 |
elif xml_data[:4] == '\x00\x00\xfe\xff': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2338 |
# UTF-32BE with BOM |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2339 |
sniffed_xml_encoding = 'utf-32be' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2340 |
xml_data = unicode(xml_data[4:], 'utf-32be').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2341 |
elif xml_data[:4] == '\xff\xfe\x00\x00': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2342 |
# UTF-32LE with BOM |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2343 |
sniffed_xml_encoding = 'utf-32le' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2344 |
xml_data = unicode(xml_data[4:], 'utf-32le').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2345 |
elif xml_data[:3] == '\xef\xbb\xbf': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2346 |
# UTF-8 with BOM |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2347 |
sniffed_xml_encoding = 'utf-8' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2348 |
xml_data = unicode(xml_data[3:], 'utf-8').encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2349 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2350 |
# ASCII-compatible |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2351 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2352 |
xml_encoding_match = re.compile('^<\?.*encoding=[\'"](.*?)[\'"].*\?>').match(xml_data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2353 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2354 |
xml_encoding_match = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2355 |
if xml_encoding_match: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2356 |
xml_encoding = xml_encoding_match.groups()[0].lower() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2357 |
if sniffed_xml_encoding and (xml_encoding in ('iso-10646-ucs-2', 'ucs-2', 'csunicode', 'iso-10646-ucs-4', 'ucs-4', 'csucs4', 'utf-16', 'utf-32', 'utf_16', 'utf_32', 'utf16', 'u16')): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2358 |
xml_encoding = sniffed_xml_encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2359 |
acceptable_content_type = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2360 |
application_content_types = ('application/xml', 'application/xml-dtd', 'application/xml-external-parsed-entity') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2361 |
text_content_types = ('text/xml', 'text/xml-external-parsed-entity') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2362 |
if (http_content_type in application_content_types) or \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2363 |
(http_content_type.startswith('application/') and http_content_type.endswith('+xml')): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2364 |
acceptable_content_type = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2365 |
true_encoding = http_encoding or xml_encoding or 'utf-8' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2366 |
elif (http_content_type in text_content_types) or \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2367 |
(http_content_type.startswith('text/')) and http_content_type.endswith('+xml'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2368 |
acceptable_content_type = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2369 |
true_encoding = http_encoding or 'us-ascii' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2370 |
elif http_content_type.startswith('text/'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2371 |
true_encoding = http_encoding or 'us-ascii' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2372 |
elif http_headers and (not http_headers.has_key('content-type')): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2373 |
true_encoding = xml_encoding or 'iso-8859-1' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2374 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2375 |
true_encoding = xml_encoding or 'utf-8' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2376 |
return true_encoding, http_encoding, xml_encoding, sniffed_xml_encoding, acceptable_content_type |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2377 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2378 |
def _toUTF8(data, encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2379 |
'''Changes an XML data stream on the fly to specify a new encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2380 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2381 |
data is a raw sequence of bytes (not Unicode) that is presumed to be in %encoding already |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2382 |
encoding is a string recognized by encodings.aliases |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2383 |
''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2384 |
if _debug: sys.stderr.write('entering _toUTF8, trying encoding %s\n' % encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2385 |
# strip Byte Order Mark (if present) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2386 |
if (len(data) >= 4) and (data[:2] == '\xfe\xff') and (data[2:4] != '\x00\x00'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2387 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2388 |
sys.stderr.write('stripping BOM\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2389 |
if encoding != 'utf-16be': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2390 |
sys.stderr.write('trying utf-16be instead\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2391 |
encoding = 'utf-16be' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2392 |
data = data[2:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2393 |
elif (len(data) >= 4) and (data[:2] == '\xff\xfe') and (data[2:4] != '\x00\x00'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2394 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2395 |
sys.stderr.write('stripping BOM\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2396 |
if encoding != 'utf-16le': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2397 |
sys.stderr.write('trying utf-16le instead\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2398 |
encoding = 'utf-16le' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2399 |
data = data[2:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2400 |
elif data[:3] == '\xef\xbb\xbf': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2401 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2402 |
sys.stderr.write('stripping BOM\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2403 |
if encoding != 'utf-8': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2404 |
sys.stderr.write('trying utf-8 instead\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2405 |
encoding = 'utf-8' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2406 |
data = data[3:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2407 |
elif data[:4] == '\x00\x00\xfe\xff': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2408 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2409 |
sys.stderr.write('stripping BOM\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2410 |
if encoding != 'utf-32be': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2411 |
sys.stderr.write('trying utf-32be instead\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2412 |
encoding = 'utf-32be' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2413 |
data = data[4:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2414 |
elif data[:4] == '\xff\xfe\x00\x00': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2415 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2416 |
sys.stderr.write('stripping BOM\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2417 |
if encoding != 'utf-32le': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2418 |
sys.stderr.write('trying utf-32le instead\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2419 |
encoding = 'utf-32le' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2420 |
data = data[4:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2421 |
newdata = unicode(data, encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2422 |
if _debug: sys.stderr.write('successfully converted %s data to unicode\n' % encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2423 |
declmatch = re.compile('^<\?xml[^>]*?>') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2424 |
newdecl = '''<?xml version='1.0' encoding='utf-8'?>''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2425 |
if declmatch.search(newdata): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2426 |
newdata = declmatch.sub(newdecl, newdata) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2427 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2428 |
newdata = newdecl + u'\n' + newdata |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2429 |
return newdata.encode('utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2430 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2431 |
def _stripDoctype(data): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2432 |
'''Strips DOCTYPE from XML document, returns (rss_version, stripped_data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2433 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2434 |
rss_version may be 'rss091n' or None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2435 |
stripped_data is the same XML document, minus the DOCTYPE |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2436 |
''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2437 |
entity_pattern = re.compile(r'<!ENTITY([^>]*?)>', re.MULTILINE) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2438 |
data = entity_pattern.sub('', data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2439 |
doctype_pattern = re.compile(r'<!DOCTYPE([^>]*?)>', re.MULTILINE) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2440 |
doctype_results = doctype_pattern.findall(data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2441 |
doctype = doctype_results and doctype_results[0] or '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2442 |
if doctype.lower().count('netscape'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2443 |
version = 'rss091n' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2444 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2445 |
version = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2446 |
data = doctype_pattern.sub('', data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2447 |
return version, data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2448 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2449 |
def parse(url_file_stream_or_string, etag=None, modified=None, agent=None, referrer=None, handlers=[]): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2450 |
'''Parse a feed from a URL, file, stream, or string''' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2451 |
result = FeedParserDict() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2452 |
result['feed'] = FeedParserDict() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2453 |
result['entries'] = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2454 |
if _XML_AVAILABLE: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2455 |
result['bozo'] = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2456 |
if type(handlers) == types.InstanceType: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2457 |
handlers = [handlers] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2458 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2459 |
f = _open_resource(url_file_stream_or_string, etag, modified, agent, referrer, handlers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2460 |
data = f.read() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2461 |
except Exception, e: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2462 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2463 |
result['bozo_exception'] = e |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2464 |
data = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2465 |
f = None |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2466 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2467 |
# if feed is gzip-compressed, decompress it |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2468 |
if f and data and hasattr(f, 'headers'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2469 |
if gzip and f.headers.get('content-encoding', '') == 'gzip': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2470 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2471 |
data = gzip.GzipFile(fileobj=_StringIO(data)).read() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2472 |
except Exception, e: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2473 |
# Some feeds claim to be gzipped but they're not, so |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2474 |
# we get garbage. Ideally, we should re-request the |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2475 |
# feed without the 'Accept-encoding: gzip' header, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2476 |
# but we don't. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2477 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2478 |
result['bozo_exception'] = e |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2479 |
data = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2480 |
elif zlib and f.headers.get('content-encoding', '') == 'deflate': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2481 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2482 |
data = zlib.decompress(data, -zlib.MAX_WBITS) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2483 |
except Exception, e: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2484 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2485 |
result['bozo_exception'] = e |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2486 |
data = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2487 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2488 |
# save HTTP headers |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2489 |
if hasattr(f, 'info'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2490 |
info = f.info() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2491 |
result['etag'] = info.getheader('ETag') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2492 |
last_modified = info.getheader('Last-Modified') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2493 |
if last_modified: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2494 |
result['modified'] = _parse_date(last_modified) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2495 |
if hasattr(f, 'url'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2496 |
result['href'] = f.url |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2497 |
result['status'] = 200 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2498 |
if hasattr(f, 'status'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2499 |
result['status'] = f.status |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2500 |
if hasattr(f, 'headers'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2501 |
result['headers'] = f.headers.dict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2502 |
if hasattr(f, 'close'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2503 |
f.close() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2504 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2505 |
# there are four encodings to keep track of: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2506 |
# - http_encoding is the encoding declared in the Content-Type HTTP header |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2507 |
# - xml_encoding is the encoding declared in the <?xml declaration |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2508 |
# - sniffed_encoding is the encoding sniffed from the first 4 bytes of the XML data |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2509 |
# - result['encoding'] is the actual encoding, as per RFC 3023 and a variety of other conflicting specifications |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2510 |
http_headers = result.get('headers', {}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2511 |
result['encoding'], http_encoding, xml_encoding, sniffed_xml_encoding, acceptable_content_type = \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2512 |
_getCharacterEncoding(http_headers, data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2513 |
if http_headers and (not acceptable_content_type): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2514 |
if http_headers.has_key('content-type'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2515 |
bozo_message = '%s is not an XML media type' % http_headers['content-type'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2516 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2517 |
bozo_message = 'no Content-type specified' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2518 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2519 |
result['bozo_exception'] = NonXMLContentType(bozo_message) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2520 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2521 |
result['version'], data = _stripDoctype(data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2522 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2523 |
baseuri = http_headers.get('content-location', result.get('href')) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2524 |
baselang = http_headers.get('content-language', None) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2525 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2526 |
# if server sent 304, we're done |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2527 |
if result.get('status', 0) == 304: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2528 |
result['version'] = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2529 |
result['debug_message'] = 'The feed has not changed since you last checked, ' + \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2530 |
'so the server sent no data. This is a feature, not a bug!' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2531 |
return result |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2532 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2533 |
# if there was a problem downloading, we're done |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2534 |
if not data: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2535 |
return result |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2536 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2537 |
# determine character encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2538 |
use_strict_parser = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2539 |
known_encoding = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2540 |
tried_encodings = [] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2541 |
# try: HTTP encoding, declared XML encoding, encoding sniffed from BOM |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2542 |
for proposed_encoding in (result['encoding'], xml_encoding, sniffed_xml_encoding): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2543 |
if not proposed_encoding: continue |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2544 |
if proposed_encoding in tried_encodings: continue |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2545 |
tried_encodings.append(proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2546 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2547 |
data = _toUTF8(data, proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2548 |
known_encoding = use_strict_parser = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2549 |
break |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2550 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2551 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2552 |
# if no luck and we have auto-detection library, try that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2553 |
if (not known_encoding) and chardet: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2554 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2555 |
proposed_encoding = chardet.detect(data)['encoding'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2556 |
if proposed_encoding and (proposed_encoding not in tried_encodings): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2557 |
tried_encodings.append(proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2558 |
data = _toUTF8(data, proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2559 |
known_encoding = use_strict_parser = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2560 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2561 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2562 |
# if still no luck and we haven't tried utf-8 yet, try that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2563 |
if (not known_encoding) and ('utf-8' not in tried_encodings): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2564 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2565 |
proposed_encoding = 'utf-8' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2566 |
tried_encodings.append(proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2567 |
data = _toUTF8(data, proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2568 |
known_encoding = use_strict_parser = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2569 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2570 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2571 |
# if still no luck and we haven't tried windows-1252 yet, try that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2572 |
if (not known_encoding) and ('windows-1252' not in tried_encodings): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2573 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2574 |
proposed_encoding = 'windows-1252' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2575 |
tried_encodings.append(proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2576 |
data = _toUTF8(data, proposed_encoding) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2577 |
known_encoding = use_strict_parser = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2578 |
except: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2579 |
pass |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2580 |
# if still no luck, give up |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2581 |
if not known_encoding: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2582 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2583 |
result['bozo_exception'] = CharacterEncodingUnknown( \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2584 |
'document encoding unknown, I tried ' + \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2585 |
'%s, %s, utf-8, and windows-1252 but nothing worked' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2586 |
(result['encoding'], xml_encoding)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2587 |
result['encoding'] = '' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2588 |
elif proposed_encoding != result['encoding']: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2589 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2590 |
result['bozo_exception'] = CharacterEncodingOverride( \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2591 |
'documented declared as %s, but parsed as %s' % \ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2592 |
(result['encoding'], proposed_encoding)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2593 |
result['encoding'] = proposed_encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2594 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2595 |
if not _XML_AVAILABLE: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2596 |
use_strict_parser = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2597 |
if use_strict_parser: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2598 |
# initialize the SAX parser |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2599 |
feedparser = _StrictFeedParser(baseuri, baselang, 'utf-8') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2600 |
saxparser = xml.sax.make_parser(PREFERRED_XML_PARSERS) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2601 |
saxparser.setFeature(xml.sax.handler.feature_namespaces, 1) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2602 |
saxparser.setContentHandler(feedparser) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2603 |
saxparser.setErrorHandler(feedparser) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2604 |
source = xml.sax.xmlreader.InputSource() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2605 |
source.setByteStream(_StringIO(data)) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2606 |
if hasattr(saxparser, '_ns_stack'): |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2607 |
# work around bug in built-in SAX parser (doesn't recognize xml: namespace) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2608 |
# PyXML doesn't have this problem, and it doesn't have _ns_stack either |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2609 |
saxparser._ns_stack.append({'http://www.w3.org/XML/1998/namespace':'xml'}) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2610 |
try: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2611 |
saxparser.parse(source) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2612 |
except Exception, e: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2613 |
if _debug: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2614 |
import traceback |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2615 |
traceback.print_stack() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2616 |
traceback.print_exc() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2617 |
sys.stderr.write('xml parsing failed\n') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2618 |
result['bozo'] = 1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2619 |
result['bozo_exception'] = feedparser.exc or e |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2620 |
use_strict_parser = 0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2621 |
if not use_strict_parser: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2622 |
feedparser = _LooseFeedParser(baseuri, baselang, known_encoding and 'utf-8' or '') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2623 |
feedparser.feed(data) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2624 |
result['feed'] = feedparser.feeddata |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2625 |
result['entries'] = feedparser.entries |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2626 |
result['version'] = result['version'] or feedparser.version |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2627 |
result['namespaces'] = feedparser.namespacesInUse |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2628 |
return result |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2629 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2630 |
if __name__ == '__main__': |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2631 |
if not sys.argv[1:]: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2632 |
print __doc__ |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2633 |
sys.exit(0) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2634 |
else: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2635 |
urls = sys.argv[1:] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2636 |
zopeCompatibilityHack() |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2637 |
from pprint import pprint |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2638 |
for url in urls: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2639 |
print url |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2640 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2641 |
result = parse(url) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2642 |
pprint(result) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2643 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2644 |
|
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2645 |
#REVISION HISTORY |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2646 |
#1.0 - 9/27/2002 - MAP - fixed namespace processing on prefixed RSS 2.0 elements, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2647 |
# added Simon Fell's test suite |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2648 |
#1.1 - 9/29/2002 - MAP - fixed infinite loop on incomplete CDATA sections |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2649 |
#2.0 - 10/19/2002 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2650 |
# JD - use inchannel to watch out for image and textinput elements which can |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2651 |
# also contain title, link, and description elements |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2652 |
# JD - check for isPermaLink='false' attribute on guid elements |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2653 |
# JD - replaced openAnything with open_resource supporting ETag and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2654 |
# If-Modified-Since request headers |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2655 |
# JD - parse now accepts etag, modified, agent, and referrer optional |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2656 |
# arguments |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2657 |
# JD - modified parse to return a dictionary instead of a tuple so that any |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2658 |
# etag or modified information can be returned and cached by the caller |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2659 |
#2.0.1 - 10/21/2002 - MAP - changed parse() so that if we don't get anything |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2660 |
# because of etag/modified, return the old etag/modified to the caller to |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2661 |
# indicate why nothing is being returned |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2662 |
#2.0.2 - 10/21/2002 - JB - added the inchannel to the if statement, otherwise its |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2663 |
# useless. Fixes the problem JD was addressing by adding it. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2664 |
#2.1 - 11/14/2002 - MAP - added gzip support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2665 |
#2.2 - 1/27/2003 - MAP - added attribute support, admin:generatorAgent. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2666 |
# start_admingeneratoragent is an example of how to handle elements with |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2667 |
# only attributes, no content. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2668 |
#2.3 - 6/11/2003 - MAP - added USER_AGENT for default (if caller doesn't specify); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2669 |
# also, make sure we send the User-Agent even if urllib2 isn't available. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2670 |
# Match any variation of backend.userland.com/rss namespace. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2671 |
#2.3.1 - 6/12/2003 - MAP - if item has both link and guid, return both as-is. |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2672 |
#2.4 - 7/9/2003 - MAP - added preliminary Pie/Atom/Echo support based on Sam Ruby's |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2673 |
# snapshot of July 1 <http://www.intertwingly.net/blog/1506.html>; changed |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2674 |
# project name |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2675 |
#2.5 - 7/25/2003 - MAP - changed to Python license (all contributors agree); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2676 |
# removed unnecessary urllib code -- urllib2 should always be available anyway; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2677 |
# return actual url, status, and full HTTP headers (as result['url'], |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2678 |
# result['status'], and result['headers']) if parsing a remote feed over HTTP -- |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2679 |
# this should pass all the HTTP tests at <http://diveintomark.org/tests/client/http/>; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2680 |
# added the latest namespace-of-the-week for RSS 2.0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2681 |
#2.5.1 - 7/26/2003 - RMK - clear opener.addheaders so we only send our custom |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2682 |
# User-Agent (otherwise urllib2 sends two, which confuses some servers) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2683 |
#2.5.2 - 7/28/2003 - MAP - entity-decode inline xml properly; added support for |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2684 |
# inline <xhtml:body> and <xhtml:div> as used in some RSS 2.0 feeds |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2685 |
#2.5.3 - 8/6/2003 - TvdV - patch to track whether we're inside an image or |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2686 |
# textInput, and also to return the character encoding (if specified) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2687 |
#2.6 - 1/1/2004 - MAP - dc:author support (MarekK); fixed bug tracking |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2688 |
# nested divs within content (JohnD); fixed missing sys import (JohanS); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2689 |
# fixed regular expression to capture XML character encoding (Andrei); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2690 |
# added support for Atom 0.3-style links; fixed bug with textInput tracking; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2691 |
# added support for cloud (MartijnP); added support for multiple |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2692 |
# category/dc:subject (MartijnP); normalize content model: 'description' gets |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2693 |
# description (which can come from description, summary, or full content if no |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2694 |
# description), 'content' gets dict of base/language/type/value (which can come |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2695 |
# from content:encoded, xhtml:body, content, or fullitem); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2696 |
# fixed bug matching arbitrary Userland namespaces; added xml:base and xml:lang |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2697 |
# tracking; fixed bug tracking unknown tags; fixed bug tracking content when |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2698 |
# <content> element is not in default namespace (like Pocketsoap feed); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2699 |
# resolve relative URLs in link, guid, docs, url, comments, wfw:comment, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2700 |
# wfw:commentRSS; resolve relative URLs within embedded HTML markup in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2701 |
# description, xhtml:body, content, content:encoded, title, subtitle, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2702 |
# summary, info, tagline, and copyright; added support for pingback and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2703 |
# trackback namespaces |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2704 |
#2.7 - 1/5/2004 - MAP - really added support for trackback and pingback |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2705 |
# namespaces, as opposed to 2.6 when I said I did but didn't really; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2706 |
# sanitize HTML markup within some elements; added mxTidy support (if |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2707 |
# installed) to tidy HTML markup within some elements; fixed indentation |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2708 |
# bug in _parse_date (FazalM); use socket.setdefaulttimeout if available |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2709 |
# (FazalM); universal date parsing and normalization (FazalM): 'created', modified', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2710 |
# 'issued' are parsed into 9-tuple date format and stored in 'created_parsed', |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2711 |
# 'modified_parsed', and 'issued_parsed'; 'date' is duplicated in 'modified' |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2712 |
# and vice-versa; 'date_parsed' is duplicated in 'modified_parsed' and vice-versa |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2713 |
#2.7.1 - 1/9/2004 - MAP - fixed bug handling " and '. fixed memory |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2714 |
# leak not closing url opener (JohnD); added dc:publisher support (MarekK); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2715 |
# added admin:errorReportsTo support (MarekK); Python 2.1 dict support (MarekK) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2716 |
#2.7.4 - 1/14/2004 - MAP - added workaround for improperly formed <br/> tags in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2717 |
# encoded HTML (skadz); fixed unicode handling in normalize_attrs (ChrisL); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2718 |
# fixed relative URI processing for guid (skadz); added ICBM support; added |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2719 |
# base64 support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2720 |
#2.7.5 - 1/15/2004 - MAP - added workaround for malformed DOCTYPE (seen on many |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2721 |
# blogspot.com sites); added _debug variable |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2722 |
#2.7.6 - 1/16/2004 - MAP - fixed bug with StringIO importing |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2723 |
#3.0b3 - 1/23/2004 - MAP - parse entire feed with real XML parser (if available); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2724 |
# added several new supported namespaces; fixed bug tracking naked markup in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2725 |
# description; added support for enclosure; added support for source; re-added |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2726 |
# support for cloud which got dropped somehow; added support for expirationDate |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2727 |
#3.0b4 - 1/26/2004 - MAP - fixed xml:lang inheritance; fixed multiple bugs tracking |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2728 |
# xml:base URI, one for documents that don't define one explicitly and one for |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2729 |
# documents that define an outer and an inner xml:base that goes out of scope |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2730 |
# before the end of the document |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2731 |
#3.0b5 - 1/26/2004 - MAP - fixed bug parsing multiple links at feed level |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2732 |
#3.0b6 - 1/27/2004 - MAP - added feed type and version detection, result['version'] |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2733 |
# will be one of SUPPORTED_VERSIONS.keys() or empty string if unrecognized; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2734 |
# added support for creativeCommons:license and cc:license; added support for |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2735 |
# full Atom content model in title, tagline, info, copyright, summary; fixed bug |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2736 |
# with gzip encoding (not always telling server we support it when we do) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2737 |
#3.0b7 - 1/28/2004 - MAP - support Atom-style author element in author_detail |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2738 |
# (dictionary of 'name', 'url', 'email'); map author to author_detail if author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2739 |
# contains name + email address |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2740 |
#3.0b8 - 1/28/2004 - MAP - added support for contributor |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2741 |
#3.0b9 - 1/29/2004 - MAP - fixed check for presence of dict function; added |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2742 |
# support for summary |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2743 |
#3.0b10 - 1/31/2004 - MAP - incorporated ISO-8601 date parsing routines from |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2744 |
# xml.util.iso8601 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2745 |
#3.0b11 - 2/2/2004 - MAP - added 'rights' to list of elements that can contain |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2746 |
# dangerous markup; fiddled with decodeEntities (not right); liberalized |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2747 |
# date parsing even further |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2748 |
#3.0b12 - 2/6/2004 - MAP - fiddled with decodeEntities (still not right); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2749 |
# added support to Atom 0.2 subtitle; added support for Atom content model |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2750 |
# in copyright; better sanitizing of dangerous HTML elements with end tags |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2751 |
# (script, frameset) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2752 |
#3.0b13 - 2/8/2004 - MAP - better handling of empty HTML tags (br, hr, img, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2753 |
# etc.) in embedded markup, in either HTML or XHTML form (<br>, <br/>, <br />) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2754 |
#3.0b14 - 2/8/2004 - MAP - fixed CDATA handling in non-wellformed feeds under |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2755 |
# Python 2.1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2756 |
#3.0b15 - 2/11/2004 - MAP - fixed bug resolving relative links in wfw:commentRSS; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2757 |
# fixed bug capturing author and contributor URL; fixed bug resolving relative |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2758 |
# links in author and contributor URL; fixed bug resolvin relative links in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2759 |
# generator URL; added support for recognizing RSS 1.0; passed Simon Fell's |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2760 |
# namespace tests, and included them permanently in the test suite with his |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2761 |
# permission; fixed namespace handling under Python 2.1 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2762 |
#3.0b16 - 2/12/2004 - MAP - fixed support for RSS 0.90 (broken in b15) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2763 |
#3.0b17 - 2/13/2004 - MAP - determine character encoding as per RFC 3023 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2764 |
#3.0b18 - 2/17/2004 - MAP - always map description to summary_detail (Andrei); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2765 |
# use libxml2 (if available) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2766 |
#3.0b19 - 3/15/2004 - MAP - fixed bug exploding author information when author |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2767 |
# name was in parentheses; removed ultra-problematic mxTidy support; patch to |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2768 |
# workaround crash in PyXML/expat when encountering invalid entities |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2769 |
# (MarkMoraes); support for textinput/textInput |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2770 |
#3.0b20 - 4/7/2004 - MAP - added CDF support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2771 |
#3.0b21 - 4/14/2004 - MAP - added Hot RSS support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2772 |
#3.0b22 - 4/19/2004 - MAP - changed 'channel' to 'feed', 'item' to 'entries' in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2773 |
# results dict; changed results dict to allow getting values with results.key |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2774 |
# as well as results[key]; work around embedded illformed HTML with half |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2775 |
# a DOCTYPE; work around malformed Content-Type header; if character encoding |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2776 |
# is wrong, try several common ones before falling back to regexes (if this |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2777 |
# works, bozo_exception is set to CharacterEncodingOverride); fixed character |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2778 |
# encoding issues in BaseHTMLProcessor by tracking encoding and converting |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2779 |
# from Unicode to raw strings before feeding data to sgmllib.SGMLParser; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2780 |
# convert each value in results to Unicode (if possible), even if using |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2781 |
# regex-based parsing |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2782 |
#3.0b23 - 4/21/2004 - MAP - fixed UnicodeDecodeError for feeds that contain |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2783 |
# high-bit characters in attributes in embedded HTML in description (thanks |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2784 |
# Thijs van de Vossen); moved guid, date, and date_parsed to mapped keys in |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2785 |
# FeedParserDict; tweaked FeedParserDict.has_key to return True if asking |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2786 |
# about a mapped key |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2787 |
#3.0fc1 - 4/23/2004 - MAP - made results.entries[0].links[0] and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2788 |
# results.entries[0].enclosures[0] into FeedParserDict; fixed typo that could |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2789 |
# cause the same encoding to be tried twice (even if it failed the first time); |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2790 |
# fixed DOCTYPE stripping when DOCTYPE contained entity declarations; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2791 |
# better textinput and image tracking in illformed RSS 1.0 feeds |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2792 |
#3.0fc2 - 5/10/2004 - MAP - added and passed Sam's amp tests; added and passed |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2793 |
# my blink tag tests |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2794 |
#3.0fc3 - 6/18/2004 - MAP - fixed bug in _changeEncodingDeclaration that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2795 |
# failed to parse utf-16 encoded feeds; made source into a FeedParserDict; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2796 |
# duplicate admin:generatorAgent/@rdf:resource in generator_detail.url; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2797 |
# added support for image; refactored parse() fallback logic to try other |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2798 |
# encodings if SAX parsing fails (previously it would only try other encodings |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2799 |
# if re-encoding failed); remove unichr madness in normalize_attrs now that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2800 |
# we're properly tracking encoding in and out of BaseHTMLProcessor; set |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2801 |
# feed.language from root-level xml:lang; set entry.id from rdf:about; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2802 |
# send Accept header |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2803 |
#3.0 - 6/21/2004 - MAP - don't try iso-8859-1 (can't distinguish between |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2804 |
# iso-8859-1 and windows-1252 anyway, and most incorrectly marked feeds are |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2805 |
# windows-1252); fixed regression that could cause the same encoding to be |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2806 |
# tried twice (even if it failed the first time) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2807 |
#3.0.1 - 6/22/2004 - MAP - default to us-ascii for all text/* content types; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2808 |
# recover from malformed content-type header parameter with no equals sign |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2809 |
# ('text/xml; charset:iso-8859-1') |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2810 |
#3.1 - 6/28/2004 - MAP - added and passed tests for converting HTML entities |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2811 |
# to Unicode equivalents in illformed feeds (aaronsw); added and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2812 |
# passed tests for converting character entities to Unicode equivalents |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2813 |
# in illformed feeds (aaronsw); test for valid parsers when setting |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2814 |
# XML_AVAILABLE; make version and encoding available when server returns |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2815 |
# a 304; add handlers parameter to pass arbitrary urllib2 handlers (like |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2816 |
# digest auth or proxy support); add code to parse username/password |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2817 |
# out of url and send as basic authentication; expose downloading-related |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2818 |
# exceptions in bozo_exception (aaronsw); added __contains__ method to |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2819 |
# FeedParserDict (aaronsw); added publisher_detail (aaronsw) |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2820 |
#3.2 - 7/3/2004 - MAP - use cjkcodecs and iconv_codec if available; always |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2821 |
# convert feed to UTF-8 before passing to XML parser; completely revamped |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2822 |
# logic for determining character encoding and attempting XML parsing |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2823 |
# (much faster); increased default timeout to 20 seconds; test for presence |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2824 |
# of Location header on redirects; added tests for many alternate character |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2825 |
# encodings; support various EBCDIC encodings; support UTF-16BE and |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2826 |
# UTF16-LE with or without a BOM; support UTF-8 with a BOM; support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2827 |
# UTF-32BE and UTF-32LE with or without a BOM; fixed crashing bug if no |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2828 |
# XML parsers are available; added support for 'Content-encoding: deflate'; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2829 |
# send blank 'Accept-encoding: ' header if neither gzip nor zlib modules |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2830 |
# are available |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2831 |
#3.3 - 7/15/2004 - MAP - optimize EBCDIC to ASCII conversion; fix obscure |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2832 |
# problem tracking xml:base and xml:lang if element declares it, child |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2833 |
# doesn't, first grandchild redeclares it, and second grandchild doesn't; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2834 |
# refactored date parsing; defined public registerDateHandler so callers |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2835 |
# can add support for additional date formats at runtime; added support |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2836 |
# for OnBlog, Nate, MSSQL, Greek, and Hungarian dates (ytrewq1); added |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2837 |
# zopeCompatibilityHack() which turns FeedParserDict into a regular |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2838 |
# dictionary, required for Zope compatibility, and also makes command- |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2839 |
# line debugging easier because pprint module formats real dictionaries |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2840 |
# better than dictionary-like objects; added NonXMLContentType exception, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2841 |
# which is stored in bozo_exception when a feed is served with a non-XML |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2842 |
# media type such as 'text/plain'; respect Content-Language as default |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2843 |
# language if not xml:lang is present; cloud dict is now FeedParserDict; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2844 |
# generator dict is now FeedParserDict; better tracking of xml:lang, |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2845 |
# including support for xml:lang='' to unset the current language; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2846 |
# recognize RSS 1.0 feeds even when RSS 1.0 namespace is not the default |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2847 |
# namespace; don't overwrite final status on redirects (scenarios: |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2848 |
# redirecting to a URL that returns 304, redirecting to a URL that |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2849 |
# redirects to another URL with a different type of redirect); add |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2850 |
# support for HTTP 303 redirects |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2851 |
#4.0 - MAP - support for relative URIs in xml:base attribute; fixed |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2852 |
# encoding issue with mxTidy (phopkins); preliminary support for RFC 3229; |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2853 |
# support for Atom 1.0; support for iTunes extensions; new 'tags' for |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2854 |
# categories/keywords/etc. as array of dict |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2855 |
# {'term': term, 'scheme': scheme, 'label': label} to match Atom 1.0 |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2856 |
# terminology; parse RFC 822-style dates with no time; lots of other |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2857 |
# bug fixes |
c3d098d6fafa
Added feedparser to soc/utils and modified svn:externals for vendor to include feedparser svn
Pawel Solyga <Pawel.Solyga@gmail.com>
parents:
diff
changeset
|
2858 |
#4.1 - MAP - removed socket timeout; added support for chardet library |