diff -r 5ff1fc726848 -r c6bca38c1cbf parts/django/docs/ref/contrib/gis/layermapping.txt --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/parts/django/docs/ref/contrib/gis/layermapping.txt Sat Jan 08 11:20:57 2011 +0530 @@ -0,0 +1,220 @@ +.. _ref-layermapping: + +==================================== +``LayerMapping`` data import utility +==================================== + +.. module:: django.contrib.gis.utils.layermapping + :synopsis: Spatial data import utility for GeoDjango models. + +.. currentmodule:: django.contrib.gis.utils + +The :class:`LayerMapping` class provides a way to map the contents of +vector spatial data files (e.g. shapefiles) intoto GeoDjango models. + +This utility grew out of the author's personal needs to eliminate +the code repetition that went into pulling geometries and fields out of +a vector layer, converting to another coordinate system (e.g. WGS84), and +then inserting into a GeoDjango model. + +.. note:: + + Use of :class:`LayerMapping` requires GDAL. + +.. warning :: + + GIS data sources, like shapefiles, may be very large. If you find + that :class:`LayerMapping` is using too much memory, set + :setting:`DEBUG` to ``False`` in your settings. When :setting:`DEBUG` + is set to ``True``, Django :ref:`automatically logs ` + *every* SQL query -- thus, when SQL statements contain geometries, it is + easy to consume more memory than is typical. + +Example +======= + +1. You need a GDAL-supported data source, like a shapefile (here we're using + a simple polygon shapefile, ``test_poly.shp``, with three features):: + + >>> from django.contrib.gis.gdal import DataSource + >>> ds = DataSource('test_poly.shp') + >>> layer = ds[0] + >>> print layer.fields # Exploring the fields in the layer, we only want the 'str' field. + ['float', 'int', 'str'] + >>> print len(layer) # getting the number of features in the layer (should be 3) + 3 + >>> print layer.geom_type # Should be 'Polygon' + Polygon + >>> print layer.srs # WGS84 in WKT + GEOGCS["GCS_WGS_1984", + DATUM["WGS_1984", + SPHEROID["WGS_1984",6378137,298.257223563]], + PRIMEM["Greenwich",0], + UNIT["Degree",0.017453292519943295]] + +2. Now we define our corresponding Django model (make sure to use ``syncdb``):: + + from django.contrib.gis.db import models + + class TestGeo(models.Model): + name = models.CharField(max_length=25) # corresponds to the 'str' field + poly = models.PolygonField(srid=4269) # we want our model in a different SRID + objects = models.GeoManager() + def __unicode__(self): + return 'Name: %s' % self.name + +3. Use :class:`LayerMapping` to extract all the features and place them in the + database:: + + >>> from django.contrib.gis.utils import LayerMapping + >>> from geoapp.models import TestGeo + >>> mapping = {'name' : 'str', # The 'name' model field maps to the 'str' layer field. + 'poly' : 'POLYGON', # For geometry fields use OGC name. + } # The mapping is a dictionary + >>> lm = LayerMapping(TestGeo, 'test_poly.shp', mapping) + >>> lm.save(verbose=True) # Save the layermap, imports the data. + Saved: Name: 1 + Saved: Name: 2 + Saved: Name: 3 + +Here, :class:`LayerMapping` just transformed the three geometries from the +shapefile in their original spatial reference system (WGS84) to the spatial +reference system of the GeoDjango model (NAD83). If no spatial reference +system is defined for the layer, use the ``source_srs`` keyword with a +:class:`~django.contrib.gis.gdal.SpatialReference` object to specify one. + +``LayerMapping`` API +==================== + +.. class:: LayerMapping(model, data_source, mapping[, layer=0, source_srs=None, encoding=None, transaction_mode='commit_on_success', transform=True, unique=True, using='default']) + +The following are the arguments and keywords that may be used during +instantiation of ``LayerMapping`` objects. + +================= ========================================================= +Argument Description +================= ========================================================= +``model`` The geographic model, *not* an instance. + +``data_source`` The path to the OGR-supported data source file + (e.g., a shapefile). Also accepts + :class:`django.contrib.gis.gdal.DataSource` instances. + +``mapping`` A dictionary: keys are strings corresponding to + the model field, and values correspond to + string field names for the OGR feature, or if the + model field is a geographic then it should + correspond to the OGR geometry type, + e.g., ``'POINT'``, ``'LINESTRING'``, ``'POLYGON'``. +================= ========================================================= + +===================== ===================================================== +Keyword Arguments +===================== ===================================================== +``layer`` The index of the layer to use from the Data Source + (defaults to 0) + +``source_srs`` Use this to specify the source SRS manually (for + example, some shapefiles don't come with a '.prj' + file). An integer SRID, WKT or PROJ.4 strings, and + :class:`django.contrib.gis.gdal.SpatialReference` + objects are accepted. + +``encoding`` Specifies the character set encoding of the strings + in the OGR data source. For example, ``'latin-1'``, + ``'utf-8'``, and ``'cp437'`` are all valid encoding + parameters. + +``transaction_mode`` May be ``'commit_on_success'`` (default) or + ``'autocommit'``. + +``transform`` Setting this to False will disable coordinate + transformations. In other words, geometries will + be inserted into the database unmodified from their + original state in the data source. + +``unique`` Setting this to the name, or a tuple of names, + from the given model will create models unique + only to the given name(s). Geometries will from + each feature will be added into the collection + associated with the unique model. Forces + the transaction mode to be ``'autocommit'``. + +``using`` New in version 1.2. Sets the database to use when + importing spatial data. Default is ``'default'`` +===================== ===================================================== + +``save()`` Keyword Arguments +---------------------------- + +.. method:: LayerMapping.save([verbose=False, fid_range=False, step=False, progress=False, silent=False, stream=sys.stdout, strict=False]) + +The ``save()`` method also accepts keywords. These keywords are +used for controlling output logging, error handling, and for importing +specific feature ranges. + +=========================== ================================================= +Save Keyword Arguments Description +=========================== ================================================= +``fid_range`` May be set with a slice or tuple of + (begin, end) feature ID's to map from + the data source. In other words, this + keyword enables the user to selectively + import a subset range of features in the + geographic data source. + +``progress`` When this keyword is set, status information + will be printed giving the number of features + processed and successfully saved. By default, + progress information will be printed every 1000 + features processed, however, this default may + be overridden by setting this keyword with an + integer for the desired interval. + +``silent`` By default, non-fatal error notifications are + printed to ``sys.stdout``, but this keyword may + be set to disable these notifications. + +``step`` If set with an integer, transactions will + occur at every step interval. For example, if + ``step=1000``, a commit would occur after the + 1,000th feature, the 2,000th feature etc. + + +``stream`` Status information will be written to this file + handle. Defaults to using ``sys.stdout``, but + any object with a ``write`` method is supported. + +``strict`` Execution of the model mapping will cease upon + the first error encountered. The default value + (``False``) + behavior is to attempt to continue. + +``verbose`` If set, information will be printed + subsequent to each model save + executed on the database. +=========================== ================================================= + +Troubleshooting +=============== + +Running out of memory +--------------------- + +As noted in the warning at the top of this section, Django stores all SQL +queries when ``DEBUG=True``. Set ``DEBUG=False`` in your settings, and this +should stop excessive memory use when running ``LayerMapping`` scripts. + +MySQL: ``max_allowed_packet`` error +----------------------------------- + +If you encounter the following error when using ``LayerMapping`` and MySQL:: + + OperationalError: (1153, "Got a packet bigger than 'max_allowed_packet' bytes") + +Then the solution is to increase the value of the ``max_allowed_packet`` +setting in your MySQL configuration. For example, the default value may +be something low like one megabyte -- the setting may be modified in MySQL's +configuration file (``my.cnf``) in the ``[mysqld]`` section:: + + max_allowed_packet = 10M