======================

Database API reference

======================

Once you've created your `data models`_, Django automatically gives you a

database-abstraction API that lets you create, retrieve, update and delete

objects. This document explains that API.

.. _`data models`: ../model_api/

Throughout this reference, we'll refer to the following models, which comprise

a weblog application::

    class Blog(models.Model):

        name = models.CharField(maxlength=100)

        tagline = models.TextField()

        def __str__(self):

            return self.name

    class Author(models.Model):

        name = models.CharField(maxlength=50)

        email = models.URLField()

        def __str__(self):

            return self.name

    class Entry(models.Model):

        blog = models.ForeignKey(Blog)

        headline = models.CharField(maxlength=255)

        body_text = models.TextField()

        pub_date = models.DateTimeField()

        authors = models.ManyToManyField(Author)

        def __str__(self):

            return self.headline

Creating objects

================

To represent database-table data in Python objects, Django uses an intuitive

system: A model class represents a database table, and an instance of that

class represents a particular record in the database table.

To create an object, instantiate it using keyword arguments to the model class,

then call ``save()`` to save it to the database.

You import the model class from wherever it lives on the Python path, as you

may expect. (We point this out here because previous Django versions required

funky model importing.)

Assuming models live in a file ``mysite/blog/models.py``, here's an example::

    from mysite.blog.models import Blog

    b = Blog(name='Beatles Blog', tagline='All the latest Beatles news.')

    b.save()

This performs an ``INSERT`` SQL statement behind the scenes. Django doesn't hit

the database until you explicitly call ``save()``.

The ``save()`` method has no return value.

To create an object and save it all in one step see the `create`__ method.

__ `create(**kwargs)`_

Auto-incrementing primary keys

------------------------------

If a model has an ``AutoField`` -- an auto-incrementing primary key -- then

that auto-incremented value will be calculated and saved as an attribute on

your object the first time you call ``save()``.

Example::

    b2 = Blog(name='Cheddar Talk', tagline='Thoughts on cheese.')

    b2.id     # Returns None, because b doesn't have an ID yet.

    b2.save()

    b2.id     # Returns the ID of your new object.

There's no way to tell what the value of an ID will be before you call

``save()``, because that value is calculated by your database, not by Django.

(For convenience, each model has an ``AutoField`` named ``id`` by default

unless you explicitly specify ``primary_key=True`` on a field. See the

`AutoField documentation`_.)

.. _AutoField documentation: ../model_api/#autofield

Explicitly specifying auto-primary-key values

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If a model has an ``AutoField`` but you want to define a new object's ID

explicitly when saving, just define it explicitly before saving, rather than

relying on the auto-assignment of the ID.

Example::

    b3 = Blog(id=3, name='Cheddar Talk', tagline='Thoughts on cheese.')

    b3.id     # Returns 3.

    b3.save()

    b3.id     # Returns 3.

If you assign auto-primary-key values manually, make sure not to use an

already-existing primary-key value! If you create a new object with an explicit

primary-key value that already exists in the database, Django will assume

you're changing the existing record rather than creating a new one.

Given the above ``'Cheddar Talk'`` blog example, this example would override

the previous record in the database::

    b4 = Blog(id=3, name='Not Cheddar', tagline='Anything but cheese.')

    b4.save()  # Overrides the previous blog with ID=3!

See _`How Django knows to UPDATE vs. INSERT`, below, for the reason this

happens.

Explicitly specifying auto-primary-key values is mostly useful for bulk-saving

objects, when you're confident you won't have primary-key collision.

Saving changes to objects

=========================

To save changes to an object that's already in the database, use ``save()``.

Given a ``Blog`` instance ``b5`` that has already been saved to the database,

this example changes its name and updates its record in the database::

    b5.name = 'New name'

    b5.save()

This performs an ``UPDATE`` SQL statement behind the scenes. Django doesn't hit

the database until you explicitly call ``save()``.

The ``save()`` method has no return value.

How Django knows to UPDATE vs. INSERT

-------------------------------------

You may have noticed Django database objects use the same ``save()`` method

for creating and changing objects. Django abstracts the need to use ``INSERT``

or ``UPDATE`` SQL statements. Specifically, when you call ``save()``, Django

follows this algorithm:

    * If the object's primary key attribute is set to a value that evaluates to

      ``True`` (i.e., a value other than ``None`` or the empty string), Django 

      executes a ``SELECT`` query to determine whether a record with the given 

      primary key already exists.

    * If the record with the given primary key does already exist, Django

      executes an ``UPDATE`` query.

    * If the object's primary key attribute is *not* set, or if it's set but a

      record doesn't exist, Django executes an ``INSERT``.

The one gotcha here is that you should be careful not to specify a primary-key

value explicitly when saving new objects, if you cannot guarantee the

primary-key value is unused. For more on this nuance, see

"Explicitly specifying auto-primary-key values" above.

Retrieving objects

==================

To retrieve objects from your database, you construct a ``QuerySet`` via a

``Manager`` on your model class.

A ``QuerySet`` represents a collection of objects from your database. It can

have zero, one or many *filters* -- criteria that narrow down the collection

based on given parameters. In SQL terms, a ``QuerySet`` equates to a ``SELECT``

statement, and a filter is a limiting clause such as ``WHERE`` or ``LIMIT``.

You get a ``QuerySet`` by using your model's ``Manager``. Each model has at

least one ``Manager``, and it's called ``objects`` by default. Access it

directly via the model class, like so::

    Blog.objects  # <django.db.models.manager.Manager object at ...>

    b = Blog(name='Foo', tagline='Bar')

    b.objects     # AttributeError: "Manager isn't accessible via Blog instances."

(``Managers`` are accessible only via model classes, rather than from model

instances, to enforce a separation between "table-level" operations and

"record-level" operations.)

The ``Manager`` is the main source of ``QuerySets`` for a model. It acts as a

"root" ``QuerySet`` that describes all objects in the model's database table.

For example, ``Blog.objects`` is the initial ``QuerySet`` that contains all

``Blog`` objects in the database.

Retrieving all objects

----------------------

The simplest way to retrieve objects from a table is to get all of them.

To do this, use the ``all()`` method on a ``Manager``.

Example::

    all_entries = Entry.objects.all()

The ``all()`` method returns a ``QuerySet`` of all the objects in the database.

(If ``Entry.objects`` is a ``QuerySet``, why can't we just do ``Entry.objects``?

That's because ``Entry.objects``, the root ``QuerySet``, is a special case

that cannot be evaluated. The ``all()`` method returns a ``QuerySet`` that

*can* be evaluated.)

Filtering objects

-----------------

The root ``QuerySet`` provided by the ``Manager`` describes all objects in the

database table. Usually, though, you'll need to select only a subset of the

complete set of objects.

To create such a subset, you refine the initial ``QuerySet``, adding filter

conditions. The two most common ways to refine a ``QuerySet`` are:

``filter(**kwargs)``

    Returns a new ``QuerySet`` containing objects that match the given lookup

    parameters.

``exclude(**kwargs)``

    Returns a new ``QuerySet`` containing objects that do *not* match the given

    lookup parameters.

The lookup parameters (``**kwargs`` in the above function definitions) should

be in the format described in `Field lookups`_ below.

For example, to get a ``QuerySet`` of blog entries from the year 2006, use

``filter()`` like so::

    Entry.objects.filter(pub_date__year=2006)

(Note we don't have to add an ``all()`` -- ``Entry.objects.all().filter(...)``.

That would still work, but you only need ``all()`` when you want all objects

from the root ``QuerySet``.)

Chaining filters

~~~~~~~~~~~~~~~~

The result of refining a ``QuerySet`` is itself a ``QuerySet``, so it's

possible to chain refinements together. For example::

    Entry.objects.filter(

        headline__startswith='What').exclude(

            pub_date__gte=datetime.now()).filter(

                pub_date__gte=datetime(2005, 1, 1))

...takes the initial ``QuerySet`` of all entries in the database, adds a

filter, then an exclusion, then another filter. The final result is a

``QuerySet`` containing all entries with a headline that starts with "What",

that were published between January 1, 2005, and the current day.

Filtered QuerySets are unique

-----------------------------

Each time you refine a ``QuerySet``, you get a brand-new ``QuerySet`` that is

in no way bound to the previous ``QuerySet``. Each refinement creates a

separate and distinct ``QuerySet`` that can be stored, used and reused.

Example::

    q1 = Entry.objects.filter(headline__startswith="What")

    q2 = q1.exclude(pub_date__gte=datetime.now())

    q3 = q1.filter(pub_date__gte=datetime.now())

These three ``QuerySets`` are separate. The first is a base ``QuerySet``

containing all entries that contain a headline starting with "What". The second

is a subset of the first, with an additional criteria that excludes records

whose ``pub_date`` is greater than now. The third is a subset of the first,

with an additional criteria that selects only the records whose ``pub_date`` is

greater than now. The initial ``QuerySet`` (``q1``) is unaffected by the

refinement process.

QuerySets are lazy

------------------

``QuerySets`` are lazy -- the act of creating a ``QuerySet`` doesn't involve

any database activity. You can stack filters together all day long, and Django

won't actually run the query until the ``QuerySet`` is *evaluated*.

When QuerySets are evaluated

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can evaluate a ``QuerySet`` in the following ways:

    * **Iteration.** A ``QuerySet`` is iterable, and it executes its database

      query the first time you iterate over it. For example, this will print

      the headline of all entries in the database::

          for e in Entry.objects.all():

              print e.headline

    * **Slicing.** As explained in `Limiting QuerySets`_ below, a ``QuerySet``

      can be sliced, using Python's array-slicing syntax. Usually slicing a

      ``QuerySet`` returns another (unevaluated )``QuerySet``, but Django will

      execute the database query if you use the "step" parameter of slice

      syntax.

    * **repr().** A ``QuerySet`` is evaluated when you call ``repr()`` on it.

      This is for convenience in the Python interactive interpreter, so you can

      immediately see your results when using the API interactively.

    * **len().** A ``QuerySet`` is evaluated when you call ``len()`` on it.

      This, as you might expect, returns the length of the result list.

      Note: *Don't* use ``len()`` on ``QuerySet``\s if all you want to do is

      determine the number of records in the set. It's much more efficient to

      handle a count at the database level, using SQL's ``SELECT COUNT(*)``,

      and Django provides a ``count()`` method for precisely this reason. See

      ``count()`` below.

    * **list().** Force evaluation of a ``QuerySet`` by calling ``list()`` on

      it. For example::

          entry_list = list(Entry.objects.all())

      Be warned, though, that this could have a large memory overhead, because

      Django will load each element of the list into memory. In contrast,

      iterating over a ``QuerySet`` will take advantage of your database to

      load data and instantiate objects only as you need them.

Limiting QuerySets

------------------

Use Python's array-slicing syntax to limit your ``QuerySet`` to a certain

number of results. This is the equivalent of SQL's ``LIMIT`` and ``OFFSET``

clauses.

For example, this returns the first 5 objects (``LIMIT 5``)::

    Entry.objects.all()[:5]

This returns the fifth through tenth objects (``OFFSET 5 LIMIT 5``)::

    Entry.objects.all()[5:10]

Generally, slicing a ``QuerySet`` returns a new ``QuerySet`` -- it doesn't

evaluate the query. An exception is if you use the "step" parameter of Python

slice syntax. For example, this would actually execute the query in order to

return a list of every *second* object of the first 10::

    Entry.objects.all()[:10:2]

To retrieve a *single* object rather than a list

(e.g. ``SELECT foo FROM bar LIMIT 1``), use a simple index instead of a

slice. For example, this returns the first ``Entry`` in the database, after

ordering entries alphabetically by headline::

    Entry.objects.order_by('headline')[0]

This is roughly equivalent to::

    Entry.objects.order_by('headline')[0:1].get()

Note, however, that the first of these will raise ``IndexError`` while the

second will raise ``DoesNotExist`` if no objects match the given criteria.

QuerySet methods that return new QuerySets

------------------------------------------

Django provides a range of ``QuerySet`` refinement methods that modify either

the types of results returned by the ``QuerySet`` or the way its SQL query is

executed.

``filter(**kwargs)``

~~~~~~~~~~~~~~~~~~~~

Returns a new ``QuerySet`` containing objects that match the given lookup

parameters.

The lookup parameters (``**kwargs``) should be in the format described in

`Field lookups`_ below. Multiple parameters are joined via ``AND`` in the

underlying SQL statement.

``exclude(**kwargs)``

~~~~~~~~~~~~~~~~~~~~~

Returns a new ``QuerySet`` containing objects that do *not* match the given

lookup parameters.

The lookup parameters (``**kwargs``) should be in the format described in

`Field lookups`_ below. Multiple parameters are joined via ``AND`` in the

underlying SQL statement, and the whole thing is enclosed in a ``NOT()``.

This example excludes all entries whose ``pub_date`` is the current date/time

AND whose ``headline`` is "Hello"::

    Entry.objects.exclude(pub_date__gt=datetime.date(2005, 1, 3), headline='Hello')

In SQL terms, that evaluates to::

    SELECT ...

    WHERE NOT (pub_date > '2005-1-3' AND headline = 'Hello')

This example excludes all entries whose ``pub_date`` is the current date/time

OR whose ``headline`` is "Hello"::

    Entry.objects.exclude(pub_date__gt=datetime.date(2005, 1, 3)).exclude(headline='Hello')

In SQL terms, that evaluates to::

    SELECT ...

    WHERE NOT pub_date > '2005-1-3'

    AND NOT headline = 'Hello'

Note the second example is more restrictive.

``order_by(*fields)``

~~~~~~~~~~~~~~~~~~~~~

By default, results returned by a ``QuerySet`` are ordered by the ordering

tuple given by the ``ordering`` option in the model's ``Meta``. You can

override this on a per-``QuerySet`` basis by using the ``order_by`` method.

Example::

    Entry.objects.filter(pub_date__year=2005).order_by('-pub_date', 'headline')

The result above will be ordered by ``pub_date`` descending, then by

``headline`` ascending. The negative sign in front of ``"-pub_date"`` indicates

*descending* order. Ascending order is implied. To order randomly, use ``"?"``,

like so::

    Entry.objects.order_by('?')

To order by a field in a different table, add the other table's name and a dot,

like so::

    Entry.objects.order_by('blogs_blog.name', 'headline')

There's no way to specify whether ordering should be case sensitive. With

respect to case-sensitivity, Django will order results however your database

backend normally orders them.

``distinct()``

~~~~~~~~~~~~~~

Returns a new ``QuerySet`` that uses ``SELECT DISTINCT`` in its SQL query. This

eliminates duplicate rows from the query results.

By default, a ``QuerySet`` will not eliminate duplicate rows. In practice, this

is rarely a problem, because simple queries such as ``Blog.objects.all()``

don't introduce the possibility of duplicate result rows.

However, if your query spans multiple tables, it's possible to get duplicate

results when a ``QuerySet`` is evaluated. That's when you'd use ``distinct()``.

``values(*fields)``

~~~~~~~~~~~~~~~~~~~

Returns a ``ValuesQuerySet`` -- a ``QuerySet`` that evaluates to a list of

dictionaries instead of model-instance objects.

Each of those dictionaries represents an object, with the keys corresponding to

the attribute names of model objects.

This example compares the dictionaries of ``values()`` with the normal model

objects::

    # This list contains a Blog object.

    >>> Blog.objects.filter(name__startswith='Beatles')

    [Beatles Blog]

    # This list contains a dictionary.

    >>> Blog.objects.filter(name__startswith='Beatles').values()

    [{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}]

``values()`` takes optional positional arguments, ``*fields``, which specify

field names to which the ``SELECT`` should be limited. If you specify the

fields, each dictionary will contain only the field keys/values for the fields

you specify. If you don't specify the fields, each dictionary will contain a

key and value for every field in the database table.

Example::

    >>> Blog.objects.values()

    [{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}],

    >>> Blog.objects.values('id', 'name')

    [{'id': 1, 'name': 'Beatles Blog'}]

A ``ValuesQuerySet`` is useful when you know you're only going to need values

from a small number of the available fields and you won't need the

functionality of a model instance object. It's more efficient to select only

the fields you need to use.

Finally, note a ``ValuesQuerySet`` is a subclass of ``QuerySet``, so it has all

methods of ``QuerySet``. You can call ``filter()`` on it, or ``order_by()``, or

whatever. Yes, that means these two calls are identical::

    Blog.objects.values().order_by('id')

    Blog.objects.order_by('id').values()

The people who made Django prefer to put all the SQL-affecting methods first,

followed (optionally) by any output-affecting methods (such as ``values()``),

but it doesn't really matter. This is your chance to really flaunt your

individualism.

``dates(field, kind, order='ASC')``

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Returns a ``DateQuerySet`` -- a ``QuerySet`` that evaluates to a list of

``datetime.datetime`` objects representing all available dates of a particular

kind within the contents of the ``QuerySet``.

``field`` should be the name of a ``DateField`` or ``DateTimeField`` of your

model.

``kind`` should be either ``"year"``, ``"month"`` or ``"day"``. Each

``datetime.datetime`` object in the result list is "truncated" to the given

``type``.

    * ``"year"`` returns a list of all distinct year values for the field.

    * ``"month"`` returns a list of all distinct year/month values for the field.

    * ``"day"`` returns a list of all distinct year/month/day values for the field.

``order``, which defaults to ``'ASC'``, should be either ``'ASC'`` or

``'DESC'``. This specifies how to order the results.

Examples::

    >>> Entry.objects.dates('pub_date', 'year')

    [datetime.datetime(2005, 1, 1)]

    >>> Entry.objects.dates('pub_date', 'month')

    [datetime.datetime(2005, 2, 1), datetime.datetime(2005, 3, 1)]

    >>> Entry.objects.dates('pub_date', 'day')

    [datetime.datetime(2005, 2, 20), datetime.datetime(2005, 3, 20)]

    >>> Entry.objects.dates('pub_date', 'day', order='DESC')

    [datetime.datetime(2005, 3, 20), datetime.datetime(2005, 2, 20)]

    >>> Entry.objects.filter(headline__contains='Lennon').dates('pub_date', 'day')

    [datetime.datetime(2005, 3, 20)]

``none()``