Appendix C: Database API Reference

Creating Objects

To create an object, instantiate it using keyword arguments to the model class, and then call save() to save it to the database:

>>> from mysite.blog.models import Blog
>>> b = Blog(name='Beatles Blog', tagline='All the latest Beatles news.')
>>> b.save()

This performs an INSERT SQL statement behind the scenes. Django doesnt hit the database until you explicitly call save() .

The save() method has no return value.

To create an object and save it all in one step see the create manager method discussed shortly.

What Happens When You Save?

When you save an object, Django performs the following steps:

Emit a pre_save signal. This provides a notification that an object is about to be saved. You can register a listener that will be invoked whenever this signal is emitted. These signals are still in development and werent documented when this book went to press; check the online documentation for the latest information.

Preprocess the data. Each field on the object is asked to perform any automated data modification that the field may need to perform.

Most fields do no preprocessing the field data is kept as is. Preprocessing is only used on fields that have special behavior, like file fields.

Prepare the data for the database. Each field is asked to provide its current value in a data type that can be written to the database.

Most fields require no data preparation. Simple data types, such as integers and strings, are ready to write as a Python object. However, more complex data types often require some modification. For example, DateFields use a Python datetime object to store data. Databases dont store datetime objects, so the field value must be converted into an ISO-compliant date string for insertion into the database.

Insert the data into the database. The preprocessed, prepared data is then composed into an SQL statement for insertion into the database.

Emit a post_save signal. As with the pre_save signal, this is used to provide notification that an object has been successfully saved. Again, these signals are not yet documented.

Autoincrementing Primary Keys

For convenience, each model is given an autoincrementing primary key field named id unless you explicitly specify primary_key=True on a field (see the section titled AutoField in Appendix B).

If your model has an AutoField , that autoincremented value will be calculated and saved as an attribute on your object the first time you call save() :

>>> b2 = Blog(name='Cheddar Talk', tagline='Thoughts on cheese.')
>>> b2.id     # Returns None, because b doesn't have an ID yet.
None

>>> b2.save()
>>> b2.id     # Returns the ID of your new object.
14

Theres no way to tell what the value of an ID will be before you call save() , because that value is calculated by your database, not by Django.

If a model has an AutoField but you want to define a new objects ID explicitly when saving, just define it explicitly before saving, rather than relying on the autoassignment of the ID:

>>> b3 = Blog(id=3, name='Cheddar Talk', tagline='Thoughts on cheese.')
>>> b3.id
3
>>> b3.save()
>>> b3.id
3

If you assign auto-primary-key values manually, make sure not to use an already existing primary key value! If you create a new object with an explicit primary key value that already exists in the database, Django will assume youre changing the existing record rather than creating a new one.

Given the preceding 'Cheddar Talk' blog example, this example would override the previous record in the database:

>>> b4 = Blog(id=3, name='Not Cheddar', tagline='Anything but cheese.')
>>> b4.save()  # Overrides the previous blog with ID=3!

Explicitly specifying auto-primary-key values is mostly useful for bulk-saving objects, when youre confident you wont have primary key collision.

Saving Changes to Objects

To save changes to an object thats already in the database, use save() .

Given a Blog instance b5 that has already been saved to the database, this example changes its name and updates its record in the database:

>>> b5.name = 'New name'
>>> b5.save()

This performs an UPDATE SQL statement behind the scenes. Again, Django doesnt hit the database until you explicitly call save() .

How Django Knows When to UPDATE and When to INSERT

You may have noticed that Django database objects use the same save() method for creating and changing objects. Django abstracts the need to use INSERT or UPDATE SQL statements. Specifically, when you call save() , Django follows this algorithm:

If the objects primary key attribute is set to a value that evaluates to True (i.e., a value other than None or the empty string), Django executes a SELECT query to determine whether a record with the given primary key already exists.

If the record with the given primary key does already exist, Django executes an UPDATE query.

If the objects primary key attribute is not set, or if its set but a record doesnt exist, Django executes an INSERT .

Because of this, you should be careful not to specify a primary key value explicitly when saving new objects if you cannot guarantee the primary key value is unused.

Updating ForeignKey fields works exactly the same way; simply assign an object of the right type to the field in question:

>>> joe = Author.objects.create(name="Joe")
>>> entry.author = joe
>>> entry.save()

Django will complain if you try to assign an object of the wrong type.

Retrieving Objects

Throughout the book youve seen objects retrieved using code like the following:

>>> blogs = Blog.objects.filter(author__name__contains="Joe")

There are quite a few moving parts behind the scenes here: when you retrieve objects from the database, youre actually constructing a QuerySet using the models Manager . This QuerySet knows how to execute SQL and return the requested objects.

Appendix B looked at both of these objects from a model-definition point of view; now well look at how they operate.

A QuerySet represents a collection of objects from your database. It can have zero, one, or many filters criteria that narrow down the collection based on given parameters. In SQL terms, a QuerySet equates to a SELECT statement, and a filter is a limiting clause such as WHERE or LIMIT .

You get a QuerySet by using your models Manager . Each model has at least one Manager , and its called objects by default. Access it directly via the model class, like so:

>>> Blog.objects
<django.db.models.manager.Manager object at 0x137d00d>

Manager s are accessible only via model classes, rather than from model instances, to enforce a separation between table-level operations and record-level operations:

>>> b = Blog(name='Foo', tagline='Bar')
>>> b.objects
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Manager isn't accessible via Blog instances.

The Manager is the main source of QuerySets for a model. It acts as a root QuerySet that describes all objects in the models database table. For example, Blog.objects is the initial QuerySet that contains all Blog objects in the database.

Caching and QuerySets

Each QuerySet contains a cache, to minimize database access. Its important to understand how it works, in order to write the most efficient code.

In a newly created QuerySet , the cache is empty. The first time a QuerySet is evaluated and, hence, a database query happens Django saves the query results in the QuerySet s cache and returns the results that have been explicitly requested (e.g., the next element, if the QuerySet is being iterated over). Subsequent evaluations of the QuerySet reuse the cached results.

Keep this caching behavior in mind, because it may bite you if you dont use your QuerySet``s correctly. For example, the following will create two ``QuerySet s, evaluate them, and throw them away:

print [e.headline for e in Entry.objects.all()]
print [e.pub_date for e in Entry.objects.all()]

That means the same database query will be executed twice, effectively doubling your database load. Also, theres a possibility the two lists may not include the same database records, because an Entry may have been added or deleted in the split second between the two requests.

To avoid this problem, simply save the QuerySet and reuse it:

queryset = Poll.objects.all()
print [p.headline for p in queryset] # Evaluate the query set.
print [p.pub_date for p in queryset] # Reuse the cache from the evaluation.

Filtering Objects

The simplest way to retrieve objects from a table is to get all of them. To do this, use the all() method on a Manager :

>>> Entry.objects.all()

The all() method returns a QuerySet of all the objects in the database.

Usually, though, youll need to select only a subset of the complete set of objects. To create such a subset, you refine the initial QuerySet , adding filter conditions. Youll usually do this using the filter() and/or exclude() methods:

>>> y2006 = Entry.objects.filter(pub_date__year=2006)
>>> not2006 = Entry.objects.exclude(pub_date__year=2006)

filter() and exclude() both take field lookup arguments, which are discussed in detail shortly.

Chaining Filters

The result of refining a QuerySet is itself a QuerySet , so its possible to chain refinements together, for example:

>>> qs = Entry.objects.filter(headline__startswith='What')
>>> qs = qs..exclude(pub_date__gte=datetime.datetime.now())
>>> qs = qs.filter(pub_date__gte=datetime.datetime(2005, 1, 1))

This takes the initial QuerySet of all entries in the database, adds a filter, then an exclusion, and then another filter. The final result is a QuerySet containing all entries with a headline that starts with What that were published between January 1, 2005, and the current day.

Its important to point out here that QuerySets are lazy the act of creating a QuerySet doesnt involve any database activity. In fact, the three preceding lines dont make any database calls; you can chain filters together all day long and Django wont actually run the query until the QuerySet is evaluated .

You can evaluate a QuerySet in any following ways:

Iterating : A QuerySet is iterable, and it executes its database query the first time you iterate over it. For example, the following QuerySet isnt evaluated until its iterated over in the for loop:

qs = Entry.objects.filter(pub_date__year=2006)
qs = qs.filter(headline__icontains="bill")
for e in qs:
    print e.headline

This prints all headlines from 2006 that contain bill but causes only one database hit.

Printing it : A QuerySet is evaluated when you call repr() on it. This is for convenience in the Python interactive interpreter, so you can immediately see your results when using the API interactively.

Slicing : As explained in the upcoming Limiting QuerySets section, a QuerySet can be sliced using Pythons array-slicing syntax. Usually slicing a QuerySet returns another (unevaluated)``QuerySet``, but Django will execute the database query if you use the step parameter of slice syntax.

Converting to a list : You can force evaluation of a QuerySet by calling list() on it, for example:

>>> entry_list = list(Entry.objects.all())

Be warned, though, that this could have a large memory overhead, because Django will load each element of the list into memory. In contrast, iterating over a QuerySet will take advantage of your database to load data and instantiate objects only as you need them.

Filtered QuerySets Are Unique

Each time you refine a QuerySet , you get a brand-new QuerySet that is in no way bound to the previous QuerySet . Each refinement creates a separate and distinct QuerySet that can be stored, used, and reused:

q1 = Entry.objects.filter(headline__startswith="What")
q2 = q1.exclude(pub_date__gte=datetime.now())
q3 = q1.filter(pub_date__gte=datetime.now())

These three QuerySets are separate. The first is a base QuerySet containing all entries that contain a headline starting with What. The second is a subset of the first, with an additional criterion that excludes records whose pub_date is greater than now. The third is a subset of the first, with an additional criterion that selects only the records whose pub_date is greater than now. The initial QuerySet (q1 ) is unaffected by the refinement process.

Limiting QuerySets

Use Pythons array-slicing syntax to limit your QuerySet to a certain number of results. This is the equivalent of SQLs LIMIT and OFFSET clauses.

For example, this returns the first five entries (LIMIT 5 ):

>>> Entry.objects.all()[:5]

This returns the sixth through tenth entries (OFFSET 5 LIMIT 5 ):

>>> Entry.objects.all()[5:10]

Generally, slicing a QuerySet returns a new QuerySet it doesnt evaluate the query. An exception is if you use the step parameter of Python slice syntax. For example, this would actually execute the query in order to return a list of every second object of the first ten:

>>> Entry.objects.all()[:10:2]

To retrieve a single object rather than a list (e.g., SELECT foo FROM bar LIMIT 1 ), use a simple index instead of a slice. For example, this returns the first Entry in the database, after ordering entries alphabetically by headline:

>>> Entry.objects.order_by('headline')[0]

This is roughly equivalent to the following:

>>> Entry.objects.order_by('headline')[0:1].get()

Note, however, that the first of these will raise IndexError while the second will raise DoesNotExist if no objects match the given criteria.

Query Methods That Return New QuerySets

Django provides a range of QuerySet refinement methods that modify either the types of results returned by the QuerySet or the way its SQL query is executed. These methods are described in the sections that follow. Some of the methods take field lookup arguments, which are discussed in detail a bit later on.

filter(**lookup)

Returns a new QuerySet containing objects that match the given lookup parameters.

exclude(**kwargs)

Returns a new QuerySet containing objects that do not match the given lookup parameters.

order_by(*fields)

By default, results returned by a QuerySet are ordered by the ordering tuple given by the ordering option in the models metadata (see Appendix B). You can override this for a particular query using the order_by() method:

>> Entry.objects.filter(pub_date__year=2005).order_by('-pub_date', 'headline')

This result will be ordered by pub_date descending, then by headline ascending. The negative sign in front of "-pub_date" indicates descending order. Ascending order is assumed if the - is absent. To order randomly, use "?" , like so:

>>> Entry.objects.order_by('?')

distinct()

Returns a new QuerySet that uses SELECT DISTINCT in its SQL query. This eliminates duplicate rows from the query results.

By default, a QuerySet will not eliminate duplicate rows. In practice, this is rarely a problem, because simple queries such as Blog.objects.all() dont introduce the possibility of duplicate result rows.

However, if your query spans multiple tables, its possible to get duplicate results when a QuerySet is evaluated. Thats when youd use distinct() .

values(*fields)

Returns a special QuerySet that evaluates to a list of dictionaries instead of model-instance objects. Each of those dictionaries represents an object, with the keys corresponding to the attribute names of model objects:

# This list contains a Blog object.
>>> Blog.objects.filter(name__startswith='Beatles')
[Beatles Blog]

# This list contains a dictionary.
>>> Blog.objects.filter(name__startswith='Beatles').values()
[{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}]

values() takes optional positional arguments, *fields , which specify field names to which the SELECT should be limited. If you specify the fields, each dictionary will contain only the field keys/values for the fields you specify. If you dont specify the fields, each dictionary will contain a key and value for every field in the database table:

>>> Blog.objects.values()
[{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}],
>>> Blog.objects.values('id', 'name')
[{'id': 1, 'name': 'Beatles Blog'}]

This method is useful when you know youre only going to need values from a small number of the available fields and you wont need the functionality of a model instance object. Its more efficient to select only the fields you need to use.

dates(field, kind, order)

Returns a special QuerySet that evaluates to a list of datetime.datetime objects representing all available dates of a particular kind within the contents of the QuerySet .

The field argument must be the name of a DateField or DateTimeField of your model. The kind argument must be either "year" , "month" , or "day" . Each datetime.datetime object in the result list is truncated to the given type :

"year" returns a list of all distinct year values for the field.

"month" returns a list of all distinct year/month values for the field.

"day" returns a list of all distinct year/month/day values for the field.

order , which defaults to 'ASC' , should be either 'ASC' or 'DESC' . This specifies how to order the results.

Here are a few examples:

>>> Entry.objects.dates('pub_date', 'year')
[datetime.datetime(2005, 1, 1)]

>>> Entry.objects.dates('pub_date', 'month')
[datetime.datetime(2005, 2, 1), datetime.datetime(2005, 3, 1)]

>>> Entry.objects.dates('pub_date', 'day')
[datetime.datetime(2005, 2, 20), datetime.datetime(2005, 3, 20)]

>>> Entry.objects.dates('pub_date', 'day', order='DESC')
[datetime.datetime(2005, 3, 20), datetime.datetime(2005, 2, 20)]

>>> Entry.objects.filter(headline__contains='Lennon').dates('pub_date', 'day')
[datetime.datetime(2005, 3, 20)]

select_related()

Returns a QuerySet that will automatically follow foreign key relationships, selecting that additional related-object data when it executes its query. This is a performance booster that results in (sometimes much) larger queries but means later use of foreign key relationships wont require database queries.

The following examples illustrate the difference between plain lookups and select_related() lookups. Heres standard lookup:

# Hits the database.
>>> e = Entry.objects.get(id=5)

# Hits the database again to get the related Blog object.
>>> b = e.blog

And heres select_related lookup:

# Hits the database.
>>> e = Entry.objects.select_related().get(id=5)

# Doesn't hit the database, because e.blog has been prepopulated
# in the previous query.
>>> b = e.blog

select_related() follows foreign keys as far as possible. If you have the following models:

class City(models.Model):
    # ...

class Person(models.Model):
    # ...
    hometown = models.ForeignKey(City)

class Book(models.Model):
    # ...
    author = models.ForeignKey(Person)

then a call to Book.objects.select_related().get(id=4) will cache the related Person and the related City :

>>> b = Book.objects.select_related().get(id=4)
>>> p = b.author         # Doesn't hit the database.
>>> c = p.hometown       # Doesn't hit the database.

>>> b = Book.objects.get(id=4) # No select_related() in this example.
>>> p = b.author         # Hits the database.
>>> c = p.hometown       # Hits the database.

Note that select_related() does not follow foreign keys that have null=True .

Usually, using select_related() can vastly improve performance because your application can avoid many database calls. However, in situations with deeply nested sets of relationships, select_related() can sometimes end up following too many relations and can generate queries so large that they end up being slow.

extra()

Sometimes, the Django query syntax by itself cant easily express a complex WHERE clause. For these edge cases, Django provides the extra() QuerySet modifier a hook for injecting specific clauses into the SQL generated by a QuerySet .

By definition, these extra lookups may not be portable to different database engines (because youre explicitly writing SQL code) and violate the DRY principle, so you should avoid them if possible.

Specify one or more of params , select , where , or tables . None of the arguments is required, but you should use at least one of them.

The select argument lets you put extra fields in the SELECT clause. It should be a dictionary mapping attribute names to SQL clauses to use to calculate that attribute:

>>> Entry.objects.extra(select={'is_recent': "pub_date > '2006-01-01'"})

As a result, each Entry object will have an extra attribute, is_recent , a Boolean representing whether the entrys pub_date is greater than January 1, 2006.

The next example is more advanced; it does a subquery to give each resulting Blog object an entry_count attribute, an integer count of associated Entry objects:

>>> subq = 'SELECT COUNT(*) FROM blog_entry WHERE blog_entry.blog_id = blog_blog.id'
>>> Blog.objects.extra(select={'entry_count': subq})

(In this particular case, were exploiting the fact that the query will already contain the blog_blog table in its FROM clause.)

You can define explicit SQL WHERE clauses perhaps to perform nonexplicit joins by using where . You can manually add tables to the SQL FROM clause by using tables .

where and tables both take a list of strings. All where parameters are ANDed to any other search criteria:

>>> Entry.objects.extra(where=['id IN (3, 4, 5, 20)'])

The select and where parameters described previously may use standard Python database string placeholders: '%s' to indicate parameters the database engine should automatically quote. The params argument is a list of any extra parameters to be substituted:

>>> Entry.objects.extra(where=['headline=%s'], params=['Lennon'])

Always use params instead of embedding values directly into select or where because params will ensure values are quoted correctly according to your particular database.

Heres an example of the wrong way:

Entry.objects.extra(where=["headline='%s'" % name])

Heres an example of the correct way:

Entry.objects.extra(where=['headline=%s'], params=[name])

QuerySet Methods That Do Not Return QuerySets

The following QuerySet methods evaluate the QuerySet and return something otherthan a QuerySet a single object, value, and so forth.

get(**lookup)

Returns the object matching the given lookup parameters, which should be in the format described in the Field Lookups section. This raises AssertionError if more than one object was found.

get() raises a DoesNotExist exception if an object wasnt found for the given parameters. The DoesNotExist exception is an attribute of the model class, for example:

>>> Entry.objects.get(id='foo') # raises Entry.DoesNotExist

The DoesNotExist exception inherits from django.core.exceptions.ObjectDoesNotExist , so you can target multiple DoesNotExist exceptions:

>>> from django.core.exceptions import ObjectDoesNotExist
>>> try:
...     e = Entry.objects.get(id=3)
...     b = Blog.objects.get(id=1)
... except ObjectDoesNotExist:
...     print "Either the entry or blog doesn't exist."

create(**kwargs)

This is a convenience method for creating an object and saving it all in one step. It lets you compress two common steps:

>>> p = Person(first_name="Bruce", last_name="Springsteen")
>>> p.save()

into a single line:

>>> p = Person.objects.create(first_name="Bruce", last_name="Springsteen")

get_or_create(**kwargs)

This is a convenience method for looking up an object and creating one if it doesnt exist. It returns a tuple of (object, created) , where object is the retrieved or created object and created is a Boolean specifying whether a new object was created.

This method is meant as a shortcut to boilerplate code and is mostly useful for data-import scripts, for example:

try:
    obj = Person.objects.get(first_name='John', last_name='Lennon')
except Person.DoesNotExist:
    obj = Person(first_name='John', last_name='Lennon', birthday=date(1940, 10, 9))
    obj.save()

This pattern gets quite unwieldy as the number of fields in a model increases. The previous example can be rewritten using get_or_create() like so:

obj, created = Person.objects.get_or_create(
    first_name = 'John',
    last_name  = 'Lennon',
    defaults   = {'birthday': date(1940, 10, 9)}
)

Any keyword arguments passed to get_or_create() except an optional one called defaults will be used in a get() call. If an object is found, get_or_create() returns a tuple of that object and False . If an object is not found, get_or_create() will instantiate and save a new object, returning a tuple of the new object and True . The new object will be created according to this algorithm:

defaults = kwargs.pop('defaults', {})
params = dict([(k, v) for k, v in kwargs.items() if '__' not in k])
params.update(defaults)
obj = self.model(**params)
obj.save()

In English, that means start with any non-'defaults' keyword argument that doesnt contain a double underscore (which would indicate a nonexact lookup). Then add the contents of defaults , overriding any keys if necessary, and use the result as the keyword arguments to the model class.

If you have a field named defaults and want to use it as an exact lookup in get_or_create() , just use 'defaults__exact' like so:

Foo.objects.get_or_create(
    defaults__exact = 'bar',
    defaults={'defaults': 'baz'}
)

Note

As mentioned earlier, get_or_create() is mostly useful in scripts that need to parse data and create new records if existing ones arent available. But if you need to use get_or_create() in a view, please make sure to use it only in POST requests unless you have a good reason not to. GET requests shouldnt have any effect on data; use POST whenever a request to a page has a side effect on your data.

count()

Returns an integer representing the number of objects in the database matching the QuerySet . count() never raises exceptions. Heres an example:

# Returns the total number of entries in the database.
>>> Entry.objects.count()
4

# Returns the number of entries whose headline contains 'Lennon'
>>> Entry.objects.filter(headline__contains='Lennon').count()
1

count() performs a SELECT COUNT(*) behind the scenes, so you should always use count() rather than loading all of the records into Python objects and calling len() on the result.

Depending on which database youre using (e.g., PostgreSQL or MySQL), count() may return a long integer instead of a normal Python integer. This is an underlying implementation quirk that shouldnt pose any real-world problems.

in_bulk(id_list)

Takes a list of primary key values and returns a dictionary mapping each primary key value to an instance of the object with the given ID, for example:

>>> Blog.objects.in_bulk([1])
{1: Beatles Blog}
>>> Blog.objects.in_bulk([1, 2])
{1: Beatles Blog, 2: Cheddar Talk}
>>> Blog.objects.in_bulk([])
{}

IDs of objects that dont exist are silently dropped from the result dictionary. If you pass in_bulk() an empty list, youll get an empty dictionary.

latest(field_name=None)

Returns the latest object in the table, by date, using the field_name provided as the date field. This example returns the latest Entry in the table, according to the pub_date field:

>>> Entry.objects.latest('pub_date')

If your models Meta specifies get_latest_by , you can leave off the field_name argument to latest() . Django will use the field specified in get_latest_by by default.

Like get() , latest() raises DoesNotExist if an object doesnt exist with the given parameters.

Field Lookups

Field lookups are how you specify the meat of an SQL WHERE clause. Theyre specified as keyword arguments to the QuerySet methods filter() , exclude() , and get() .

Basic lookup keyword arguments take the form field__lookuptype=value (note the double underscore). For example:

>>> Entry.objects.filter(pub_date__lte='2006-01-01')

translates (roughly) into the following SQL:

SELECT * FROM blog_entry WHERE pub_date <= '2006-01-01';

If you pass an invalid keyword argument, a lookup function will raise TypeError .

The supported lookup types follow.

exact

Performs an exact match:

>>> Entry.objects.get(headline__exact="Man bites dog")

This matches any object with the exact headline Man bites dog.

If you dont provide a lookup type that is, if your keyword argument doesnt contain a double underscore the lookup type is assumed to be exact .

For example, the following two statements are equivalent:

>>> Blog.objects.get(id__exact=14) # Explicit form
>>> Blog.objects.get(id=14) # __exact is implied

This is for convenience, because exact lookups are the common case.

iexact

Performs a case-insensitive exact match:

>>> Blog.objects.get(name__iexact='beatles blog')

This will match 'Beatles Blog' , 'beatles blog' , 'BeAtLes BLoG' , and so forth.

contains

Performs a case-sensitive containment test:

Entry.objects.get(headline__contains='Lennon')

This will match the headline 'Today Lennon honored' but not 'today lennon honored' .

SQLite doesnt support case-sensitive LIKE statements; when using SQLite,``contains`` acts like icontains .

Escaping Percent Signs and Underscores in LIKE Statements

The field lookups that equate to LIKE SQL statements (iexact , contains , icontains , startswith , istartswith , endswith , and iendswith ) will automatically escape the two special characters used in LIKE statements the percent sign and the underscore. (In a LIKE statement, the percent sign signifies a multiple-character wildcard and the underscore signifies a single-character wildcard.)

This means things should work intuitively, so the abstraction doesnt leak. For example, to retrieve all the entries that contain a percent sign, just use the percent sign as any other character:

Entry.objects.filter(headline__contains='%')

Django takes care of the quoting for you. The resulting SQL will look something like this:

SELECT ... WHERE headline LIKE '%\%%';

The same goes for underscores. Both percentage signs and underscores are handled for you transparently.

icontains

Performs a case-insensitive containment test:

>>> Entry.objects.get(headline__icontains='Lennon')

Unlike contains , icontains will match 'today lennon honored' .

gt, gte, lt, and lte

These represent greater than, greater than or equal to, less than, and less than or equal to:

>>> Entry.objects.filter(id__gt=4)
>>> Entry.objects.filter(id__lt=15)
>>> Entry.objects.filter(id__gte=0)

These queries return any object with an ID greater than 4, an ID less than 15, and an ID greater than or equal to 1, respectively.

Youll usually use these on numeric fields. Be careful with character fields since character order isnt always what youd expect (i.e., the string 4 sorts after the string 10).

in

Filters where a value is on a given list:

Entry.objects.filter(id__in=[1, 3, 4])

This returns all objects with the ID 1, 3, or 4.

startswith

Performs a case-sensitive starts-with:

>>> Entry.objects.filter(headline__startswith='Will')

This will return the headlines Will he run? and Willbur named judge, but not Who is Will? or will found in crypt.

istartswith

Performs a case-insensitive starts-with:

>>> Entry.objects.filter(headline__istartswith='will')

This will return the headlines Will he run?, Willbur named judge, and will found in crypt, but not Who is Will?

endswith and iendswith

Perform case-sensitive and case-insensitive ends-with:

>>> Entry.objects.filter(headline__endswith='cats')
>>> Entry.objects.filter(headline__iendswith='cats')

range

Performs an inclusive range check:

>>> start_date = datetime.date(2005, 1, 1)
>>> end_date = datetime.date(2005, 3, 31)
>>> Entry.objects.filter(pub_date__range=(start_date, end_date))

You can use range anywhere you can use BETWEEN in SQL for dates, numbers, and even characters.

year, month, and day

For date/datetime fields, perform exact year, month, or day matches:

# Year lookup
>>>Entry.objects.filter(pub_date__year=2005)

# Month lookup -- takes integers
>>> Entry.objects.filter(pub_date__month=12)

# Day lookup
>>> Entry.objects.filter(pub_date__day=3)

# Combination: return all entries on Christmas of any year
>>> Entry.objects.filter(pub_date__month=12, pub_date_day=25)

isnull

Takes either True or False , which correspond to SQL queries of IS NULL and IS NOT NULL , respectively:

>>> Entry.objects.filter(pub_date__isnull=True)

__isnull=True vs. __exact=None

There is an important difference between __isnull=True and __exact=None . __exact=None will always return an empty result set, because SQL requires that no value is equal to NULL . __isnull determines if the field is currently holding the value of NULL without performing a comparison.

search

A Boolean full-text search that takes advantage of full-text indexing. This is like contains but is significantly faster due to full-text indexing.

Note this is available only in MySQL and requires direct manipulation of the database to add the full-text index.

The pk Lookup Shortcut

For convenience, Django provides a pk lookup type, which stands for primary_key.

In the example Blog model, the primary key is the id field, so these three statements are equivalent:

>>> Blog.objects.get(id__exact=14) # Explicit form
>>> Blog.objects.get(id=14) # __exact is implied
>>> Blog.objects.get(pk=14) # pk implies id__exact

The use of pk isnt limited to __exact queries any query term can be combined with pk to perform a query on the primary key of a model:

# Get blogs entries  with id 1, 4, and 7
>>> Blog.objects.filter(pk__in=[1,4,7])

# Get all blog entries with id > 14
>>> Blog.objects.filter(pk__gt=14)

pk lookups also work across joins. For example, these three statements are equivalent:

>>> Entry.objects.filter(blog__id__exact=3) # Explicit form
>>> Entry.objects.filter(blog__id=3) # __exact is implied
>>> Entry.objects.filter(blog__pk=3) # __pk implies __id__exact

Complex Lookups with Q Objects

Keyword argument queries in filter() and so on are ANDed together. If you need to execute more complex queries (e.g., queries with OR statements), you can use Q objects.

A Q object (django.db.models.Q ) is an object used to encapsulate a collection of keyword arguments. These keyword arguments are specified as in the Field Lookups section.

For example, this Q object encapsulates a single LIKE query:

Q(question__startswith='What')

Q objects can be combined using the & and | operators. When an operator is used on two Q objects, it yields a new Q object. For example, this statement yields a single Q object that represents the OR of two "question__startswith" queries:

Q(question__startswith='Who') | Q(question__startswith='What')

This is equivalent to the following SQL WHERE clause:

WHERE question LIKE 'Who%' OR question LIKE 'What%'

You can compose statements of arbitrary complexity by combining Q objects with the & and | operators. You can also use parenthetical grouping.

Each lookup function that takes keyword arguments (e.g., filter() , exclude() , get() ) can also be passed one or more Q objects as positional (not-named) arguments. If you provide multiple Q object arguments to a lookup function, the arguments will be ANDed together, for example:

Poll.objects.get(
    Q(question__startswith='Who'),
    Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6))
)

roughly translates into the following SQL:

SELECT * from polls WHERE question LIKE 'Who%'
    AND (pub_date = '2005-05-02' OR pub_date = '2005-05-06')

Lookup functions can mix the use of Q objects and keyword arguments. All arguments provided to a lookup function (be they keyword arguments or Q objects) are ANDed together. However, if a Q object is provided, it must precede the definition of any keyword arguments. For example, the following:

Poll.objects.get(
    Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)),
    question__startswith='Who')

would be a valid query, equivalent to the previous example, but this:

# INVALID QUERY
Poll.objects.get(
    question__startswith='Who',
    Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)))

would not be valid.

You can find some examples online at http://www.djangoproject.com/documentation/0.96/models/or_lookups/.

Related Objects

When you define a relationship in a model (i.e., a ForeignKey , OneToOneField , or ManyToManyField ), instances of that model will have a convenient API to access the related object(s).

For example, an Entry object e can get its associated Blog object by accessing the blog attribute e.blog .

Django also creates API accessors for the other side of the relationship the link from the related model to the model that defines the relationship. For example, a Blog object b has access to a list of all related Entry objects via the entry_set attribute: b.entry_set.all() .

All examples in this section use the sample Blog , Author , and Entry models defined at the top of this page.

Lookups That Span Relationships

Django offers a powerful and intuitive way to follow relationships in lookups, taking care of the SQL JOIN s for you automatically behind the scenes. To span a relationship, just use the field name of related fields across models, separated by double underscores, until you get to the field you want.

This example retrieves all Entry objects with a Blog whose name is 'Beatles Blog' :

>>> Entry.objects.filter(blog__name__exact='Beatles Blog')

This spanning can be as deep as youd like.

It works backward, too. To refer to a reverse relationship, just use the lowercase name of the model.

This example retrieves all Blog objects that have at least one Entry whose headline contains 'Lennon' :

>>> Blog.objects.filter(entry__headline__contains='Lennon')

Foreign Key Relationships

If a model has a ForeignKey , instances of that model will have access to the related (foreign) object via a simple attribute of the model, for example:

e = Entry.objects.get(id=2)
e.blog # Returns the related Blog object.

You can get and set via a foreign key attribute. As you may expect, changes to the foreign key arent saved to the database until you call save() , for example:

e = Entry.objects.get(id=2)
e.blog = some_blog
e.save()

If a ForeignKey field has null=True set (i.e., it allows NULL values), you can assign None to it:

e = Entry.objects.get(id=2)
e.blog = None
e.save() # "UPDATE blog_entry SET blog_id = NULL ...;"

Forward access to one-to-many relationships is cached the first time the related object is accessed. Subsequent accesses to the foreign key on the same object instance are cached, for example:

e = Entry.objects.get(id=2)
print e.blog  # Hits the database to retrieve the associated Blog.
print e.blog  # Doesn't hit the database; uses cached version.

Note that the select_related() QuerySet method recursively prepopulates the cache of all one-to-many relationships ahead of time:

e = Entry.objects.select_related().get(id=2)
print e.blog  # Doesn't hit the database; uses cached version.
print e.blog  # Doesn't hit the database; uses cached version.

select_related() is documented in the QuerySet Methods That Return New QuerySets section.

Reverse Foreign Key Relationships

Foreign key relationships are automatically symmetrical a reverse relationship is inferred from the presence of a ForeignKey pointing to another model.

If a model has a ForeignKey , instances of the foreign key model will have access to a Manager that returns all instances of the first model. By default, this Manager is named FOO_set , where FOO is the source model name, lowercased. This Manager returns QuerySets , which can be filtered and manipulated as described in the Retrieving Objects section.

Heres an example:

b = Blog.objects.get(id=1)
b.entry_set.all() # Returns all Entry objects related to Blog.

# b.entry_set is a Manager that returns QuerySets.
b.entry_set.filter(headline__contains='Lennon')
b.entry_set.count()

You can override the FOO_set name by setting the related_name parameter in the ForeignKey() definition. For example, if the Entry model was altered to blog = ForeignKey(Blog, related_name='entries') , the preceding example code would look like this:

b = Blog.objects.get(id=1)
b.entries.all() # Returns all Entry objects related to Blog.

# b.entries is a Manager that returns QuerySets.
b.entries.filter(headline__contains='Lennon')
b.entries.count()

You cannot access a reverse ForeignKey Manager from the class; it must be accessed from an instance:

Blog.entry_set # Raises AttributeError: "Manager must be accessed via instance".

In addition to the QuerySet methods defined in the Retrieving Objects section, the ForeignKey Manager has these additional methods:

add(obj1, obj2, ...) : Adds the specified model objects to the related object set, for example:

b = Blog.objects.get(id=1)
e = Entry.objects.get(id=234)
b.entry_set.add(e) # Associates Entry e with Blog b.

create(**kwargs) : Creates a new object, saves it, and puts it in the related object set. It returns the newly created object:

b = Blog.objects.get(id=1)
e = b.entry_set.create(headline='Hello', body_text='Hi', pub_date=datetime.date(2005, 1, 1))
# No need to call e.save() at this point -- it's already been saved.

This is equivalent to (but much simpler than) the following:

b = Blog.objects.get(id=1)
e = Entry(blog=b, headline='Hello', body_text='Hi', pub_date=datetime.date(2005, 1, 1))
e.save()

Note that theres no need to specify the keyword argument of the model that defines the relationship. In the preceding example, we dont pass the parameter blog to create() . Django figures out that the new Entry objects blog field should be set to b .

remove(obj1, obj2, ...) : Removes the specified model objects from the related object set:

b = Blog.objects.get(id=1)
e = Entry.objects.get(id=234)
b.entry_set.remove(e) # Disassociates Entry e from Blog b.

In order to prevent database inconsistency, this method only exists on ForeignKey objects where null=True . If the related field cant be set to None (NULL ), then an object cant be removed from a relation without being added to another. In the preceding example, removing e from b.entry_set() is equivalent to doing e.blog = None , and because the blog ForeignKey doesnt have null=True , this is invalid.

clear() : Removes all objects from the related object set:

b = Blog.objects.get(id=1)
b.entry_set.clear()

Note this doesnt delete the related objects it just disassociates them.

Just like remove() , clear() is only available on ForeignKey``s where ``null=True .

To assign the members of a related set in one fell swoop, just assign to it from any iterable object, for example:

b = Blog.objects.get(id=1)
b.entry_set = [e1, e2]

If the clear() method is available, any pre-existing objects will be removed from the entry_set before all objects in the iterable (in this case, a list) are added to the set. If the clear() method is not available, all objects in the iterable will be added without removing any existing elements.

Each reverse operation described in this section has an immediate effect on the database. Every addition, creation, and deletion is immediately and automatically saved to the database.

Many-to-Many Relationships

Both ends of a many-to-many relationship get automatic API access to the other end. The API works just as a reverse one-to-many relationship (described in the previous section).

The only difference is in the attribute naming: the model that defines the ManyToManyField uses the attribute name of that field itself, whereas the reverse model uses the lowercased model name of the original model, plus '_set' (just like reverse one-to-many relationships).

An example makes this concept easier to understand:

e = Entry.objects.get(id=3)
e.authors.all() # Returns all Author objects for this Entry.
e.authors.count()
e.authors.filter(name__contains='John')

a = Author.objects.get(id=5)
a.entry_set.all() # Returns all Entry objects for this Author.

Like ForeignKey , ManyToManyField can specify related_name . In the preceding example, if the ManyToManyField in Entry had specified related_name='entries' , then each Author instance would have an entries attribute instead of entry_set .

How Are the Backward Relationships Possible?

Other object-relational mappers require you to define relationships on both sides. The Django developers believe this is a violation of the DRY (Dont Repeat Yourself) principle, so Django requires you to define the relationship on only one end. But how is this possible, given that a model class doesnt know which other model classes are related to it until those other model classes are loaded?

The answer lies in the INSTALLED_APPS setting. The first time any model is loaded, Django iterates over every model in INSTALLED_APPS and creates the backward relationships in memory as needed. Essentially, one of the functions of INSTALLED_APPS is to tell Django the entire model domain.

Queries Over Related Objects

Queries involving related objects follow the same rules as queries involving normal value fields. When specifying the value for a query to match, you may use either an object instance itself or the primary key value for the object.

For example, if you have a Blog object b with id=5 , the following three queries would be identical:

Entry.objects.filter(blog=b) # Query using object instance
Entry.objects.filter(blog=b.id) # Query using id from instance
Entry.objects.filter(blog=5) # Query using id directly

Deleting Objects

The delete method, conveniently, is named delete() . This method immediately deletes the object and has no return value:

e.delete()

You can also delete objects in bulk. Every QuerySet has a delete() method, which deletes all members of that QuerySet . For example, this deletes all Entry objects with a pub_date year of 2005:

Entry.objects.filter(pub_date__year=2005).delete()

When Django deletes an object, it emulates the behavior of the SQL constraint ON DELETE CASCADE in other words, any objects that had foreign keys pointing at the object to be deleted will be deleted along with it, for example:

b = Blog.objects.get(pk=1)
# This will delete the Blog and all of its Entry objects.
b.delete()

Note that delete() is the only QuerySet method that is not exposed on a Manager itself. This is a safety mechanism to prevent you from accidentally requesting Entry.objects.delete() and deleting all the entries. If you do want to delete all the objects, then you have to explicitly request a complete query set:

Entry.objects.all().delete()

Extra Instance Methods

In addition to save() and delete() , a model object might get any or all of the following methods.

get_FOO_display()

For every field that has choices set, the object will have a get_FOO_display() method, where FOO is the name of the field. This method returns the human-readable value of the field. For example, in the following model:

GENDER_CHOICES = (
    ('M', 'Male'),
    ('F', 'Female'),
)
class Person(models.Model):
    name = models.CharField(max_length=20)
    gender = models.CharField(max_length=1, choices=GENDER_CHOICES)

each Person instance will have a get_gender_display() method:

>>> p = Person(name='John', gender='M')
>>> p.save()
>>> p.gender
'M'
>>> p.get_gender_display()
'Male'

get_next_by_FOO(kwargs) and get_previous_by_FOO(kwargs)

For every DateField and DateTimeField that does not have null=True , the object will have get_next_by_FOO() and get_previous_by_FOO() methods, where FOO is the name of the field. This returns the next and previous object with respect to the date field, raising the appropriate DoesNotExist exception when appropriate.

Both methods accept optional keyword arguments, which should be in the format described in the Field Lookups section.

Note that in the case of identical date values, these methods will use the ID as a fallback check. This guarantees that no records are skipped or duplicated. For a full example, see the lookup API samples at http://www.djangoproject.com/documentation/0.96/models/lookup/.

get_FOO_filename()

For every FileField , the object will have a get_FOO_filename() method, where FOO is the name of the field. This returns the full filesystem path to the file, according to your MEDIA_ROOT setting.

Note that ImageField is technically a subclass of FileField , so every model with an ImageField will also get this method.

get_FOO_url()

For every FileField , the object will have a get_FOO_url() method, where FOO is the name of the field. This returns the full URL to the file, according to your MEDIA_URL setting. If the value is blank, this method returns an empty string.

get_FOO_size()

For every FileField , the object will have a get_FOO_size() method, where FOO is the name of the field. This returns the size of the file, in bytes. (Behind the scenes, it uses os.path.getsize .)

save_FOO_file(filename, raw_contents)

For every FileField , the object will have a save_FOO_file() method, where FOO is the name of the field. This saves the given file to the filesystem, using the given file name. If a file with the given file name already exists, Django adds an underscore to the end of the file name (but before the extension) until the file name is available.

get_FOO_height() and get_FOO_width()

For every ImageField , the object will have get_FOO_height() and get_FOO_width() methods, where FOO is the name of the field. This returns the height (or width) of the image, as an integer, in pixels.

Shortcuts

As you develop views, you will discover a number of common idioms in the way you use the database API. Django encodes some of these idioms as shortcuts that can be used to simplify the process of writing views. These functions are in the django.shortcuts module.

get_object_or_404()

One common idiom to use get() and raise Http404 if the object doesnt exist. This idiom is captured by get_object_or_404() . This function takes a Django model as its first argument and an arbitrary number of keyword arguments, which it passes to the default managers get() function. It raises Http404 if the object doesnt exist, for example:

# Get the Entry with a primary key of 3
e = get_object_or_404(Entry, pk=3)

When you provide a model to this shortcut function, the default manager is used to execute the underlying get() query. If you dont want to use the default manager, or if you want to search a list of related objects, you can provide get_object_or_404() with a Manager object instead:

# Get the author of blog instance e with a name of 'Fred'
a = get_object_or_404(e.authors, name='Fred')

# Use a custom manager 'recent_entries' in the search for an
# entry with a primary key of 3
e = get_object_or_404(Entry.recent_entries, pk=3)

get_list_or_404()

get_list_or_404 behaves the same way as get_object_or_404() , except that it uses filter() instead of get() . It raises Http404 if the list is empty.

Falling Back to Raw SQL

If you find yourself needing to write an SQL query that is too complex for Djangos database mapper to handle, you can fall back into raw SQL statement mode.

The preferred way to do this is by giving your model custom methods or custom manager methods that execute queries. Although theres nothing in Django that requires database queries to live in the model layer, this approach keeps all your data access logic in one place, which is smart from a code organization standpoint. For instructions, see Appendix B..

Finally, its important to note that the Django database layer is merely an interface to your database. You can access your database via other tools, programming languages, or database frameworks theres nothing Django-specific about your database.

The Django Book