Django REST Framework: Setting up prefetching for nested serializers
<p>My Django-powered app with a DRF API is working fine, but I've started to run into performance issues as the database gets populated with actual data. I've done some profiling with Django Debug Toolbar and found that many of my endpoints issue tens to hundreds of queries in the course of returning their data.</p>
<p>I expected this, since I hadn't previously optimized anything with regard to database queries. Now that I'm setting up prefetching, however, I'm having trouble making use of properly prefetched serializer data when that serializer is nested in a different serializer. I've been using this <a href="http://ses4j.github.io/2015/11/23/optimizing-slow-django-rest-framework-performance/" rel="noreferrer">awesome post</a> as a guide for how to think about the different ways to prefetch.</p>
<p>Currently, my <code>ReadingGroup</code> serializer does prefetch properly when I hit the <code>/api/readinggroups/</code> endpoint. My issue is the <code>/api/userbookstats/</code> endpoint, which returns all <code>UserBookStats</code> objects. The related serializer, <code>UserBookStatsSerializer</code>, has a nested <code>ReadingGroupSerializer</code>.</p>
<p>The models, serializers, and viewsets are as follows:</p>
<p><strong>models.py</strong></p>
<pre><code>class ReadingGroup(models.model):
    owner = models.ForeignKeyField(settings.AUTH_USER_MODEL)
    users = models.ManyToManyField(settings.AUTH_USER_MODEL)
    book_type = models.ForeignKeyField(BookType)
    ....
    &lt;other group related fields&gt;

   def __str__(self):
     return '%s group: %s' % (self.name, self.book_type)

class UserBookStats(models.Model):
    reading_group = models.ForeignKey(ReadingGroup)
    user = models.ForeignKey(settings.AUTH_USER_MODEL)
    alias = models.CharField()

    total_books_read = models.IntegerField(default=0)
    num_books_owned = models.IntegerField(default=0)
    fastest_read_time = models.IntegerField(default=0)
    average_read_time = models.IntegerField(default=0)
</code></pre>
<p><strong>serializers.py</strong></p>
<pre><code>class ReadingGroupSerializer(serializers.ModelSerializer):
    users = UserSerializer(many = True,read_only=True)
    owner = UserSerializer(read_only=True)

    class Meta:
      model = ReadingGroup
      fields = ('url', 'id','owner', 'users')

    @staticmethod
    def setup_eager_loading(queryset):
      #select_related for 'to-one' relationships
      queryset = queryset.select_related('owner')

      #prefetch_related for 'to-many' relationships
      queryset = queryset.prefetch_related('users')

      return queryset

class UserBookStatsSerializer(serializers.HyperlinkedModelSerializer):
    reading_group = ReadingGroupSerializer()
    user = UserSerializer()
    awards = AwardSerializer(source='award_set', many=True)

    class Meta:
      model = UserBookStats
      fields = ('url', 'id', 'alias', 'total_books_read', 'num_books_owned', 
              'average_read_time', 'fastest_read_time', 'awards')

    @staticmethod
    def setup_eager_loading(queryset):
      #select_related for 'to-one' relationships
      queryset = queryset.select_related('user')

      #prefetch_related for 'to-many' relationships
      queryset = queryset.prefetch_related('awards_set')

      #setup prefetching for nested serializers
      groups = Prefetch('reading_group', queryset ReadingGroup.objects.prefetch_related('userbookstats_set'))        
      queryset = queryset.prefetch_related(groups)

      return queryset
</code></pre>
<p><strong>views.py</strong></p>
<pre><code>class ReadingGroupViewset(views.ModelViewset):

  def get_queryset(self):
    qs = ReadingGroup.objects.all()
    qs = self.get_serializer_class().setup_eager_loading(qs)
    return qs

class UserBookStatsViewset(views.ModelViewset):

  def get_queryset(self):
    qs = UserBookStats.objects.all()
    qs = self.get_serializer_class().setup_eager_loading(qs)
    return qs
</code></pre>
<p>I've optimized the prefetching for the <code>ReadingGroup</code> endpoint (I actually posted about eliminating duplicate queries for that endpoint <a href="https://stackoverflow.com/questions/39628948/django-rest-framework-duplicate-db-queries-due-to-model-str-method">here</a>), and now I'm working on the <code>UserBookStats</code> endpoint.</p>
<p>The issue I'm having is that, with my current <code>setup_eager_loading</code> in the <code>UserBookStatsSerializer</code>, it doesn't appear to use the prefetching set up by the eager loading method in the <code>ReadingGroupSerializer</code>. I'm still a little hazy on the syntax for the <code>Prefetch</code> object - I was inspired by <a href="https://stackoverflow.com/questions/35021461/django-rest-serializer-doing-n1-database-calls-for-multiple-nested-relationship">this</a> excellent answer to try that approach.</p>
<p>Obviously the <code>get_queryset</code> method of <code>UserBookStatsViewset</code> doesn't call <code>setup_eager_loading</code> for the <code>ReadingGroup</code> objects, but I'm sure there's a way to accomplish the same prefetching.</p>