Here some tips about Django REST Framework (DRF) with Django 3.

Official website

Quick install

> pip install djangorestframework

Then in settings.py:

INSTALLED_APPS = [
    ...
    'rest_framework',
]

Let's go!

 

API from a remote database

Let's go create a Django REST API from a remote DB with Django 3.

It can be interesting because I should be able to provide an API from a tool by accessing only to his DB. This whatever the system used (Python, PHP, Java ...) and whatever my own knowledge of the system used.

In my development folder:

> django-admin startproject my_apis

Here my project will manage APIs, that is why I call it my_apis, it will only be used for that. But you can create an API in an existing Django project.

 

So I would do an API targeting a remote tool, with his own DB. First let's go create our Django app in my_apis folder, named app_remote1, then migrations and a super-user:

> python manage.py startapp app_remote1
> python manage.py migrate
> python manage.py createsuperuser
...
> python manage.py runserver

 

app_remote1 refers to the remote tool. This app will contain all information about our API (models and other Django files, the API itself ...). Indeed to build an Django REST API, we need a basic Django model.

Now if you start the server, you access to http://127.0.0.1:8000/ and http://127.0.0.1:8000/admin.

Remote connection

OK, let's go adding our remote DB into our Django instance. In settings.py: 

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    },
 
    'db_remote1': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'db',
        'USER': 'user-db',
        'PASSWORD': 'password',
        'HOST': 'host',
        'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"}
    },
}

 

OK. But we need a router to guide our queries when we will use the remote DB in our models. Thanks to books.agiliq where I took the code below. Beside your settings.py, a new router_remote1.py file:

class MyRouter1:
 
    def db_for_read(self, model, **hints):
        if model._meta.app_label == 'app_remote1':
            return 'db_remote1'
        return None
 
    def db_for_write(self, model, **hints):
        if model._meta.app_label == 'app_remote1':
            return 'db_remote1'
        return None
 
    def allow_relation(self, obj1, obj2, **hints):
        if obj1._meta.app_label == 'app_remote1' or \
           obj2._meta.app_label == 'app_remote1':
           return True
        return None
 
    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if app_label == 'app_remote1':
            return db == 'db_remote1'
        return None

You understand: now each time we will need this remote database in our code, we will be able to point on it mentioning his app_label.

 

Integrate your router adding this in settings.py:

DATABASE_ROUTERS = [
    'my_apis.router_remote1.MyRouter1',
]

 

Only now we can use our remote DB in our model, app_remote1\models.py:

from django.db import models
 
class ModelFromRemoteTable1(models.Model):
    id = models.AutoField(primary_key=True)
    name = models.CharField(max_length=200)
    email = models.EmailField()
    country = models.CharField(max_length=200)
 
    class Meta:
        app_label = 'app_remote1'
        db_table = 'your_table'
        managed = False
 
    def __str__(self):
         return str(self.name)

Of course the used table and his fields must exist. Look the meta managed = False, very important to not migrate your table with Django ...

Also very important the app_label to redirect to the well database, the remote.

 

Now display our remote data in Django with app_remote1\admin.py:

from django.contrib import admin
from .models import ModelFromRemoteTable1
 
class RemoteTable1Admin(admin.ModelAdmin):
 
    list_display = ('id', 'name', 'email', 'country')
    list_display_links = None
    search_fields = ['name', 'email', 'country']
 
    actions = None
    enable_change_view = False
 
    def has_add_permission(self, request):
        return False
 
    def has_change_permission(self, request):
        return False
 
    def has_delete_permission(self, request, obj=None):
        return False
 
admin.site.register(ModelFromRemoteTable1, RemoteTable1Admin)

Look, we do careful to limit authorizations. Indeed we do not want to allow edit from our Django admin, we just build an API, we need the model only. The admin will allow to read our data, it is enough.

 

Integrate your app adding this in settings.py:

INSTALLED_APPS = [
...
 
    'app_remote1',
    'rest_framework',
]

Now if you start your server, you access to the remote data.

API

OK, let's go create a serializer, in a new file app_remote1\serializers.py:

from rest_framework import serializers
from rest_framework.reverse import reverse
 
from .models import ModelFromRemoteTable1
 
class AppRemote1Serializer(serializers.HyperlinkedModelSerializer):
 
    class Meta:
        model = ModelFromRemoteTable1
        fields = ['url', 'name', 'email', 'country']

 

Now we can create our API, as a view, in app_remote1\views.py:

from django.shortcuts import render
from .models importModelFromRemoteTable1
 
from rest_framework import viewsets
from rest_framework import permissions
 
from app_remote1.serializers import AppRemote1Serializer
 
 
class AppRemote1ViewSet(viewsets.ModelViewSet):
    queryset =ModelFromRemoteTable1.objects.all().order_by('id')
    serializer_class =AppRemote1Serializer
    permission_classes = [permissions.IsAdminUser]

 

Let's go displaying our API in app_remote1\urls.py:

from django.contrib import admin
from django.urls import include, path
from rest_framework import routers
from app_remote1 import views
 
router = routers.DefaultRouter()
router.register(r'app_remote1', views.AppRemote1ViewSet)
 
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]

Look the router: we need it to display our API, but it has nothing to do with our first router above. Indeed our first router just handles the remote connection, this second router handles our API.

 

Hop! Do not forget to add a pagination limit in your settings.py. Without your json will slow down, maybe crashing the page:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination',
    'PAGE_SIZE': 100
}

And go to http://127.0.0.1:8000/, you get your remote API.

Get URL endpoints from another field than the id

Please read this tip until the end because I describe all errors I committed before to find a good solution. Or just go to see Final codes below.

OK, I wanted update data from a tool (tool A), after users updates from another tool (tool B). The perfect field of action for Django REST framework I guess. So I built my own API with Django 3.

But I faced a problem: the relationship between the data had to be on the email field. And by default Django REST framework uses ids to create URL endpoints of each record. Indeed it seems to be this kind of URLs which allow updates by record (PUT command with httpie for example).

These are the ones I need to include in the code that will run in the forms of tool B!

Example of httpie command to update fields through a REST API

> http -a UserLogin:PassWord PUT http://127.0.0.1:8000/users/4/ firstname="Gretha" lastname="Thunberg"

The URL contained above is the endpoint of a record, using its id (4) from tool A. But tool B does not know this id, and does not have to know it. He just want to update firstname and lastname according an email relationship. Kwarghhh!

First I tried to generate URL endpoints with emails, hoping something like:

... http://127.0.0.1:8000/users/greta-thunberg@earth.com/ ...

I know, it's weird. No matter, I never succeeded.

So I had to find a way to get the good URL endpoints without knowing their ids, comparing emails.

Email field and URL endpoint for each record being provided in the json from my API, I use the code below:

1. Json authentication, then reading
2. Json querying to recover URL endpoint according email value
3. Build of the PUT command, then execution

 

import requests, json, subprocess
 
REQUEST_URL = 'http://127.0.0.1:8000/users/?format=json'
login = 'DjangoLogin'
password = 'DjangoPassWord'
response = requests.get(REQUEST_URL, auth=(login, password))
 
json_data = response.text.encode('utf-8', 'ignore')
readable_json = json.loads(json_data)
 
email_reference = YOUR_EMAIL_FIELD_TOOL_B
new_firstname = YOUR_FIRSTNAME_FIELD_TOOL_B
new_lastname = YOUR_LASTNAME_FIELD_TOOL_B
 
match_count = 0
 
for results in readable_json['results']:
    match_count += 1
    if results['email'] == email_reference and results['email'] is not None and match_count != 1:
        my_url = results['url']
 
        my_cmd = 'http -a ' + login + ':' + password + ' PUT ' + my_url + ' firstname="' + new_firstname + '"' + ' lastname="' + new_lastname + '"'
 
        p = subprocess.Popen(my_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out, err = p.communicate()

I check that email is not empty (is not None), and I add an iteration to retrieve only the first result of the data from tool A (match_count). It will also be better for your tests.

But with huge data ...

To work our json must display all your data. Indeed for now we search the email value into all the dataset. What if you store a lot of data?

Add a pagination, in setting.py, for example only 1000 records by page:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination',
    'PAGE_SIZE': 1000
}

 

OK, if you refresh and test, pagination from our json appears in URLs with the final &offset=NUMBER, where NUMBER is the record from you want to start to displaying.

So now you have to iterate through your json, until 500 000 for example:

import requests, json, subprocess
 
login = 'DjangoLogin'
password = 'DjangoPassWord'
 
email_reference = YOUR_EMAIL_FIELD_TOOL_B
new_firstname = YOUR_FIRSTNAME_FIELD_TOOL_B
new_lastname = YOUR_LASTNAME_FIELD_TOOL_B
 
REQUEST_URL_PART = 'http://127.0.0.1:8000/users/?format=json&offset='
 
match_count_A = 0
while match_count_A < 500000:
    REQUEST_URL = REQUEST_URL_PART + str(match_count_A)
    response = requests.get(REQUEST_URL, auth=(login, password))
    json_data = response.text.encode('utf-8', 'ignore')
    readable_json = json.loads(json_data)
    match_count_A += 1000
 
    match_count_B = 0
    for results in readable_json['results']:
       match_count_B += 1
        if results['email'] == email_reference and results['email'] is not None and match_count_B != 1:
            my_url = results['url']
 
            my_cmd = 'http -a ' + login + ':' + password + ' PUT ' + my_url + ' firstname="' + new_firstname + '"' + ' lastname="' + new_lastname + '"'
 
            p = subprocess.Popen(my_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
            out, err = p.communicate()

I know, it's still weird ...

Humm ... Even with this, I tried with a huge database ... Not really the good solution. Run through the json can be very long.

 

Final codes

No, I better to filter my json to find the well record before any while or update. It is possible creating another view in our API to access records filtering fields in URLs, email field for us.

A def get_queryset in yourapp/views.py :

from django.shortcuts import render
from .models import YourModel
 
from rest_framework import viewsets
from rest_framework import permissions
 
from YourApp.serializers import YourSerializer
 
class YourViewsetForEmail(viewsets.ModelViewSet):
    queryset = YourModel.objects.all()
    serializer_class = YourSerializer
    permission_classes = [permissions.IsAdminUser]
 
    def get_queryset(self):
        email = self.request.query_params.get('email', None)
        return YourModel.objects.filter(email=email)

 

In your project/urls.py:

from django.contrib import admin
from django.urls import include, path
from rest_framework import routers
from YourApp import views
 
router = routers.DefaultRouter()
router.register(r'users', views.YourFirstAPIViewset)
router.register(r'users_by_email', views.YourViewsetForEmail)
 
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]

I note this will create an aesthetic problem in the URLs mentioned in the json, but it is only aesthetic (true URLS still work), I will fix it later.

OK, now this URL displays the well record: http://127.0.0.1:8000/users_by_email/?email=Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.

In json: http://127.0.0.1:8000/users_by_email/?format=json&email=Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.

And the record endpoint (mandatory to update) still is: http://127.0.0.1:8000/users/ID/

So an example of a final code to update records through our API according their email will be:

import requests, json, subprocess
 
login = 'DjangoLogin'
password = 'DjangoPassWord'
 
email_reference = YOUR_EMAIL_FIELD_TOOL_B
new_firstname = YOUR_FIRSTNAME_FIELD_TOOL_B
new_lastname = YOUR_LASTNAME_FIELD_TOOL_B
 
REQUEST_URL_PART = 'http://127.0.0.1:8000/users_by_email/?format=json&email='
REQUEST_URL = REQUEST_URL_PART + email_reference
 
response = requests.get(REQUEST_URL, auth=(login, password))
json_data = response.text.encode('utf-8', 'ignore')
readable_json = json.loads(json_data)
 
for results in readable_json['results']:
    if results['email'] == email_reference and results['email'] is not None:
 
        my_id = results['id']
        my_url = 'http://127.0.0.1:8000/users/' + str(my_id) + '/'
 
        my_cmd = 'http -a ' + login + ':' + password + ' PUT ' + my_url + ' firstname="'+ new_firstname +'" lastname=" ' + new_lastname + '"'
 
        p = subprocess.Popen(my_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out, err = p.communicate()