Why Async Django Is The Future of Python Web Development

Introduction

Async Django is an extension of the popular Django web framework that adds support for writing asynchronous code using Python’s async/await syntax. Async/await is a feature added in Python 3.5 that allows for more efficient use of system resources by allowing code to run asynchronously.

By using Async Django, developers can write more scalable and efficient web applications that can handle a large number of requests with less overhead. It allows for non-blocking I/O operations and can handle multiple requests at the same time, which can greatly improve the performance of the application.

Async Django includes a number of built-in features, including support for asynchronous views, middleware, and database operations. It also provides a number of utilities for working with asynchronous code, including the ability to run tasks in the background using the Celery task queue.

Usage

Here’s an example usage of Async Django:

Let’s say you have a Django web application that needs to fetch data from an external API and then display that data to the user. In a synchronous Django application, this process would involve blocking the request until the API response is received, potentially causing other requests to wait and slowing down the overall performance of the application.

With Async Django, you can make this process asynchronous and non-blocking. Here’s how it could work:

Define an asynchronous view that will handle the API request and response:

from django.http import JsonResponse
import httpx

async def my_async_view(request):
    async with httpx.AsyncClient() as client:
        response = await client.get('https://api.example.com/data')
        data = response.json()
    return JsonResponse(data)

In your Django URL configuration, specify the URL pattern that will use this asynchronous view:

from django.urls import path
from . import views

urlpatterns = [
    path('my-data/', views.my_async_view),
]

Now, when a user makes a request to the /my-data/ URL, the my_async_view function will send an asynchronous request to the external API, freeing up the main thread to handle other requests in the meantime. Once the API response is received, the function will return a JSON response to the user.

This is just one example of how Async Django can be used to improve the performance and scalability of a web application. By using asynchronous code, you can handle more concurrent requests and make more efficient use of system resources, leading to faster response times and a better user experience.

Use cases

Here are some main use cases for Async Django:

Handling high-volume, I/O-bound requests: async code can help improve the performance of web applications that need to handle a large volume of requests that involve I/O-bound operations, such as fetching data from external APIs or reading from a database.
Real-time web applications: Async Django can be used to build real-time web applications that require server-side WebSocket connections, such as chat apps, multiplayer games, or collaborative document editors.
Background task processing: Async Django can be used in conjunction with Celery to process background tasks asynchronously, such as sending emails, generating reports, or performing data analysis.
Asynchronous APIs: Async Django can be used to build APIs that handle a large number of concurrent requests, such as RESTful APIs or GraphQL APIs.
Scraping and web crawling: Async Django can be used to build web scrapers and crawlers that need to fetch and process a large volume of data from the web.
Machine learning and data science: Async Django can be used in conjunction with libraries like TensorFlow or PyTorch to build machine learning models and handle large-scale data processing tasks asynchronously.

Does Django ORM support async?

With Django's current state (as of Django 5.0), you can't directly run all ORM queries asynchronously. Here's a breakdown of what's possible and what's not:

Queries You Can (Mostly) Run Asynchronously:

Simple Queries: Retrieving data using basic .filter(), .get(), or .all() methods within an asynchronous view is generally okay. However, keep in mind:
- If the database queries are slow, they can still block the event loop and negate the benefits of async.
- Complex queries with heavy filtering or aggregations might introduce blocking behavior.

Queries You Can't Run Asynchronously (Yet):

Transactions: As of Django 5.0, transactions are not yet supported in asynchronous mode. This means you can't use asynchronous versions of save() or other methods that involve database transactions.
Certain ORM Methods: Some specific methods within the Django ORM might not have asynchronous equivalents, requiring you to stick to their synchronous counterparts.

Alternative Approaches for Full Asynchronicity:

Third-Party Libraries: Libraries like django-async-orm aim to provide asynchronous equivalents for ORM methods. However, these libraries might still have limitations or require patching models.
Alternative Async Database Tools: For a fully asynchronous approach, consider aiomysql or asyncpg for direct, low-level interaction with your database engine. This gives you full async control but requires handling the mapping between low-level interactions and Django models.

Django's lack of full transaction support in asynchronous views (as of Django 5.0) stems from the inherent complexities of managing transactions in a concurrent environment. Here's a breakdown of the challenges:

Synchronous Nature of Transactions:

Traditional database transactions involve a series of operations treated as a single unit. If any part fails, the entire transaction is rolled back.
Django's ORM currently operates synchronously, meaning it assumes a single thread of execution that can atomically execute the transaction steps.

Challenges in Asynchronous World:

In an asynchronous environment, multiple requests might be handled concurrently, potentially accessing and modifying the same data.
Ensuring atomicity (all or nothing execution) and isolation (data consistency across concurrent requests) becomes more intricate.

Potential Issues Without Async Transactions:

Data Inconsistency: If two async requests attempt to modify the same data concurrently without proper isolation, it could lead to inconsistencies in the database.
Partial Commits: Without proper transaction management, some parts of a data update might be committed, while others fail, leaving the database in an unexpected state.

Possible Solutions (Under Development):

Database-Level Locking: Some databases offer row-level or table-level locking mechanisms that could be leveraged to ensure data consistency during concurrent async operations. However, this can introduce performance overhead.
Optimistic Locking: This strategy involves checking for data modifications before saving changes. If a conflict is detected, the update is rejected, and the user might need to retry. It can help prevent lost updates but requires additional logic in your application.

Future of Async Transactions in Django:

The Django developers are actively exploring ways to introduce async transaction support in the ORM. This might involve integrating with database-level locking mechanisms or implementing optimistic locking strategies within the framework.
Stay tuned for updates in future Django releases to see if and how async transactions are implemented.

Why is ASGI needed for async support in Django?

ASGI (Asynchronous Server Gateway Interface) is a Python specification that defines how web servers communicate with web applications written in Python. It's the successor to WSGI (Web Server Gateway Interface), which is used for synchronous applications.

Here's a breakdown of ASGI, its workings, and its importance for asynchronous Django:

What is ASGI?

ASGI builds upon WSGI, but it's designed to handle asynchronous tasks. This means it can efficiently manage applications that can perform multiple operations concurrently without blocking.

ASGI applications are typically callables (functions) that receive messages from the web server and send responses back.

How ASGI Works:

Web Server Initiation: The web server (e.g., Daphne, Uvicorn) starts the ASGI application.
Receiving Requests: The ASGI application receives an initial message from the server containing information about the incoming HTTP request.
Asynchronous Handling: The application can handle the request asynchronously. This means it can start working on other tasks while waiting for I/O operations (like network calls or database interactions) to complete.
Sending Responses: Once the application has processed the request, it sends a response message back to the server, which then forwards it to the client.
Handling Multiple Requests: The ASGI application can handle multiple requests concurrently. This is because it doesn't have to wait for one request to finish before starting another.

Why ASGI is Needed for Async Django:

Synchronous Limitations: Traditional WSGI is designed for synchronous applications, meaning they can only handle one request at a time. This can be inefficient for applications that involve a lot of I/O operations.
Unlocking Async Potential: Django's recent versions (starting from 3.0) introduced support for asynchronous views and middleware. ASGI is essential for these features to work effectively. With ASGI, Django applications can take advantage of concurrency and handle requests more efficiently.
Improved Scalability: By enabling asynchronous handling, ASGI allows Django applications to handle more concurrent requests without sacrificing performance. This translates to better scalability for web applications built with Django.

Is async Django faster than sync?

The speed of Sync and Async Django can vary depending on the specific use case and implementation details. However, in general, Async Django is faster than Sync Django in situations where the application needs to handle a large number of concurrent requests or perform I/O-bound operations such as accessing databases or making API calls.

In Sync Django, each incoming request is processed sequentially, which means that if a request takes a long time to process, it can cause other requests to wait. This can lead to longer response times and slower performance, especially under heavy load.

On the other hand, Async Django can handle multiple requests concurrently and asynchronously, which means that it can process more requests in less time. Async Django can also perform I/O-bound operations asynchronously, allowing it to make multiple requests to external services or access databases without waiting for each operation to complete before moving on to the next one.

However, it’s important to note that Async Django is not always faster than Sync Django. If an application is primarily CPU-bound, such as performing complex calculations, then the benefits of Async Django may be limited. Additionally, writing asynchronous code requires more careful consideration and attention to detail, as it can be more complex and error-prone than synchronous code.

Overall, Async Django can provide significant performance benefits in the right circumstances, but it’s important to carefully consider the specific use case and implementation details to determine if it’s the right choice for a particular application.

Why writing asynchronous code is probably a good idea

Let's review a simple task that is common in web development - making API call.

Here's a detailed explanation of how the event loop in Python orchestrates asynchronous API requests:

1. Setting the Stage:

You define an asynchronous function (coroutine) to handle the API request. This function typically uses libraries like aiohttp to make the request and process the response.
Inside the coroutine, you use the await keyword before operations that might take time, such as network calls.

2. The Event Loop Takes Control:

When you run your asynchronous program, the Python interpreter creates an event loop. This loop acts as a central coordinator for all asynchronous tasks.
When your coroutine reaches an await statement, the following happens:
- The event loop recognizes that the operation might take time.
- It temporarily suspends the execution of your coroutine.
- This frees up the CPU to handle other tasks.

3. Making the API Request:

The event loop doesn't actually perform the API request itself. Instead, it delegates the task to the operating system's network layer.
The operating system initiates the request and sends it out to the internet.

4. Meanwhile, on the Event Loop:

While the API request is in transit, the event loop doesn't sit idle. It can:
- Handle other asynchronous tasks in your program.
- Process events from the user interface (if your application has one).
- Respond to other network requests (if you're handling multiple connections).

5. The Response Arrives:

Once the API server responds to your request, the response data travels back to your program.
The operating system receives the response and notifies the event loop about its availability.

6. The Event Loop Resumes the Coroutine:

The event loop identifies that the awaited operation (the API request) has completed.
It finds the suspended coroutine and places it back on the queue of tasks ready to run.
When the CPU becomes available, the event loop resumes the execution of the coroutine from the point where it was suspended.

7. Processing the Response:

When your coroutine resumes, it can now access the response data because the await statement has resolved.
Your code can then process the response, extract the information it needs, and potentially make further API calls or perform other tasks.

8. The Cycle Continues:

The event loop continues this cycle of managing tasks, suspending coroutines with await, handling other tasks while waiting for responses, and resuming coroutines when responses arrive. This ensures your program remains responsive and can handle multiple concurrent requests efficiently.

Conclusion

When you write django project you probably do a lot of common tasks that can be optimised by asynchronous approach. DB queries, API calls, working with files, working with media files. This means that you should probably look into utilizing django async support to perform multiple async tasks concurrently.

Denis Learns Tech