looking at the release notes and the source, it looks like the asyncio interface is still using the old drivers, like psycopg2 for postgresql, and using a threadpool.
we at SQLAlchemy came up with a way to interface asyncio frontend and backend (like asyncpg for the driver) while maintaining all the "in the middle" code as synchronous style. the dark secret is that for this approach you have to use greenlet to propagate the "await" operations. It's been in our release for 18 months now with 1M downloads a day and there have been no problems reported with it, so i continue to wonder why nobody else seems to want to look at this approach. one downside which nobody has complained about yet is that it makes profiling tools like cProfile harder to use, but seems not to have come up yet.
there's also another approach, which is that you rewrite all your "in the middle" code as pure asyncio, then create a dummy task to implement your synchronous API. dark secret for that one is you had to rewrite everything and I'm not sure if there's other performance implications for having awaits all throughout code that's running only one task per thread.
Mike, thanks for this reply and your work on SQLAlchemy. My employer uses your framework, but not yet the newest versions featuring async.
Previously, and still in my personal projects, I have been a Django and Django ORM person.
I got away with by and large avoiding SQL statements in building web apps and have sometimes struggled to understand why SQL statements seemed much more forward-facing in SQLAlchemy in comparison.
Then I listened to your recent Talk Python podcast episode with M. Kennedy [1] and it helped me understand where the framework is coming from with regards to expectations for SQL capability from a developer that wants to use SQLAlchemy.
I believe the focus was much more so on the dark secrets you mention above, though and colored in how you discovered those.
If anyone wants to learn more about async or SQLAlchemy, I highly recommend listening.
As an aside, it was also cool to hear you talking about your project because for me, it humanized you as the maintainer. You sounded pretty chill, which was for some reason not what I was expecting.
I hope you can make it back on to that podcast with less of a gap between appearances next time.
thanks for sqlalchemy. it's a shining star in the Python package ecosystem.
> interface asyncio frontend and backend (like asyncpg for the driver) while maintaining all the "in the middle" code as synchronous style. the dark secret is that for this approach you have to use greenlet to propagate the "await" operations. It's been in our release for 18 months now with 1M downloads a day and there have been no problems reported with it, so i continue to wonder why nobody else seems to want to look at this approach.
do you have a blog post or some other kind of writeup to read about it? maybe nobody else had the same idea and it works so well nobody is aware that it can be done...
Is it a shining star or a rough dirty pragmatic get-the-job-done tool with a heart of gold thats approachable to newcomers and grizzled veterans alike. The trenches are too dirty for shining stars.
Someone mentioned it on a Django forum thread about async ORM [0]. It seems to have been ignored. Super cool idea and execution! I hope it gets some more exposure.
It includes running Django, Celery, gunicorn, Postgres, Redis, esbuild and Tailwind through Docker and Docker Compose. It's set up for development and production.
I was going to say you should offer a tutorial around how this can be used. It is much more useful to have a walk through when people put out such products.
And then I decided not to say that because it is up to you how to spend your time and why should you make a tutorial for people if you don't want to.
And then I realized, you have actually made dozens and this is the work you do. I think I took a Flask-docker course of yours on Udemy a few years ago. It was pretty good, and "forward leaning" for the time.
I know you're not allowed to shill your product on here, but presumably I can.
By the way the version of the course on Udemy doesn't get updated. The full version of the course is at https://buildasaasappwithflask.com/ which includes lifetime free updates (it has over 10+ hours of extra updates and new features not on the Udemy version). Udemy is a really bad platform for instructors which is why I've been trying hard to decouple myself from them over time.
As for a tutorial on using this repo specifically, did you find the readme file lacking in any way? I tried to write the readme to be fully self contained. It would expect prior Docker knowledge tho. I did give a talk at DockerCon going over a bunch of Docker best practices at https://nickjanetakis.com/blog/best-practices-around-product.... That talk live demos a Flask example app not Django but everything still applies to Django. The same patterns are used in both projects. It can act as a mini-tutorial on this repo.
I was going to say that you should really camelcase your url, because it's hard to see what it's saying, but even when you do that...
https://buildASASSAppWithFlask.com/ doesn't really help that much hahaha.
Thanks for this. I've been learning django and have been struggling to find a good example of a way to integrate modern js into django. Unfortunately not as much support as rails + esbuild
> That seems to be using WSGI, not ASGI, though, so no natively `async` views..?
One does not simply go async. Changing your entire app to go full async has serious implications, requires every library you use to support it and requires a completely different mode of thinking.
The example project includes an ASGI file if you want to use an async app server like uvicorn with async views instead of gunicorn. That would involve changing a couple of lines of code. That's very much a "per user" decision and in my opinion shouldn't be the default for the project when 4.1 only landed a few hours ago. It's going to take a long time until apps are ready to go async by default, both from waiting on libraries to support it and the community to shift towards this style of building apps.
I know one does not simply go async, sure, but I thought it should be noted (especially considering how async-centric this release is) that the repo doesn't immediately allow for async.
Also, "4.1 only landed a few hours ago" isn't the perfect argument in my books since support for ASGI and async views in general was added back in 3.0 and 3.1:
> I believe gunicorn with the uvicorn worker class is the recommended production server.
Yep for production that works. In development tho I don't think the gunicorn --reload flag works with that worker type.
There is an issue open for this at
https://github.com/benoitc/gunicorn/issues/2339 and since I last tried about a year ago I guess they have an unofficial "workaround solution" in one of the comments but it involves making a custom class to support reloading but that requires copying in some code into your project and also using different worker types in dev vs prod.
I think this is a good example of why I don't default to uvicorn and async in the project with Django 4.1. The whole package of "async everything" isn't ready for prime time in my opinion. It will improve over time but it's a waiting game at this point. Early adapters can easily swap it in if they want.
Interesting looks like it might actually be a python bug. Somehow just changing from sys.exit(0) -> os._exit(0) apparently fixes it.
"There is a problem with using the sys.exit() in a forked process (child processes or workers) instead of using the os._exit() that is mentioned in python documentation."
Great to see improving support for async here. I am a big fan, but IMO python async code is just different from other async langauges. I used async extensively in python and compared to other languages it is a pain to use because unless 100% of the the libs you want support it you will get stuck on some sync library. Async in python is not even a 2nd class citizen. Its more like a 3rd class citizen. Its gets better every year but it is minuscule compared to regular sync code, so very few tutorials mention it and nobody teaches it as the default way of doing things.
In order for async to be popular it needs to be the default way of writing python. Right now its an after thought for writing “high performance” python. For example asyncio version of redis got 72,114 downloads this month. The sync version got 26,825,663. Thats not even in the same universe.
Historically, using something like gevent is a more drop-in solution for python if you want lightweight threads, but it comes with its own problems. Gevent gives python what Zig has that automatically turns all sync IO calls to async and your app is now magically using greenthreads.
However I think gevent in the past was the crutch that prevented a lot of lib authors from writing async libs.
Hey, I wanted to chime in and say that I too am underwhelmed with the support for async in Python libs. Coming from a node.js background, it falls short because you constantly have to think about whether async code is calling sync code, or vice versa, and whether you need an executor, threadpool, etc. Just a mess that gets in the way of productivity IMO.
However, things have been getting better, and in case you haven't seen it, AIORedis[0] appears to be the de facto standard for async in Python (a little better with 1,724,389 downloads this month). It's popular enough that it's been merged with the official redis-py driver[1].
> In order for async to be popular it needs to be the default way of writing python. Right now its an after thought for writing “high performance” python.
Might this have to do with the fact that the (lack of) performance of asyncio is exactly the issue here?
I remember reading a blog post by zzzeek (the SQLAlchemy author who's also participating in our discussion here) about precisely this issue a while ago. Not sure but it might have been this one:
Yes, though I do think the python async ecosystem is gradually moving from a "3rd class citizen" to a "2nd class citizen". There are sync+async libraries like HTTPX that are gaining support. I don't expect it to really ever be a "1st class citizen" (supported as well as sync in pretty much every library everywhere) because of backward compatibility and ultimately async is just harder to program. Sometimes developer time is more important than the performance wins of async.
> Sometimes developer time is more important than the performance wins of async.
Also, a fair fraction of applications don't see a meaningful benefit compared to other things the developers could be spending time on. Async is not only harder to write in many cases but can force you to deal with new problems which might be harder to manage (e.g. if you use non-trivial amounts of memory, bounding your peak usage can be a challenge).
I've used async a fair amount and it's worked really well in network-heavy uses (HTTPX is really great) but outside of those the savings have been a lot less than people thought, independent of the language used (I'm thinking of a colleague who spent a lot of time trying to beat classic Python with Go and ended up with about a 10% win after a month or so of work because the problem had enough CPU/memory contention to prevent greater savings — that's not a trivial win but it definitely wouldn't have been enough to justify changing languages on its own).
The most anticipated Django release for me personally. Async ORM access is a massive quality of life improvememt, same for async CBV. The later is also a blocker for Django Rest Framework going async.
Sure most CRUD applications work perfectly fine using WSGI, but for anyone using other protocols like WS or MQTT in their app this is kind of a big deal.
Seriously: why use Django for a REST-like service? I inherited an application that (probably) evolved from form-based to REST with a JS frontend, and I fail to see the point. There's a model, a serializer, handlers to implement relations, and multiple views per data type. It's so much overhead, and everything has to be kept in sync.
I went with Django Rest Framework when I used it for a project because it provided things I would otherwise spend says implementing out of the box. The one thing I definitely did implement for myself was filtering, the built-in stuff was kind of mediocre for our needs, outside of that I want to say we left pagination and everything else as-is. It was a bit of a nightmare to look up documentation on though I will admit, if you don't know the Google Buzzword Soup you got to look up. Previously to that I was using CherryPy and building my entire kitchen sink myself, so it had nice moments and bad moments, but it definitely saved me on time and effort.
You typically do Django + REST because you are already comfortable with Django or because you were already using it, although I could argue, you could consider making your API project stand alone and using CherryPy or even FastAPI for a much nicer experience. The downside there being you wouldn't be using Django's ORM. I really do wish Django's ORM would be spun off into a stand alone project that Django then imports, so anyone else can use it in other non-django specific projects.
I remember watching a beautiful talk from some pycon whose details I can’t remember about using django as if it was a micro-framework.
It basically boils down to rejecting the folder structure that django-admin sets up for you, importing stuff yourself and doing some basic initialisation/configuration.
Once you do that, django kinda somehow behaves like a micro-framework, with the significant difference that you can import and use the advanced features if you want/need to.
I guess I prefer to keep it light and use a true microframework like flask for this use case. I find I can similarly import Flask extensions without bringing in a full heavy framework
Proper Django Rest Framework should basically have zero view code. Treat Serializers as forms, point at them in the appropriate generic view (or viewset) subclass, and you are done.
If you are writing view code in DRF, you are probably doing it wrong.
I don’t think this is fair - the DRF views are a great starting point for a project but I don’t think I’ve ever worked on one where they’re totally sufficient without at least some modification in places if you don’t want to make use of signals (which are very difficult to debug!).
Even things as simple as adding logging to calls on certain end points will require you do provide your own view method (which might then call the mixin’s provided method).
I don't find it has a lot of overhead if you keep it simple. No need to use class based views for REST services, just add function-based views with the @api_view decorator.
You can then within minutes return JSON coming from your data model by also adding an additional decorator to make it render JSON output.
If returning data directly from managed ORM objects, you will need to serialize it, but you can easily build an API without doing that and just return a dict.
It all depends how it is written. While Django and REST is quite strongly opinionated, you can still make it difficult to maintain if you want to.
That being said, Django REST allows you to move fast. It's very very easy to knock out a REST API and data models plus some basic admin panel.
Once your API matures - you can then reimplement it in Go or other platform.
It really shines if you are doing simple CRUD APIs where you can lean on autogenerated routes, views, and serializers. Since you can get full CRUD for a model in something like 10 lines of code.
When you need to wire up custom serializers per model (eg specifying subsets of fields or making some read-only) and wire up non-CRUD actions then I think you can fall down a slippery slope where you write as much (or more) code than if you just used say marshmallow and flask to manually write your API.
Also you get the Django model admin for free, so for internal services you have a nice CRUD admin for operators.
I think it’s a great tool for accelerating your first 100-200kloc of API code. Beyond that it can start to creak at the seams depending on your usecase. (I’d make the same assessment about Django itself FWIW).
It's a really easy framework that just takes a few days to get proficient in. It is very well documented and because it comes with so much "overhead" you don't have to get your own team to make a set of different mistakes building the features as needed.
every python developer has accidentally made django at least once.
you start with "i will do in flask because it's easier". then you need to add authorization/authentication. and then templating, ORM, maybe some RESTful flask library to make endpoints a bit easier... and now you have django, but in a different way.
I just make normal money and I don't want to have lunch with Jeff Bezos. Maybe in this scenario you lose money each time you hear it and you're so in debt that Jeff Bezos has basically purchased you :P.
It makes sense as backend for a native iOS/Android app, and if you already have that REST endpoint then it can make sense to re-use it for the web-version of your app.
But if you're not doing an iOS/Android app (most websites?), then I generally agree it's a lot more work compared to form-based / server-side-templates (though sometimes can give a _slightly_ better user experience, _if_ done correctly.)
HTMX seems to be growing as a "REST endpoint" alternative, but again not as useful if you're also doing a native iOS/Android app.
In this day and age, mobile apps are _almost_ a must so REST is everywhere.
A well-implemented REST app is pretty nice, business logic divorced from all UI and API lends itself nicely as a comfortable and maintainable programming environment. More often than not, being RESTful from the get-go tends to have positive returns down the line.
> In this day and age, mobile apps are _almost_ a must so REST is everywhere.
This really isn't true. Mobile apps are far more expensive to develop (everything is doubled, you're dealing with periodic forced updates outside of your development schedule, etc.) and there are a ton of apps which don't need anything you can't do in a browser.
One of the earliest things to do on a project is agreeing on what level of complexity makes sense for you. If your project doesn't have a known hard requirement for mobile apps and correspondingly large team sizes, a classic Django app is likely to let you iterate an order of magnitude faster at lower cost overall.
I completely agree. I have been using GraphQL in Django with WebSockets and ASGI and it was a bit of pain in the ass to get everything working, but with this changes things will get much smoother.
`QuerySet` now provides an asynchronous interface for all data access operations. These are named as-per the existing synchronous operations but with an `a` prefix, for example `acreate()`, `aget()`, and so on.
> The new interface allows you to write asynchronous code without needing to wrap ORM operations in `sync_to_async()`:
async for author in Author.objects.filter(name__startswith="A"):
book = await author.books.afirst()
> Note that, at this stage, the underlying database operations remain synchronous, with contributions ongoing to push asynchronous support down into the SQL compiler, and integrate asynchronous database drivers. The new asynchronous queryset interface currently encapsulates the necessary sync_to_async() operations for you, and will allow your code to take advantage of developments in the ORM’s asynchronous support as it evolves. […] See Asynchronous queries for details and limitations.
## Asynchronous handlers for class-based views
> View subclasses may now define async HTTP method handlers:
import asyncio
from django.http import HttpResponse
from django.views import View
class AsyncView(View):
async def get(self, request, *args, **kwargs):
# Perform view logic using await.
await asyncio.sleep(1)
return HttpResponse("Hello async world!")
I don't fully understand the sync_to_async() operation here. So the underlying database is not async, then is Django just tossing the sync DB call onto a thread or something?
Yes, it is just being put on a thread. A future Django release will use async database drivers. Django is very incrementally adding support for async. I don't believe they have started work on async templates yet, which will be one of the last major areas before the framework is fully async.
Yes, they're working from the top of the stack down, so first it was views+middleware, now it's the top layer of the database stack, and in future releases they'll work on pushing native async lower and lower in the stack.
Basically every function call that does db-queries needs an "a"-prefixed version. aget(), acreate(), aget_or_create(), acount(), aexists(), aiterator(), __aiter__, etc. And in future versions they'll work on the lower level, undocumented api, like _afetch_all(), aprefetch_related_objects(), compiler.aexecute_sql(), etc.
Eventually you'll probably need to switch database drivers to something that actually supports async.
Why do they need Async templates? When a template renders it should be pure data and fully on the CPU. It seems Async templates would encourage a bad behavior of doing n+1 queries inside the HTML.
My immediate thought would be custom template tags. Like `{% popular_articles %}` would have to connect to the DB and load the articles. I am sure there are other reasons too.
Also, async views allow for better scaling. Say your view needs to make an API call to a third party service within a request-response cycle. Currently, this will block an entire worker/thread while doing a blocking request.
With async, we can use async HTTP libraries and scale these WAY better.
Or just run 20 workers and let the OS handle async. OSes are pretty good at this. Async is pretty hard to reason about.
Async is beneficial when you make multiple blocking (http) request to a resource at the same time and later fold that into one response. Parallel stuff. Or when you are forced to run a single thread. Or when you are memory bound.
Reality is that most requests depend on previous requests quite often. In the latter case there is no benefit in terms of speed for the user.
Might be unpopular but still good to stick to it. I tend to associate excessive enthusiasm for pythons async await framework as a sign of immaturity and potential for doing things without understanding them fully. I’d never trust async code from anyone except the best python engineers. Most of them out there can’t even debug linear code, adding async to the mix only makes it worse. At least you can trust the time stamps.
>I tend to associate excessive enthusiasm for pythons async await framework as a sign of immaturity and potential for doing things without understanding them fully.
Couldn't it be the inverse? I.e. your inexeperience with async code in other languages, and not understanding it fully, translating into "new thing aversion"?
I think my experience in async in other languages has shown how rough async in Python is.
Unless async is the default python, it will always be an after thought and introduce needless complexity when working on large projects that require a lot of external libs.
Or when accidentally some blocking database driver is used, even when django's ORM is async, a query with mongodb's driver is not async. Or just the filesystem... This harms performance badly, especially if the idea was to run only a few threads.
And indeed, there's an important caveat in the Django 4.1 docs;
Note that, at this stage, the underlying database operations remain synchronous, with contributions ongoing to push asynchronous support down into the SQL compiler, and integrate asynchronous database drivers....
> Or just run 20 workers and let the OS handle async.
That just means you can only handle 20 concurrent requests before performance drops off a cliff. If the third-party service you are talking to is slow, then you’ll just end up with 20 workers all waiting for the service to respond, while other requests pile up. With async, those workers could still be handling other requests. Adding more workers because you are blocking on i/o works for low traffic services, not for anything remotely busy.
> Adding more workers because you are blocking on i/o works for low traffic services, not for anything remotely busy.
Yes.
That just means in very bad cases, you can only handle 1 "concurrent request" before performance drops off a cliff. If the third-party service you are talking to is that slow, then you’ll just end up with bufferbloat (high latency in hidden queues) waiting for the service to respond, requests piling up.
With async, unless you know what you're doing and handle back pressure, work pile up even worse, the service stops working and you get 0 throughput instead.
Agree! Up the number of workers to 40, fix your application in another way... or indeed go with async stuff! ;-)
I was just saying that async quite often is not of much help and just complicates things. Of course it has its use! Your case is a great example of converting a django view into an async view.
Why would you block a process that could use CPU cycles to do something useful just to wait on I/O? With the right primitives async is not "hard" nor inconvenient. Maybe you're referring to concurrency?
No I am referring to async, really. Accidentally blocking operations like open() or psycopg or mongodb or some included library which (accidentally) uses any of that stuff. Basically any library requiring requests is not usable. Or worse: open().
It can be hard to determine if something is doing something blocking and if it is, it is hard to debug. Especially in python world which, unlike javascript, was not async from the start.
I don't understand, are you saying async is hard or that a poor implementation is a source of issues (e.g. async function that calls sync primitives)? Because I totally agree with the latter but I have no idea what "hard" means. Very little changes in terms of syntax (assuming support for async/await ops) or reasoning.
> Or just run 20 workers and let the OS handle async. OSes are pretty good at this. Async is pretty hard to reason about.
def slow_sync_view(request):
# these requests won't be executed in parallel;
# async version could eliminate this extra latency
foo = requests.get('https://www.google.com/humans.txt').text
bar = requests.get('https://checkip.amazonaws.com').text
return HttpResponse(f'{foo}\n{bar}')
I think you are being down voted because a straight port of this code to async would still run one request after the other.
You would have to queue one task for each request in the event loop and then await for them both to gain some parallelism in the I/O section of the code.
Yes, if you do WebSockets, async really is a must if you don't want to have one thread per active connection. Each thread takes 8mb of virtual memory, not sure how much actual memory.
It's also nice for a backend that's doing a lot of proxying apis to another backend.
This is going to make django-ninja even easier to use, so it's very welcome.
And if you like django and never tried django ninja, it's like DRF and FastAPI had a baby: https://django-ninja.rest-framework.com/. You define your endpoints using pydantic type hints, and it generates the API, validators and docs all in one go.
Django is supposed to be opinionated and batteries-included, Django Software Foundation ought to shepherd some REST package. There's ninja, DRF, etc. and a few more on the horizon. I agree with the other poster, RESTful projects with views one encounters: often complicated and not easily maintainable. DSF should take over django-ninja. :)
Django is single-handedly responsible for making me fall in love with programming. I am now using Elixir/Phoenix and 100% sold on the benefits of working with immutable data, but I still keep an eye on developments in the Python/Django ecosystem. Seeing Django edge closer to having a fully async stack is very exciting, and should make it a more viable platform for existing users as well as newcomers.
i have dabbled with web programming several times in the past, with a variety of frameworks (sinatra, rails, phoenix, fastapi, and now django). django is the first time i have felt truly productive - its architecture is the perfect balance between sinatra (doesn't do enough) and rails/phoenix (where i keep getting lost in a maze of generated code). also the orm really gets out of your way.
I agree. Django is the only framework that makes sense to me. I have been doing web programming full time since 2006, using only vanilla code ( there were no frameworks back then). Django is the only framework that tempts me.
as someone who is re-learning to code now after a 15 year absence, django is for sure making it feel amazing. Sometimes folks in this community overthink normal people's actual needs when it comes to programming tools, Django seems to support most everyday uses just fine.
Am I bad SWE if I do not care for for async in django?
I am sure a a lot of django devs will like it, but if I need the performance that I would get from going async, I would not being using django as my web framework.
FWIW I like django / python for a lot of things, just not for performance.
Enabling async means those already invested in Django can do more on one machine without a complete rewrite, and those considering it can adopt it with fewer reservations. There will always be more scalable platforms out there, but enabling async with Django helps it serve the “sweet spot” in terms of both ergonomics and scalability.
Probably not. If you need async for DB "stuff" to take advantage of your resources, I think it is a rare thing, and likely you would not want to use the ORM anyway, and then can just do whatever you want without it.
No, I agree with you. It's 2022 and I have thought only once that it would be neat to have async methods from Django, but only as some 'cool thing to try'.
I don't actually actually know if there would be any substantial benefit. It's like when I tried PyPy as my django interpreter. It gave a few percent speed increase in "real world" but created extra work for deployment.
I've been using Django non-stop from 0.96, and it just works, and you can build around it. If I was running a trading clearing house, or Instagram, I'd probably care about marginal improvements, and then use another language. But for Django, just tuning the workers per server node, and using django-rq jobs works really well.
Kudos for the enthusiasts working on this though, because 'async' is a new Python feature/meme, and supporting it is just being a good ecosystem player.
Right, though they're stuck on 1.8 (upgraded to 1.8 only so they could use python3), and really just use it for request/response/views, not ORM or anything else.
> Django doesn't default support REST endpoints, does it?
You just have to manually call json.parse(request.body) for incoming requests and return JsonResponse(data). And you generally need to serialize the data yourself.
Django does have some built-in model serialization, but that's mostly for import/export data, not really for sending to the browser.
Async doesn't make your code faster, though there are cases where it lets you handle more requests with a single server (especially if you have code that is async). But moreover, if you have libraries that use async code, it's really not ideal if the thing that runs those libraries isn't async. You don't have to like or want it, but it's there for the folks who want or need it.
> Am I bad SWE if I do not care for for async in django?
In a way, yes.
Async in django means that with relatively minor adjustments to your codebase you may significantly increase your requests over time throughput.
And as someone else pointed out, if you have database queries that do not depend on each other you can effectively wait for their results in parallel, which might this time not only increase throughput but also lower the total request time.
It’s a very valuable development that brings significant gain for relatively minor changes.
If you are a professional django developer you should not dismiss such developments like that.
I'll qualify mine by pointing out that it doesn't have a database and is just a web client, a veneer, over a very secure API that really does all of the work. The Django I use is nothing but a template rendering thing.
Once the underlying database queries become async, I can see it becoming popular. Many views do multiple database queries that aren't dependent on each other, which would be a quick win to make async.
Since ultimately anything you build in Django will eventually go into the ORM layer, it won't become anything unless the ORM is fully async. This release brings the async API to it, but:
> Note that, at this stage, the underlying database operations remain synchronous
That kinda defeats the whole purpose of an ASGI app.
>That kinda defeats the whole purpose of an ASGI app.
Unless I’m missing something, wouldn’t an async ORM provide significant benefit for apps with an internet-based DB connection? I would think the underlying DB ops would be much faster than sending a request and waiting on a response.
You don't use async for CPU. Python async is actually asyncio, so I/O and network such as calling external API, etc.. It'll make integration much easier to reason about.
I use the event loop to defer the (io heavy) post-processing of some requests without involving an off-process task runner like Celery. I use a custom implementation of this: https://django-simple-task.readthedocs.io/how-it-works.html It is way simpler when you don't need strong guaranties and checkpoints persistence.
There are a lot of comments about async ORM, but to me the support for application-level (e.g. if you use forms, admin, DRF, Graphene) validation of DB-level constraints is a super cool feature that solves a real pain point - having to duplicate this logic at both levels, or forgetting to do so.
Except DRF doesn’t call clean() anymore (as of 3.0?), so you have to reimplement all your validation logic in your serializers anyway. Always seemed like a huge mistake to me for a framework that’s supposed to feel like a natural extension of Django.
we at SQLAlchemy came up with a way to interface asyncio frontend and backend (like asyncpg for the driver) while maintaining all the "in the middle" code as synchronous style. the dark secret is that for this approach you have to use greenlet to propagate the "await" operations. It's been in our release for 18 months now with 1M downloads a day and there have been no problems reported with it, so i continue to wonder why nobody else seems to want to look at this approach. one downside which nobody has complained about yet is that it makes profiling tools like cProfile harder to use, but seems not to have come up yet.
there's also another approach, which is that you rewrite all your "in the middle" code as pure asyncio, then create a dummy task to implement your synchronous API. dark secret for that one is you had to rewrite everything and I'm not sure if there's other performance implications for having awaits all throughout code that's running only one task per thread.