I've been involved in two separate projects where the backend tech stack choice was essentially "we'll just use Node". Both projects used Express, and it is pretty good, but almost everything else was different.
Both projects had (probably still have) a culture where if someone needed something, they'd first google for an npm, pick something, and just npm install that. Bleeding edge was viewed as a good thing. Kind of reminds me of the times when a new jQuery plugin was added for every little thing.
Soon, both projects will be using a large number of abandoned or obsolete npm packages which have been superseded by even more bleeding edge stuff.
Some day the dust will settle and preferred solutions will emerge for most things. But right now, man, it's a wild west.
If you grew up in it, it may not have seemed crazy. I played with rails 1, ignored 2, came in to a project with 3 RC1. loads of stuff breaking, but "they never do more than 3 RCs - we'll be fine before launch". I think we launched with RC5 (deadline was immovable). Felt to me like half the gems we needed were broken with Rails 3, and the answer was "just go fork it, fix some of these things, and use that". Felt fragile as hell, but the rest of the team acted like this just "how you roll" with Rails. I didn't expect that in something with a v3 on it, but perhaps because it was a major version bump it was more accepted?
I agree with all of the above. However, when a younger dev asks me which framework/language he/she should learn, my default response is: "If you're interested in employability, learn Node/JS. Practicality, go with Django/Python. Otherwise, Lisp (Clojure(script), Scheme, Common Lisp, hy)."
I'm surprised that Java isn't on here. I have seen piles and piles of Node jobs, and lots of Django positions (in the "cooler" corporate shops), but it seems like Java is still an outlandishly entrenched language, at least in giant companies. Do you think Node is better to learn for employability because every CS program in the country is churning out Java devs?
I think it's more the "JS" and less the "Node". If you're building a web application, for the most part, Javascript is part of the equation, especially as frontend dev has shifted from "write a few lines of html" to "render an application".
From there, Node isn't that big -- it's a few libraries for interacting with the system. The assorted framework stacks and whatever haven't really sorted themselves out yet.
From a technical standpoint, what Node does well is that it's basically async-IO-by-default, which is very nice for real world performance, whereas in most-every other environment I've worked with async-IO feels like an unwieldy addon where example code is hard to come by.
Java doesn't get much love. I've been looking for remote employment opportunities (I'm not based in the US), and a huge percentage of the postings I see are for Node.js and RoR. For Python shops it's almost always a requirement for Django experience.
Well, I was speaking within the context of web applications, but I do think Java wins in the "business app" realm, which most younger people want nothing to do with.
Also, 3 to 5 years from now is where I'm aiming at (JS will be "stable but still cool").
I don't think Javascript will ever be stable the way Java is. It will probably end up close to where Python is today. It's not just matter of time, it's a matter of fundamental design choices in the language.
My guess would be they have a Node app that serves the front end views (pages), but the data is gathered by making requests to the Django REST backend.
Yeah. Node is pretty much all about npm install this and that. I like pyramid but sometimes I do miss the battery included Django offers out of the box. Flask is my 3rd choice only because of the global, makes testing a bit hard.
We had to maintain a Pylons codebase that we never had the time to "port" to Pyramid (you know, rapidly growing startup, technical debt... then you get purchased by a big company and technical debt becomes a non-issue.) God, I hated a lot of that ecosystem. Lots of libraries that didn't work as promised (WebOb, WebTest, I'm looking at you) and upgrading them was impossible because newer versions broke Pylons.
All in all, I've always been a Django advocate. I know people hate the "batteries included" approach, but it guarantees that all parts work together well, and you'd be surprised how much of it is easily replaceable/removable. I've even used Django as a bare-bones router/request handler framework with great results.
I feel your pain. I used to work on a project started out with Pylons. But we got to give credit to Pylons & Pyramids folks. Despite being the least "famous" (well everyone has a different scale) these days, they still manage to pull in a lot of good code. Pyramid is my goto if I want to roll out everything on myself (once you know how to use things like tween it's really fantastic). I still can't get my mind around principal though. :{
I should go back to Django again, I started out with 0.90 when I first learned about MVC with Python.
Unless you need to extend or modify the behavior in an unsupported manner.
Django got much better since early 1.x versions, and gets better with every release, but I still find myself monkey-patching its internals (or copy-pasting classes with some minor changes) once in a few months.
RDBMS ORM : knex.js and bookshelf.js, way more usable than django orm.
forms : I gave up on form builders few years back, write them by hand
templates : EJS(prefer personally), Jade
authentication : passport.js, just works
file storage : node's fs module for local file's, aws node's sdk for s3
sessions : express-session with redis client and all
REST : express.js (supported by IBM, of all companies, heh), and restify, built over express
internationalization and localization : No idea, haven't had to do this one yet
caching : depends on the use-case, for request-response level caching, check out express middleware for the same, use ioredis for other things
logging : metalogger
mail : I mostly use a mandrill client, but I bet you'd find a few really good mailing implementations. You have an industry grade mail-server (haraka) in node world, so finding a mailing lib shouldn't be hard
sitemaps : Again, haven't needed them yet
RSS/ATOM feeds : haven't needed them yet
XSS : tackled by EJS, Jade etc templating engines
CSRF : express-csrf middleware ftw
clickjacking : helmet
SQL injection : knex.js
Secure connection and cookies : express + express-cookies middleware.
Finding each of these libs is a matter of 10-15 min of research, looking at number of downloads, frequency of downloads, contributors etc. And they just fit together really well.
You mostly don't need all the features you listed in all of your projects. Using node's libraries, with some effort, you can build a framework tailor-made for your project.
Yep. I'm going to get slammed on this but I love Django precisely because of the lack of choice. Just like Guido's "there is one way, and its the right way" mentality carried over from the core Python base, you see that same mentality it in Django itself. Sure, if you end up fighting too much against templates you can swap in jinja2, or an ORM in goes SQLAlchemy, but out of the box you have a pretty decent stack ready to use. It's the same reason why I like ASP.NET C# MVC. At this point, with all the NuGet packages, I can swap in any component I want basically, if I find the stock MS components lacking. Entity Framework not meeting my business needs? In goes NHibernate or Glass.Mapper.
I'm going to get a lot of hell for this since this is startup-land-latest-and-greatest-just-graduated-from-Stanford territory, but WebSphere and J2EE really got everything down for a god damn reliable infrastructure, REST or WSDL compatible, authentication/authorization via any provider you want ranging from Novell Netware to LDAP or AD, caching can be thrown in _every level_, if an appserver goes down, another node picks up the abandoned client sessions, you're getting high availability at just an absolutely remarkable level. Hate on maven (I've been down in the trenches with Scons and the GNU autotools set, you can get far worse) all you want but the ability to freeze and clone all of your dependencies into a local .m2 repository allows you to be 100% confident if an upgrade breaks functionality, regress & problem solved. I'm not even a J2EE advocate, more of a "please just work, I don't have the mental capacity to learn 60 packages + 15 workaround shims until all browsers are ES6 compatible.
I really like a lot of the options of node out there. Tons of packages are really useful and innovative (as opposed to Yet-another-templating-system). There are quite a few interactive projects that the v8 platform lends itself to, especially with socket.io, a lot more flexibility that didn't exist before. But between ReactJS, Flow, JSX (which confuses me even more when I context shift because I write C# with TypeScript), and 50 other packages out there, I'm completely lost.
From a managerial perspective, its going to be hell to maintain those projects 3 years down the line when all of your 22 year old engineers have moved onto their own startups/friends startups/across the country because their long-time-partner got into Harvard med/etc. You'll be left searching for someone with expertise in Jade, Express, Node, Mongoose, Nginx, Socket.io, JSX, [insert 10 other modules which will become critical to your architecture as time goes on]. It's great for the engineer who lands inherits that project on a 1099 capacity because they'll be able to bill whatever they want, but absolutely terrible in terms of long-term maintainability (and yes, I realize technical debt is often a "bond-offering" (heh) worth making in exchange for a faster MVP in some cases).
Yeah. Hearing the word "legacy" at the sole mention of $MORE_THAN_ONE_YEAR_OLD_FRAMEWORK made me smile. Food for thought on this:
- Joel Spolsky in Things You Should Never Do, Part I [1], saying source code doesn't rust.
- Old Code Rusts [2], arguing that source code doesn't change by itself in the way that a car can rust "by itself", but the environment in which the code is used is always changing
So maybe Django is already old fart technology? Well if it is, then it's a pretty powerful, solid, productive, reliable piece of old fart technology standing upon other pieces of old fart technology that, despite environmental changes (as defined in [2]), remain powerful, solid, productive, reliable tech powering many websites. I'd just cheer with young_dev to party on, Wayne, and go with whatever floats his/her boat.
And yet it's been improved upon. IMO, SCTP offers a superior set of features. TCP wins on ubiquity and SCTP loses by virtue of not making it past firewalls/routers configured to drop traffic that's not TCP/UDP/ICMP.
Uhm. There's plenty of downsides to TCP/IP. It's just very very hard to replace, to the point of not being worth it. Take a look at something like QUIC for an idea of improvements.
The one thing I wish was slicker in the Python/Django world is deployment.
We just got done migrating five complete websites to new hardware. In that process one had to be completely re-written from scratch and the other four had various levels of tweaking, adjusting and fixing.
The process was smooth as silk. No problems at all with deployment, live site testing, etc.
Oh, yes, these were all PHP sites.
I don't want to go into a Python vs. PHP discussion here. I'll simply say I am not a fan of PHP and by far prefer Python for most web (and other) work.
And still, I have to say, when it is time to deploy a Python site we have to go through a checklist to make sure every little gizmo that needs to be in place is in place. Well, now we have Ansible scripts, but that's besides the point. With PHP we just FTP from our local repository to the live server and it's up and running with almost no setup.
I also have to generalize and say that most of the Python/Django sites we've worked on are far, far more complex and involved than the typical PHP sites we see. This is where, to me at least, Python/Django shine.
Anyhow, maybe we are being lazy. Don't know. It'd be interesting to hear from others who might do a lot of Python/Django work.
The simplest way to deploy Django nowadays is uwsgi. It can do everything, including serving static files with good performance.
Setting up some routes, the virtual environment, and env variables is a breeze. It's easy to test locally to see if it's breaking, and trivial to restart.
uWSGI is a pretty amazing piece of tech. A small binary which can do practically everything you need out a web server, really fast, and simple to configure.
Might I point you to Fabric? IMO, it's much, much, much easier to work with than Ansible. At my startup, we've got a nice, 100-line or so script which does all the deployment work for our Django application. (Most of that isn't even Python-specific.)
Fabric exclusively uses the Paramiko SSH library, which means that it can't use any of the newer OpenSSH ciphers and is kinda slow. Ansible can use either. In my opinion, it's easier to work with than Fabric, too.
> With PHP we just FTP from our local repository to the live server and it's up and running with almost no setup.
Technically you can do that with Python as well depending on your setup. Although you may need an extra step such as "touching" your wsgi.py file. If you aren't caching your templates[1] (which you should in production though), you can upload template files and see instant changes.
I sometimes hear this, and I think its a case of comparing apples to oranges.
Yes, if you have a brochureware site with a few bits of database interactivity, you can just FTP a few files somewhere and have your PHP app up and running. The moment you start to introduce elements such as background queues, or more than a few developers on the team, you start to get into the realms of needing something more robust and doing config management.
Facebook uses PHP. I very much doubt their deployment process is to FTP a few files to the right place.
Rather than reply individually I'll just say thanks to all who made suggestions. I'll look into this and see what we can learn. We typically use nginx, gunicorn and supervisor with Ansible.
Apart from being used by everybody, do they have any other reason for using JS? In general, I'm curious why JS is picking up so much for backend development? Is it just that people like using one language?
I work on a product where JS is about 80% of the stack. In my opinion the JS ecosystem is best categorized as a bunch of sugar-rushed chipmunks running around shouting "we need better tooling". And you know what, it kinda works and its nice. Great tooling, great third-party ecosystem, really high quality modules / frameworks backed by large companies, and all the advantages of V8 and co.
This may not be the venue for me to surface this, but I've been trying to wrap my head around the MEAN stack for about a month now, and it doesn't really even register to me as the same language. I'm using Angular on the front end and ExpressJS plus Mongoose (MongoDB) on the back end, so yeah, it's all technically Javascript, but the real learning curve was in working out how to get these big frameworks to do what I wanted. I sincerely believe that if I was using an Angular frontend to call a Django REST service, I'd have had the exact same amount of trouble grokking what was going on (async calls being the obvious exception).
When I was trying to learn Express it didn't make any sense to me, batteries were not included at the time and I'm sure they are now or you can just ask Yeoman and it shall be granted. Since then I've watched Rich Hickey talks and started to dig into Clojure and started to grok the whole computing stack. The Clojure way is to compose your own web services rather than configure them. I think express works the same way. Build what you need. Frameworks are always more productive for a hackathon or a pointy haired boss's deadline. For building a system that will be robust, maintainable, and scale to n, pure logic plus minimal libraries are superior in the long run. And far more secure, both through purity and obscurity.
You only spend some time grokking the basic syntax of the language.
Most of the time is spent getting the frameworks working and talking to one another. The frameworks having the same underlying language does not gives you much advantage.
> I'm curious why JS is picking up so much for backend development?
The reason I reach for Node on a project is to handle my websockets. I use Django REST Framework + Ember as my main tools (super fast to build with) but, based on my testing, Python can't hold a candle to Node when working with thousands of websockets. It works great for what it was designed for.
Have you tried using Tornado/Twisted for the websockets? I'm curious about performance.
I'm building a side project right now and I'm using Django running inside Tornado (it's pretty simple to setup, really) so I can both have my cake and eat it (easy APIs, and a layer I can use to maintain websockets.) I wonder if the performance would be comparable to Node?
I tried every variation I could get to work including tornado, asyncio, and pypy. In my simple "how many websockets can I open" tests python would start erroring by 10,000 (no Django, just an empty websocket) while node hit 40,000 and was still climbing. Eventually I concluded it was a case of "with sufficient thrust pigs fly just fine". I still like python; just doesn't seem to be worth the effort to try to adapt blocking libraries into a non-blocking use case.
That makes sense. I'm actually very pragmatic when it comes to this kind of stack and I'll be happy to replace my current Tornado frontend with a Node frontend when I hit the limit (for now it's a very niche usecase, just me and a few friends, so it might be that I never need to improve it.)
I use/have contributed to this project: https://github.com/hendrix/hendrix which is a wsgi wrapper that uses twisted to serve django apps. Websockets are among the first things you get out of it.
I ended up with my primary tools being DRF + React/Redux. I'm genuinely curious about Ember, but haven't had a chance to play with it. What do you think of it? How's it stack up versus React-based frontend setups, if you've evaluated them?
For new SPAs Ember rocks. Also keeping up with the latest and greatest Ember is a breeze. I would not however like to refactor an existing front end application to use Ember. Probably React is better suited for integration with existing front end applications.
Although Ember really feels like idiomatic Javascript to me and they try hard to adhere to new web standards (e.g. web components, ES6 modules) it is also very opinionated about what should go where and how.
The guy is just more uninformed about the web development landscape than you are. He's like the Java or Microsoft of 2000's, that only knew their own part of the development landscape. In other words, Blub to the bone.
Do Ruby on Rails devs also get the same treatment? I know Giles Bowkett talked about Rails/node.js back in 2012 [0], but is the situation now reached a point that JS devs haven't even heard of Rails?
Comparatively young dev here -- I'd "heard" of Python/Django and Ruby waaay before I had any idea what Node was. Javascript, to me, was until recently that thing browsers ran to get pop-ups to fly all over the place. The developers I was working with just didn't talk about it.
It miffs me every time I see somebody write that, because it's wrong, and assume the name follows English orthography in some way. The 'dj' represents the affricate [d͡ʒ] in older Romani orthographies, the 'j' representing the palatalisation of [d] to [d͡ʒ]. If it was written with the Pan-Vlax orthography these days, it'd be spelled 'džango'.
I loved Django when I started using it about 4 years ago. Lately, I've switched to Flask and replaced the templating with React. I love the new way, but I still have a place in my heart for Django. Does anyone on HN still start new projects with Django or have you moved on as well?
Django-Rest-Framework still seems to beat any equivalent in any language that I've found... coupled with the admin, it's a dream. But I also use React for views, and avoid any HTML coming from the server whatsoever (except the admin, which is due for a more complete overhaul than this brings)
We built an javascript API mapping that's "isomorphic". The express server receives a request, and for each route we have a function that takes the request, uses the API mapping that sends REST calls to our django server to grab the appropriate data, sends the data through to the store (server calling createStore from redux), the body rendered using Provider from react-redux, and using the react-router, which is rendered back on to the express response. The state is pasted onto the page using JSON.stringify (with some caveats to escape </script> tags), and is hydrated back onto the elements on the page by the browser, by calling on createStore on the state pasted onto the page.
Like others have mentioned, I love django for all the batteries-included features (admin, sessions, ORM, etc). But I still haven't really found a good way to use Django with React JSX as the template engine -- rendering & caching serverside for SEO purposes, but still responsive client-side.
I haven't seen a good description of how to pull this off... but it would make a great eBook (I'd buy it!).
It seems like the Django RESTful API would be one solution... but it would result in a lot of extra REST requests (so blocking & latency) being generated during server-side rendering, so perhaps not the best solution.
To reduce blocking and latency from network calls, we spent a day or two to write a batch view library that will take a list of request bodies, parses it, use the django router to send each request to the appropriate django view, running the middleware, and in the response returns a list of response bodies. Implementing this protocol almost exactly. https://cloud.google.com/storage/docs/json_api/v1/how-tos/ba... We also had to write a piece of javascript client middleware that would delay all requests by 0.01 seconds and batch up all the requests that happen in that time to send to the batch view. Proprietary code I'm afraid we cannot share but there's the idea, it's not patented and takes about two days. We use server rendering for all non-logged in views.
I think I'm starting to get it... but perhaps I can parrot back for clarification? So you setup django as usual: REST framework with the normal REST API endpoints.
For client-side, everything works as normal: The React code starts with static values for state/props that are pre-populated (ie. the return value for getInitialState was already determined during server-side rendering). Upon app updates (eg. user interaction), the app will make asynchronous calls to the correct REST endpoints -- perhaps batched, as you mention. Client-side is the "easy" part of this one. :)
For server-side rendering you have a normal HTTPresponse-returning Django view (eg. for /). That view compiles/renders the React JSX "template" in NodeJS. Only for server-side, when you call renderToString to get the initial HTML, getInitialState is actually a series of REST requests (in javascript) which are batched, sent to django, and returned (map-style) to node as JSON for initial rendering.
Two questions:
(1) Does the REST request batching get sent to a different, special endpoint for batching? Or when you say "middleware", do you mean that any REST request is capable of being a batched request via the middleware?
(2) There seems to be some nuance in structuring getInitialState (or props) so that it makes the REST calls on the server-side, but not on the client-side. I'd be curious to see how you structured that.
PS -- You might shoot me an email (in profile) since this is somewhat OT to the thread? I'm super-appreciative of your insight!
The the server-side rendering description is a bit off. We have the Django REST framework server that never serves HTML except Django Admin. All pages go to the Node.js server which renders the HTML using React, and sends it through when the page is loaded. The same react code is also loaded onto the browser to update the HTML asynchronously later. We have two projects setup - one django project and one node project.
"Only for server-side, when you call renderToString to get the initial HTML, getInitialState is actually a series of REST requests (in javascript) which are batched, sent to django, and returned (map-style) to node as JSON for initial rendering."
Sounds mostly good, except we return the list of responses using the protocol I've linked to in the comment. (multipart response with boundaries, responses written one after the other in the same order as sent). This is taken care of by the middleware and is transparent to the rest of the node/react code. The data gets parsed as usual into JSON and used for initial rendering.
1. Yes. It's a special endpoint for batching. By "middleware" I meant client-side middleware. On django it's just a view we've written, that parses the request(s) and puts them through django's `WSGIHandler`, and extracts the responses and render them one by one into the same HttpResponse.
2. I've posted how we've done this in another comment. :)
Django is still my default goto for starting projects. I create backend data entry as an old style CRUD app. And I do the front end on wordpress and use REST API to fetch data from Django.
This is interesting. I have written a django backend written for an organization already having a wordpress site. Having never written a piece of php code i must ask: Are you aware of any resources that could be particularly helpful for the php/wordpress part? Or is it more like embedding a lot of javascript into your wordrpess site?
I write functions to interact with the REST api in PHP and use them in theme code.
Lets say you have a page for location based doctor search. When the search form in this page is submitted, wordpress makes API calls to django backend where real search happens and a list of doctors is returned to wordpress. Wordpress then genereates html and send it to the browser.
Similar process for dislaying a specific doctor details.
The CRUD and other application logic for doctor data is handled in Django. And with rest API you can then create a web based UI or a mobile application.
I use the same architecture... DRF for the API, admin for quick data manipulation, Ember for the interface and I enjoy it immensely. I am learning Phoenix because it would allow me to handle bidirectional communication in the same codebase along with other Erlang advantages. So far it doesn't seem to have anything comparable to DRF.
I used Django at my last startup ~3 years ago, but now I use Flask for any new projects. Django is great if you need all the functionality, but the more lightweight Flask fits my needs now. With flask-bootstrap, I can make something nice in a jiffy.
I do start new projects with Django all the time. It's very efficient and not too complex. Really a good fit for most site and it excels at creating APIs too.
The major reason it isn't using bootstrap is because the vast majority of changes have only been css. That was to allow for backwards compatibility for users that change the admin styling and html.
I used to use Django a lot. This news caused me to go back and check the release notes going back to 1.5. It seems like between every major point release theirs about a ~6 month gap. Furthermore the features in each release (bar migrations in 1.7, which IMO should've been in a loooong time ago) are relatively minor. Django also is fairly quick to deprecate older versions (including python versions) which is fine I guess since they're pretty on top of security and provide clear upgrade paths.
However the fact remains, as I assessed about two years ago, that the momentum behind the Django ecosystem is as stagnant as ever. Many third-party libraries are deprecated or ill-maintained. This is as much symptomatic of Python as a whole as Django, but regardless it exasperates the slow release cycles when compared to other languages and frameworks.
My money is still on JS/Clojure/Erlang/Go/etc. to carry us into the future of web development. No doubt Django will fight to the bitter end with its strong enterprise support and mature codebase. I'm ok with that, it's just not for me anymore.
I don't know what planet you're living in, but the Python ecosystem is about the most mature and modern I have ever seen for modern development in dynamically typed, interpreted languages with a strong web presence..
Take a look around and the equivalent libraries of the languages you mentioned. Honestly, the quality of similar libraries for the functionality that you need is lower, and the featureset smaller.
Javascript suffers from all the problems of a "teenage" tech: immaturity, lack of established libs, poor documentation, and some astoundingly bad coding practices out in the wild.
Clojure's ecosystem is very small, and has the same issues with code stability and lack of decent mainstream library alternatives.
Erlang, though a great language, has some issues with its unfriendly ecosystem and a general lack of libraries for many common tasks.
Go, while gaining serious momentum, still seems to suffer from a substantial lack of features to compete in the same rapid development niche that Python, Ruby and JS attempt to point to, although it was never meant for that role anyway.
If by "stagnant" you mean "mature enough that there's little need for constant refreshing of the libraries", then you are right. There's only so much you can do with a web framework, and only so many ways you can go around doing it.
There's literally nothing you can do on the newest, flashiest (read: unstable, poorly documented, non-battle-tested) frameworks that you can't do using a combination of Django and Tornado/Twisted in half the LOCs and using mature libraries. The same could be said about Ruby on Rails, or even something like Spring Boot for Java or Spray/Scala.
I don't mean to deflate innovation, but innovation for innovation's sake is rarely useful.
“saying things” is not a synonym for “trolling”. If those claims had been less vague and supported in any way, he wouldn't be getting downvoted.
Notice how the people above talking about WebSockets aren't being downvoted because it's both real and specific enough to have an actual technical discussion?
Young dev: "So, what's your backend?"
Me: "Django"
Young dev: "What?"
Me: "Django, a Python framework."
Young dev: "Oh, right, that old framework. It must suck to support a legacy app that's written in something so outdated."
Me: "Well, we kinda use Django for everything."
Young dev: "Wow, doesn't everyone just use JS these days?"
Btw, I'm 36.