Awesome that migrations are now in Django core. I do see some pitfalls regarding Historical Models though.
The problem is that application code usually changes over time. Consider a deployment that hasn't been migrated in a long time. When migrations are eventually run, the older migrations might assume application code behaved a certain way. This is why you should never use your regular application models in migrations. Both South and Django 1.7 migrations copy "shallow" versions of your models (no custom methods, save methods, etc), which helps to encourage developers to keep the migration isolated from application models that might change. The problem with Django 1.7 though is that these shallow versions retain any inheritance they had at the time the migration was written. So if that class goes away in the future, I presume the migration will break. Worse yet, if the base class behavior changes, the migration run later in time might behave differently than you intended.
For this reason, custom methods even from inheritance should not be saved in the historical model. Only the fields should be saved off from the base classes, and then merged into one class (remove any inherited classes). Any other app code needed for the migration should be directly copied into the migration itself.
That's a good set of pitfalls to watch out for. Mainly, I just find it odd that in Django 1.7, they are using frozen models to mitigate some of these problems, but don't remove or freeze the base classes, therefore leaving the same problem. I'm sure the migrations system will get better over time.
Thank you. This is closer to a much better upgrade path. The official documentation is really not a good way to have non-technical customers move from South to Django 1.7. With the new version of South, you could package both old South (perhaps some of them not yet run) as well as the new migrations. It could be better though. With this approach, they would still have to:
1) ./manage.py migrate
2) pip uninstall south
3) ./manage.py migrate
Better yet would be to have either South or Django 1.7 migrations check for the other's existence and run both sets of migrations and not force South to be removed. This would make it a truly seamless transition for the system administrator.
Passing objects to Celery and not querying for fresh objects is not always a bad practice. If you have millions of rows in your database, querying for them is going to slow you way down. In essence, the same reason you shouldn't use your database as the Celery backend is the same reason you might not want to query the database for fresh objects. It depends on your use case of course. Passing straight values/strings should be strongly considered too since serializing and passing whole objects when you only need a single value is not good either.
Oh absolutely values before objects. I said "serializing" more in the sense that pickle is always used for storing the arguments into the queue (or whatever the default serializer).
It always depends on your use-case but generally you want your application to behave correctly, which means it has to have correct/fresh data...you can't sacrifice correctness because of an inability to scale your database.
Yes. I think saying "you can't sacrifice correctness because of an inability to scale your database" is perhaps conveying the wrong message though. I mean, your very first point is about database scaling issues and the advantages of using something like RabbitMQ to avoid expensive SQL queries.
If you are processing a lot of data in Celery, you really want to try to avoid performing any database queries. This might mean re-architecting the system. You might for example have insert-only tables (immutable objects) to address this type of concern.
What happens if more traffic is coming in than Packetbeat can process? Either you are dropping that traffic or you are (perhaps unintentionally) blocking it, right?
As people have pointed out in this thread and other articles, clearly there can be issues with using /dev/urandom on boot where there is a lack of entropy. What I would love to know more than anything on this topic though, is whether it might ever make sense to use /dev/random after boot--after the CSPRNG has been properly seeded with good entropy. Specifically, I would love to hear of any real attacks. Anyone?
It makes no sense. The only difference between /dev/random and /dev/urandom is that random uses the estimator of the seed entropy (which doesn't make that much sense to begin with) to block when it "estimates" that there wasn't enough entropy in the seed, but apart from that gives numbers identical to the ones urandom gives.
Maybe I'm not following your response. What I'm saying is that there are reasons to use /dev/random at least after boot (to block for seed entropy). Saurik gives a real world example of the "cold boot" problem as other articles have done in the past. Are you saying we should be able to simply stop there and then just use /dev/urandom? If so, then why is the CSRNG re-seeded every 60 seconds? It would seem that there are reasons for doing so.
> Are you saying we should be able to simply stop there and then just use /dev/urandom?
Yes.
> If so, then why is the CSRNG re-seeded every 60 seconds? It would seem that there are reasons for doing so.
From what I understand about it (IANACryptographer), mostly to safeguard against situations where an attacker can learn the internal state of the CSPRNG at one moment. But I don't think this is too relevant for the problem at hand.
The only difference in Linux between /dev/random and /dev/urandom is that /dev/random always plays a guessing game about the seed entropy. The main argument here is, I guess, that this is a silly game. How can you really estimate entropy? It doesn't only depend on what physical sources in the machine give you, but also on what the attacker knows or can know about these sources and the state of the machine. How do you estimate that?
This was the main motivation behind the Fortuna CSPRNG by Schneier, Ferguson and Kelsey. There is no guessing going on, rather the algo is designed to not need to guess. Interestingly, the way Fortuna works seems to me would also somewhat mitigate the kind of attack DJB talks about in the article referenced by OP[1].
Be aware that this version of DDT is likely broken if you are using require.js or other AMD loaders. It may even fail in non-AMD environments if you are using jQuery. This appears to be related to DDT now loading its own jQuery in a non-AMD way. For status see:
Overall I think their approach for easy installation and configuration is a good one. I struggle with these same issues at my job--we sell products with a complex application stack to customers that often have no system administrators. The only issue I see here is with the upgrade path. Particularly for a product that is meant for file storage, I can't imagine downloading a 1TB backup file and uploading it again every time there is a new release.
The Appliance actually does not store any file data on it, so your appliance upgrades likely won't include 1TB backup files :)
The (optional) AeroFS Team Server is what would store file data in the company if you wanted to, but many of our customers actually just end up using the direct peer-to-peer syncing without a team server.
I for one agree with you. There are a few specific cases where client-side crypto has benefits regardless of the fact that you need to (at some point) trust the server. I think the best examples involve protecting history or data residing on the server up to the point of being compromised. See my other comment on here for my use case.
Having the crypto code delivered by the server to provide real-time "end-to-end" security is surely a problem. But, I believe there is a specific use case where even not-so-perfect javascript crypto increases security in a significant way. Imagine you have a system where you want to send encrypted files that are stored on the server to be downloaded later by other users. With javascript public key crypto a la PGP, the user could encrypt this file using keys stored locally. These keys are "uploaded" to the web page when they are needed but not actually sent to the server. The cipher text only is then uploaded to the server and stored. Recipient users can then download the file and decrypt in a similar manner.
So what does this help protect against? You are mitigating the situation where the server becomes compromised. In the event your server is compromised, any previously encrypted files are protected as long as the keys are not used again after compromise (since malicious javascript could be delivered to obtain the key).
The problem is that application code usually changes over time. Consider a deployment that hasn't been migrated in a long time. When migrations are eventually run, the older migrations might assume application code behaved a certain way. This is why you should never use your regular application models in migrations. Both South and Django 1.7 migrations copy "shallow" versions of your models (no custom methods, save methods, etc), which helps to encourage developers to keep the migration isolated from application models that might change. The problem with Django 1.7 though is that these shallow versions retain any inheritance they had at the time the migration was written. So if that class goes away in the future, I presume the migration will break. Worse yet, if the base class behavior changes, the migration run later in time might behave differently than you intended.
For this reason, custom methods even from inheritance should not be saved in the historical model. Only the fields should be saved off from the base classes, and then merged into one class (remove any inherited classes). Any other app code needed for the migration should be directly copied into the migration itself.