The main motivation behind building this tool was to make onboarding easier on Apache Airflow. There was no standard structure for a airflow projects and setting it up on local can be a nightmare sometimes. The simple CLI tool makes it very easy to create and test your project locally before deploying it to your production or staging environment via your CI/CD.
Right now we are using a docker-compose file which brings up all the Airflow services but we are also currently working on providing a command to control individual process.
Qubole is not a cloud but a self managed Data Platform. Deploying on Qubole means just putting all the Dag files on the machine (AWS/ GCP/ Azure) where airflow is running. Qubole provides out of the box solutions for running airflow on your cloud with a click of a button.
We offer bunch of different things (Spark, Presto, notebooks, etc) and have a great eco system build around Airflow.
Unconference - a meeting twice in a month where anyone from team present on any topics except controversial topics like politics. This helped everyone to learn completely different things
Initiatives:
More and more initiatives and idea discussion - because of this everyone started thinking everyday to build innovation and became adaptable to take risk.
1.5. Be willing to rethink the architecture when it turns out problematic. I have seen systems that probably looked good as a diagram before a line of code was written, but had to work around their own architecture with gross hacks.
Architecture is important, but organizations employing "software architects" tend to be bad at software.
The second part may read as a non-sequitur. The connection is that in organizations that have separate architect roles, architects are shielded from their mistakes because they don't work on implementation, and their work usually isn't questioned.
The worst thing I remember is a web API where some call could fail but didn't tell you, it just gave you some kind of plausible looking inert data. The call to query system status was separate, so there was always a time of check / time of use problem. Also there was a transition period when the original call did return an error but the system status API didn't yet. Nobody (I hope) comes up with such a disaster while implementing and testing it.
Looks great decision. this allows students to put their opinion in front of everyone without any resistance in class or anywhere(including those who follow old theory just because they learned in school).
For example, Darwin theory looks very illogical and nonsense to me as it is never proven, everything is based on assumptions.
This will definitely gives extra space to students thoughts to connect the dots between religion and science
Exactly for this, I have built https://IncidentOK.com which sends alert over email/slack to user if any third party services like this reports an outage or incident.
I regularly see your comment (or ones for similar services) on these types of posts.
I assume the strategy is working as you keep replying, but have you considered starting posting yourself, as a measure of "you can independently verify how quickly we spotted this outage"?
(For what it's worth, for the alert component you can do it yourself for free by subscribing to individual status pages (assuming they have the option) to the email-to-Slack integration.)
The main motivation behind building this tool was to make onboarding easier on Apache Airflow. There was no standard structure for a airflow projects and setting it up on local can be a nightmare sometimes. The simple CLI tool makes it very easy to create and test your project locally before deploying it to your production or staging environment via your CI/CD.
Right now we are using a docker-compose file which brings up all the Airflow services but we are also currently working on providing a command to control individual process.
Qubole is not a cloud but a self managed Data Platform. Deploying on Qubole means just putting all the Dag files on the machine (AWS/ GCP/ Azure) where airflow is running. Qubole provides out of the box solutions for running airflow on your cloud with a click of a button. We offer bunch of different things (Spark, Presto, notebooks, etc) and have a great eco system build around Airflow.