Just wanted to share a Colab alternative I work on called Gradient[0] (also includes a free GPU).
Some of the key differences:
- Faster storage. Colab uses Google Drive which is convenient to use but very slow. For example, training datasets often contain a large amount of small files (eg 50k images in the sample TensorFlow and PyTorch datasets). Colab will start to crawl when it tries to ingest these files which is a really standard workflow for ML/DL. It's great for toy projects eg training MNIST but not for training more interesting models that are popular in the research/professional communities today.
- Notebooks are fully persistent. With Colab, you need to re-install everything every time you start your Notebook.
- Colab instances can be shutdown (preempted) in the middle of a session leading to potential loss of work. Gradient will guarantee the entire session.
- Gradient offers the ability to add more storage and higher-end dedicated GPUs from the same environment. If you want to train a more sophisticated model that requires say a day or two of training and maybe a 1TB dataset, that's all possible. You could even use the 1-click deploy option to make your model available as an API endpoint. The free GPU tier is just an entrypoint into a full production-ready ML pipeline. With Colab, you would need to take your model somewhere else to accomplish these more advanced tasks.
- A large repository of ML templates that include all the major frameworks eg the obvious TensorFlow and PyTorch but also MXNet, Chainer, CNTK, etc. Gradient also includes a public datasets repository with a growing list of common datasets freely available to use in your projects.
Those are the main pieces but happy to elaborate on any of this or other questions!
After spending 5 minutes on the site, I could not figure out what an offering comparable to Colab Pro would cost me (i.e. single instance of a notebook hooked up to P100), and bailed. In contrast, Google puts pricing front and center, which makes me feel better about their value prop in spite of the fine print around limits and preemptibility.
Thanks for not being defensive. I do believe there's not enough competition in this space, so I wish you guys the best of luck.
Let's look at the page you linked. On the top of the page it says "free GPU" and for some reason lists three GPU options of varying capacity, all three "free". But slightly below it's very much "non-free GPU" territory. Which is it? Is it free, or are you going to hit me with a bill later? And if it's free, what's the catch? The "free" stuff on the top of the page invites me to create an instance. Hmm, ok, let's see maybe I'll see the actual pricing there? Nope, it wants me to create an account, which I'm not going to do just to see how much something costs.
I have been a user of colab and recommended to many friends. As mentioned in the other thread the session time of colab notebook is too low, so increasing it for pro is good, but still if one has a large data for testing and training, dependency on google drive or products becomes one another bottleneck, but not that bad.
My biggest concern with google product is, it might be shut down any time when any small and medium company begins to depend on it. This is the legacy of google in which anything which cannot generate revenues like it's advertising business it's shut down. So although my team use from time to time google colab, we keep an alternative always open.
Hopefully with this 9.99/month charge it might become viable business but will not be as significant as the google's primary business of advertising, so there will always be sword at the neck of this product.
The issue is not as acute in this case. These are just Jupyter notebooks, you'll be able to run them on your own hardware with minimal, if any, modifications, unless you choose to use TPUs.
We use both GPU and TPU, but use our own installed Jupyter to keep a copy as backup. Many of our developers like colab, just the reason we keep backup because we have been bitten by google shutting down services more than once. In one case we need to do a re-write of the complete app.
I found Gradient and Paperspace pricing to be incredibly opaque, and even a bit deceptive.
First of all, Colab works on a "resources are free, just not guaranteed" model. When I was about to sign up for Developer, with "Free + Low-Mid Instance Types", I expected that I would have reasonable, shared access to low-mid instances as part of my subscription.
(otherwise.. what am I paying for? Don't charge me... to charge me.)
I then found out that you have to pay a per hourly cost for low-mid instance types; making it a better buy to just use AWS and GCP directly yourself. (When you suspend instances on GCP, you don't get billed for CPU or GPU hours; just storage).
And finally, after that, I found out I had to separately for storage, as well as a $5 deposit minimum or something like that! Colab doesn't have anything like this. AWS doesn't bill a minimum of $5 for each instance, even if you spin it down after a few minutes.
And finally, I couldn't even use the instances I thought I'd get access to, because you're out of availability. Which is really questionable: I'm paying per-hour rates, on top of a SaaS monthly subscription, and plus storage costs with minimum floors (probably a high margin item for you); for a very constrained capacity of GPUs?
We overpay for AWS, and GCP (their margins are insane! esp on bandwidth) because they guarantee most people availability. If I want to deploy 3 gigantic instances for a few hours, (and I have the right quota limits); I can. Sure I'm paying a lot more than what I should, but it's like taxi drivers: you're not just paying for the cabbies time as he drives you, but also his idle time as he looks for a customer.
I don't understand what I am paying for here. I feel like I'm just being charged margin for you to wrap around GCP or AWS with a colorful UI. I do not feel comfortable with you having my credit card details.
If you made a subscription, around $20 per month maybe, that gave me reasonable access to a mid-tier GPU instance subject to capacity at no marginal cost, I would happily subscribe.
Until then, I'm going with Colab, because the beauty is that your costs are fixed and not marginal.
The Colab model of "resources are free, just not guaranteed" model” that you mentioned is literally identical to Gradient: The free tier includes spare capacity at no cost just like Colab. You can upgrade to access high-end instances eg the NVIDIA V100. These additional instances do not have a max runtime as well. I can’t see how this would be a con given this additional option is simply not available in Colab.
Gradient does not have a $5 deposit for storage, the free plan does not cost anything. There is no credit card required. In fact, Gradient doesn’t charge for storage at all.
Some more feedback: When I open the linked page, the page title instantly starts flashing between the actual title and "(2) New Messages!". At that point I knew without a doubt that this was not a product I would ever use.
The "Contact Sales" button on the landing page makes things even more confusing. On the one hand you're using abusive practices like the one above to grab attention, something I usually associate with cheap, low-quality consumer products. On the other hand, your main call-to-action is "Contact Sales", something associated with white-glove enterprise-type products.
Good idea, but it's the first premium product that I've seen where the pitch is 'you may get certain features if you subscribe'. In another words there is no guarantee and a premium subscriber may still end up with same GPU as a free user. You may end up with a high-end V100 (not available to free) might be a better pitch.
I mean, it's also 10$/month, you get what you pay for, and for most people this is enough. If you need more, you can get a dedicated GPU instance, but obviously that'll cost much more.
A fair use policy, which you seem to be referring to [1], is pretty standard fare for many 'invididual user' products to exclude heavy-use groups.
However, they could at least define expected and minimum capacity. They might omit it because the business in this – aside from capturing users in their ecosystem – is arbitraging wholesale GPU price against consumer monthly needs, along with scaling the free tier.
I would be happy with an explanation of "none of our service is guaranteed, but if we don't manage to give you your chosen GPU more than 80 percent of the time, you are free to cancel, and we will refund your final months subscription payment".
Since the subscription is monthly with no commitment, I think the viewpoint is probably something like "if it's not meeting your needs then unsubscribe." A refund of $10 is not going to be economically meaningful to folks who want to subscribe to this service. "Chosen GPU more than 80% of the time" sounds nice but it opens you up to adverse selection. They aren't looking for customers who want to pay ten bucks a month to rent 80% of a high performance GPU. That's not nearly enough money for that kind of commitment.
They have a product with guarantees, but the market price for that is literally 100 times more than this one. If you want a certainty of getting the calculation unit of your choice and not getting kicked off of it, then that's not going to cost $10/month.
It is however one of the prime examples of commercial use of open source software with no engagement of the community. colab notebooks may not run on a “vanilla” Jupyter installation because of google-only extensions to the protocol, and there has not yet been any effort on colab’s part to engage with the jupyter community. You may want to make sure your notebooks and research is not too tied in to one provider.
Colab uses real Jupyter notebooks. You can download and run them in Jupyter just fine, I do it all the time. There are a few proprietary extensions like forms (saved as comments in Python code) or colab only magics, but they’re totally optional and don’t break file format compatibility.
Mathematica's notebook definitely strongly inspired Colab's notebook. Colab is an implementation of the Jupyter notebook format and UI. Jupyter, which launched around 2011, itself was strongly inspired by (1) the IPython console from around 2003, and (2) the Sage Notebook which I launched around 2006.
I can tell you definitively that Sage Notebook is very Mathematica inspired. The IPython console looked a lot like Mathematica, mainly because Fernando Perez (who was a physicist) had used Mathematica a lot and wanted something similar but (much) better. In 2005 there was a project to make an IPython notebook interface as an OS X graphical application, which got demoed at Sage Day 1 (in Feb 2006). That motivated me to get interested in doing something similar, but using Javascript and HTML instead. I hired Alex Clemesha, who just finished his physics undergrad and was a heavy Mathematica user to work on Sage fulltime. He did a lot of work with me during 2006 to create a web-based notebook interface (and also to provide a mathematica-like graphics compatibility layer for Python, which is in Sage). The Sage notebook felt pretty similar in 2007 to what Jupyter notebook feels like, and it definitely inspired the UI. We developed Sage notebook heavily and then all sort of lost interest and moved on to other things (e.g., Jason Grout, who was involved a lot with the Sage notebook went to work at Bloomberg, where he did a massive amount of work on JupyterLab). Fortunately, Fernando Perez and others got incredible grant support and many fantastic engineers together built the Jupyter notebook. Jupyter notebook provided the same sort of cell/output UI as we had with the Sage notebook, but was much more general purpose (many kernels) and used more "modern" implementation techniques, by 2011 standards at least.
There's a lot of amazing things about the Mathematica notebook that we never even tried to implement. For example, Mathematica has a much more sophisticated nested structure. Also, by default Mathematica shares one kernel across multiple notebooks (or at least it did last time I tried it).
Just to finish the story, in 2013 I started CoCalc to make a fully realtime collaborative notebook interface. Around the same time, many other people started another project called JupyterLab that reimplemented a Jupyter notebook client using much more powerful modern approaches. In addition, there's a lot more going on regarding notebook clients these days, including Nteract, Kaggle kernels, and http://deepnote.com/. Some people like me who work on these surely played around a lot with Mathematica notebooks when they were kids :-).
Thank you for that write up! I too started with Mathematica in my early days. I can still play the start up chime in my head. Then I got into econometric softwares which led me to Julia and Jupyter from closely following Prof Sargent.
MathCAD had a WYSIWYG notebook in the same time frame, late 80s, but I have no idea which of them was first with it. It's possible that it was neither.
Mathematica certainly pushed the Notebook format as a fundamental feature, with WYSIWIG math that no one else had ..."Invented" is pretty hard to say honestly, but the popularization and technical polish, absolutely yes
It really makes more sense for actual math than Python deep learning. There isn't a lot of visualization and fancy symbols involved. The fascination with Jupyter notebooks is something I never got. You can't debug or make sense of the control flow like you can in a regular editor. Maybe it's a deep academia thing from all the researchers in this space.
Their notebooks are indeed the best I ever used : so good that I use them for the interface and not the compute power.
I don't need more compute power or longer instances as I use collab with my own hardware (I have a good enough GPU and I don't risk being disconnected), what I need is the ability to run it fully locally with my python environement (+libs) and local data that never leaves my computer (making it effectively and alternative interface for python jupyter notebooks).
I am tempted to sign up. Colab is very usable on Safari for iOS/iPad.
I invested 18 months ago in a GPU setup for home. Really convenient but I somewhat regret the purchase. I used to spin up GCP GPU instanced when needed and that was not convenient. Colab is very convenient.
$10/month for better GPUs and longer sessions seems like a good deal.
I hate to be the boy who cries "Google will cancel this service", but this offering just seems strange.
With a very low price point coupled with not that huge of a user base, this will end up making how much for Google? $1MM/month? $10MM/month? Either would be negligible for them.
Colab Pro is probably the "we want to give verified non-bots more access, but need a small monetary cost to actually stop abuse".
The collection of a credit card isn't just monetary: it also provides some sort of anti-abuse as most people have a limited number of credit cards; and even a bulk credit farm operation has a monetary cost per card (I read the going rate for a hacked card even is $100 ish).
Aged Google Accounts go for $1-2 from account farmers these days.
As a Googler using this internally, I'm pretty sure the project itself will live for quite a while longer. I have no clue about the public offering, but I don't think money is what they are after in this.
Like others said, subscription products are very profitable. The target demographic for this has disposable income and would see this as an investment. A lot of people will sign up and never use it.
Don't forget that the alternative was just free. For all we know this is the way they're breaking even on Colab to prevent shutting it down.
Unlike most other products it won't matter as much here if they cancel it. It's hard to be too dependant on an in-browser stateless tool for prototyping.
You aren't going to be running service off it or even have a permanent set up so if they cancel it you'd have very little to transition from.
Can anyone see a reason why they wouldn’t just allow you to provision (and pay for) a persistent Google Cloud VM instead?
(I currently do that manually and need port forwarding to a machine that runs Jupyter.)
It’s hard for me to understand why Colab would build such a vague pro tier instead of the simplest possible solution: let me pay for my compute.
There’s so much more potential, too; they could offer whole clusters on demand, with really simple Python integrations say using dask, or ray.
Yes, this is how I did this when I had easy access to GCP credit. Have a GPU VM with one of the ready to go images that launched jupyter on boot (or in a tmux session). Then just port forward 8888 over ssh. Worked very well and I found it far more easy to work with than colab.
This is not cheap though. A V100 runs you over $1.5k a month if you don't turn it off.
The Deep Learning VM (available as a native option when creating a new GCE VM) has a bunch of DL tools, and starts a JupyterLab instance on launch: https://cloud.google.com/deep-learning-vm
Perhaps this is more profitable. I’ve heard that Colab has become very popular for people doing certain kinds of deep learning, but that the wait for GPU instances has been frustrating. It seems a paid tier to get priority is directly targeting those users.
I've recently used Colab to run some pretty cool StyleGAN notebooks. Finally I can replicate so many cool projects without bending my head how to setup gcp, virtualbox or install tensorflow into geforce macbook.
Doubt I'll pickup deep learning as a profession by this, but it's a step forward.
I doubt they are making much (if anything) from Colab. I think people have found it very convenient (which it is!) but increasingly complaining about the limitation, so the team probably decided to up the limits and try to cover the costs.
$10/mo doesn't pay for much compute in Google Cloud. Colab Pro is offering GPUs and high memory instances, presumably under the assumption that interactive users will be mostly idle.
Under the assumption that many users on one GPU eventually end up needing more compute and migrating along the easiest cost path to GCE and TPU's, which they earn lots from.
Because they're trying to introduce a different paradigm for computing. Notebooks are portable, purpose-oriented, and allow Google to make decisions about the underlying implementation allowing for a $10 monthly price point and an incredible level of performance not unlike what they've done for BigQuery.
I haven't used colab externally, but internally in Google it is used as a scratchpad for quick and dirty trying out an idea. By the time you realize the idea may lead up to something, you also hit some specific libraries or resources you need to get which are not available in colab. Then you go ahead and build and deploy a proper solution, which I understand what your Cloud VM would lead to.
As such, I don't think there would be a market for quick and dirty scratchpads backed by rock solid but expensive dedicated compute.
You could alternatively use the google AI Platform Notebooks which are basically the same thing and billed per hour. You can now Prebake a docker images with all of the libraries that you need.
Not sure who this is geared towards. People mostly use Colab to share GPU-dependent work from what I can tell. How would that work on a paid subscription? Do others need to pay to run the notebooks you shared? Can they use their "free" account?
As far as utility for research, as a researcher, I _already have_ several local GPUs at my disposal, and I only use notebooks to kick the tires on things and visualize. The moment something starts to look like it's useful, I move it to a real *.py file where it's more maintainable and diffable.
Edit: actually I now think I know who this is geared towards. It's geared towards people who aren't going to really use it, and don't mind to pay $120/yr (+tax) for something they don't use. Which, IMO, is pretty smart.
A preemptible P100 + VM on Google Compute Engine is about ~$0.45/hr, so to exceed that value with Colaboratory Pro (ignoring conveience factors) you'd need to train for more than 22 hours in a month. Which, for deep learning, is not too unreasonable.
Reading between the lines of both the signup page and up-to-date FAQ, it seems like the free TPU in Colab notebooks will be depreciated, which isn't too surprising.
Sorry about that. We (I) had screwed up and allowed users to smash themselves into the wall of "Huh, you keep getting preempted, but there's technically a slot right here". We've re-enabled some backpressure to keep you from getting hit over and over again, and instead see a stockout error when you should probably try another (less full) zone.
I killed a lot of time with this exact issue a couple months ago as well. Quite frustrating, and it was even worse trying to include the preemptible GPUs in a kubernetes cluster. It would often take so long to add the GPU to the cluster that it would get killed before it even ran any real workload. This cycle would repeat for an hour before a machine stayed up long enough to actually schedule kubernetes pods on them.
It would be nice of GCP didn't bill for pre-emptible instances that got system killed within 10 minutes of starting; or something reasonable like that.
I doubt it makes a material difference to their (or our) bottom line, but it's just a nice gesture and makes you never feel like "damn, I literally wasted money".
While this is pretty frustrating and seemed to be the result of a bug, I would personally only use preemptible instances are when I have everything scripted, and save the state of my work when you get a termination signal.
To me, they're not really sold as "cheaper, just not 100% guaranteed"; they're sold as "cheap, it will be killed at any moments notice so".
I haven't in awhile; when I used them more heavily, I got the full 24 hours. It's possible that it's region-dependent; IIRC I had better luck with us-east1 than us-central1.
The page said "Longer running notebooks and fewer idle timeouts mean you disconnect less often." I'm guessing there is a limit but it has been increased for the pro customers. In contrast on GCE you can train for 22 hours if you want.
Have model training make checkpoints every five minutes. When the training gets interrupted, just restart it on a new instance from the last checkpoint. Repeat as necessary until hundreds of hours of training is completed.
Cloud storage bucket works better than Google Drive for both training data and checkpoints; and if you're doing training on TPUs, then I believe that you must do the data exchange via cloud storage.
I wonder who made the decision to spin this out into a commercial product; maybe it has to do with Google's push into the cloud further? I always thought Colab was just an experimental tool; it's still under the research.google domain.
I've been using Colab for over a year now. I train deep learning models on NLP and medical imaging datasets.
It's a great tool and it lets you focus on the code and the models, instead of the hardware and OS. But $9.99/month is a little expensive for my taste.
You can't customize it and if they change something you have to install software by hand sometimes. It should be $1.99/month, that's the kind of price I'd pay for this basic cloud computing service.
edit: I use Colab to play with ML models. I really don't think it's possible, for instance, to train a model on Imagenet using Colab. So Colab is similar to the microwave, if you want to cook a serious recipe you should use a real kitchen.
Not to disagree, but P100 is not a "modern" GPU. It's based on Pascal, so it's a couple of generations behind. Throughput-wise it's roughly equivalent to 1080ti. Which is still something like $600, so your point stands.
in medical research you are saying $10 / month is too much.. this might be a study in consumer psychology! Here in California, biotech secretaries drive a mercedes...
Curious to how you've been training with Colab. I tried running 50 epochs on a UNet3D model, each taking about 20 minutes to run; it was a pain in the ass because the session kept disconnecting.
A lot of comments are missing the value here: cheap and easy TPU access for hobbyist use of deep learning models that need TPUs for fine-tuning and/or inference (GPT-2, I’m looking at you).
Probably because it's easier to buy something that costs $10/month instead of "maybe it will be $0, maybe it will be $300,000, we'll see at the end of the month!"
When you're using your own money to pay for cloud resources, that unbounded worst case is pretty scary.
But you have to admit that if you're just some scientist who wants to GPU-accelerate their Python notebook, "click here to pay $9.99" is a lot better UX than "just log into the Google Cloud Console and change 83 settings!"
There's so much data in this universe, people don't know what to do with it. When people don't know what to do, an industry grows to let them "feel" they are doing something useful.
Some of the key differences:
- Faster storage. Colab uses Google Drive which is convenient to use but very slow. For example, training datasets often contain a large amount of small files (eg 50k images in the sample TensorFlow and PyTorch datasets). Colab will start to crawl when it tries to ingest these files which is a really standard workflow for ML/DL. It's great for toy projects eg training MNIST but not for training more interesting models that are popular in the research/professional communities today.
- Notebooks are fully persistent. With Colab, you need to re-install everything every time you start your Notebook.
- Colab instances can be shutdown (preempted) in the middle of a session leading to potential loss of work. Gradient will guarantee the entire session.
- Gradient offers the ability to add more storage and higher-end dedicated GPUs from the same environment. If you want to train a more sophisticated model that requires say a day or two of training and maybe a 1TB dataset, that's all possible. You could even use the 1-click deploy option to make your model available as an API endpoint. The free GPU tier is just an entrypoint into a full production-ready ML pipeline. With Colab, you would need to take your model somewhere else to accomplish these more advanced tasks.
- A large repository of ML templates that include all the major frameworks eg the obvious TensorFlow and PyTorch but also MXNet, Chainer, CNTK, etc. Gradient also includes a public datasets repository with a growing list of common datasets freely available to use in your projects.
Those are the main pieces but happy to elaborate on any of this or other questions!
[0] https://gradient.paperspace.com