I love JupyterHub, but we've hit some real headaches in having it scale. At Virginia Tech, we have an introductory course for non-computing majors where students are using Jupyter through JH. At around the 70-student mark, we have performance issues. Considering that the course is eventually meant to scale much farther (hundreds of students), we're not really sure how we can make further progress with our current resources. I hope this new version has some performance enhancements (though I don't see any in the changelog). Last I talked with anyone about this was in the NBGrader project[1], where other schools were hitting walls with scaling.
Give us some details about your setup. Are you running JH on the cloud somewhere? Or is this running on a single box at VT? We’ve got JH running on a Kube cluster with persistent storage on the NSF Jetstream cloud [1] (which you could qualify for free resources being from VT) . This set up should theoretically address the scalability concerns you are having. See the work Andrea Zonca and I have been doing [2, 3, 4, 5]. Contact me for additional details.
I'm not as up on the details as you'd probably want, but my understanding is that we're running it on a virtual server with a nice chunk of RAM and CPU locally at VT. I don't think we want to be at all reliant on external servers - this is FERPA protected data. Plus, the long term goal was to find a solution that other schools could adopt without being an R1.
Are you running into scaling issues because all students are on a single box? I think the JupyterHub proxying service should be able to handle way more than 70 users, but I could see 70 users on a single machine being problematic. I'm doing a lot of work around JupyterHub - happy to help out if I can, I can be reached at hugo@saturncloud.io
Well, it's a virtual server, so I'm not sure how that plays with things. I supposed getting more servers in play would help, but that feels more like throwing hardware at a software problem. It doesn't feel like JH should have this much overhead - is the kernel really doing so much work?
Depends on what your students are doing right? Also I'd bet your running out of ram before CPU. You should try doing the same workload without JH involved - I'd bet you'd still run out of ram. Also - I'd throw hardware at software problems all day long if I could. Hardware is cheap compared to developer time
When would someone use jupyterhub? I've been running my own notebook server for years, but it's single-user, single machine. Is hub for like providing separate jupyterlab instances for a bunch of different users/different machines?
We have it setup at work, as an alternative way to use our HPC resource compared to the traditional Linux shell + slurm usage.
User goes with the web browser to our jupyterhub URL, logs in with our usual credentials, selects a job type (amount of memory and max duration), and jupyterhub takes care of launching a jupyter kernel as a slurm batch job on a compute node in the cluster, and proxies http I/O via the jupyterhub node to the user web browser. In the jupyter notebook, users have access to the same cluster filesystems as if she would login traditionally via ssh.
Yes exactly. At my work when a new scientist joins us we just create an account and she can get started on her research within minutes. Each user gets a contained environment in which we mount a disk of shared data.
I just did this in my University lab as well. Most people aren't savvy with Linux, so having normal accounts with Jupyter port forwarding is out of the question. JupyterHub is just about the lowest friction I can possibly make it for introducing the Python data science stack to non data scientists.
IMO, Jupyter Notebook is the closest equivalent to Python as what Rstudio is for R. While Pycharm and VSCode are also preferred by some Py-based Data scientists, Jupyterhub offers almost everything that a typical IDE would do along with the traditional Notebook environment which a lot of beginners these days start with. Thus much less friction while getting started.
I would be really hesitant to comapre Jupyter Notebook to an IDE.....an example is a debugger...the only visual debugger that i have come across for jupyter is pixie debugger, which is miles behind the debugger of an IDE like Pycharm.... there is a huge list of features that jupyter needs before you can compare it to an IDE
FWIW I use the %debug magic command in Jupyter and it has been a great experience. I'm pretty ignorant of the enterprise debugging tools so take that with a grain of salt.
Debuggers are only really useful if you're trying to figure out why some object in your server doesn't do what you want it to.
I'd wager that almost no data scientists write object oriented code.. it's probably mostly done one calculation at a time. executed in the notebooks repl. So the value you get from ide debuggers is tiny, as you're already doing everything one step at a time.
Correct. RStudio has this feature, where variable values can be inspected in a sidebar. This would be a really useful feature for Jupyter, especially when running a Python kernel.
Does it work with variables that are local to a function? I don't mean inspecting global variables after having executed a cell, but local variables in the middle of a function execution.
My school uses this for its data science classes so let me provide an example. Say that you have a lab or homework assignment done in Python. It may require a lot of storage (a large dataset), compute (maybe you have a large model), or dependencies. JupyterHub is one way to setup once and then deploy to multiple users regardless of their computer.
I give talks in math classes about how math is used in the real world, and I've been looking at JupyterHub as a way to give everyone in the class a way to work through an activity with instructions & equations, diagrams, places to enter their own data, and run a little mathematical program, etc.
This is really cool and I’m impressed by the jupyter team. My favorite part is that it’s such a good product that beats the commercial products because it’s hard to figure out, I think, commercial models that support this wide range of collaborators (people who view once a month to people who author every day).
I was trying to read about whether jupyterhub is included in RStudio Connect, or if they are competing products.
Thanks. That’s why I’m trying to figure out if it’s actually jupyterhub under the covers so they’ll get these new features. Or if they are competing and RStudio will have something similar or I have to check their dev schedule.
Congratulations!
JupyterHub is a great project with high quality code and docs. Looking forward to try the named servers feature as I run a JupyterHub instance that spawns servers inside containers based on a single image which inevitably tends to grow as I add libraries. Being able to manage multiple servers should allow me to split the image into smaller specialized images.
I once tried running jupyter inside docker...the downside i saw was conda install of jupyter is around 1.2 gb, making the container pretty heavy...if you are fine with this then all thats left is to expose the port to the container on which you want your jupyter server to run and changing some config after generating jupyter_config file for remote access.
Hi, I’ve been working on setting up JupyterHub on a DGX-1 for the past week. It’s straightforward, you run JupyterHub on your machine and you have it launch servers for each user with nvidia-docker.
At work we use conda which works great, though nvidia libraries are usually installed by admins system wide. But for anything python related, conda is perfect.
Thanks for the answer :)
what is the advantages of using conda vs installing the packages by hand? Tried to document my self a bit before but I still dont know the real advantages in a modern Linux system
Edit: Also, there is a distribution for Windows that’s handy when your employer has you using Windows. And, depending on your software approval process, it can be convenient to get one package approved (Anaconda) instead of every package Anaconda includes.
Conda gives you two real advantages: the first is straightforward virtual environments (you could also use venv) and secondly better dependency management than pip in some cases.
There are certain packages which aren't available on pip that conda-forge provides e.g. for a while OpenCV wasn't available through pypi.
Remember you can use pip within conda, and you can install directly (e.g. setup.py) within a conda env.
Docker is mostly useful if you want to mix and match different versions of CUDA/cudnn etc. If you just want to run an isolated Python environment, then Conda will do the job.
Conda is "just" a distribution and package management tool. As a beginner it is a good idea to just go with it, but installing all packages using pip is just as good, in the end. For reference: https://www.xkcd.com/1987/
In bioinformatics there is a trend for systems like qiime2 to be basically impossible to install via pip and to have conda and docker be the only options. In part this is because rarely do bioinformatics pipelines rely only on python but on a multitude of existing programs that need to be installed.
Jupyter Notebooks / JupyterHub and BinderHub are, in my humble opinion, most relevant for the future of education, as a tool for teaching, for science, as a format for replication, and for everything in between, see the data sciences. And even more!
First of all, AWS SageMaker is really a ML system that happens to include Jupyter notebooks as a component. But if you are talking about just the Jupyter notebook part, then I would say - you could use JupyterHub to build your own implementation of SageMaker (you would want to use kubespawner and some deployment of kubernetes if you wanted to scale to multiple nodes). For example, I run https://www.saturncloud.io/, and we orchestrate JupyterHub to do just that.
JupyterHub is more flexible - for example, you could deploy JupyterHub to one beefy server and have Jupyter deployed for many users, which could all read data from a shared filesystem. that kind of thing is not easy to do with SageMaker since everything runs on a separate ec2 instance.
[1] https://github.com/jupyter/nbgrader/issues/530#issuecomment-...