This looks very cool. There's no overview of how it actually works. It seems to launch a goroutine for each Jupyter code cell but any variables defined are also available for use in later cells so I'm not sure how that works.
Just playing with the code in the mybinder.org it feels responsive enough. As well as for data analysis, this will be a great way to do those quick checks that I like to do in python to confirm how an api works. I've been meaning to start using golang more and this might be how I do it.
As someone who hasn't worked with Jupyter beyond a simple iPython shell, this comment sent me down a bit of a rabbit hole. If anyone's interested, here are some links.
This is super cool, but I’d like to know more. You can’t reload plugins without changing the plugin path and any shared references won’t get GC’d, from what I remember. I tried to do server handler hot reloading via plugins and ran into this a while ago.
For information, it's not the only Go kernel for Jupyter.
There is also https://github.com/gopherdata/gophernotes which uses a different approach: contains a portable Go interpreter, instead of invoking the Go compiler for each Jupyter cell as lgo does (lgo is currently linux only)
I am gomacro author's (gophernotes uses gomacro as Go interpreter) and I find the comparison needs an update.
Among other things, gomacro now supports interfaces - although gophernotes still has to catch up.
It's also partially incorrect: gophernotes and gomacro are type safe
I guess I will have to open an issue...
Also, IMHO the comparison criteria are chosen based on what lgo author considers important - not necessarily what's important for someone else.
Some quick examples that are important for me: portability on non-Linux systems, redefining a variable or function (lgo can only shadow them), redefining a method, allowing unused local variables...
I have great respect for what yunabe accomplished with lgo, including the deep integration with Jupyter (displaying HTML, Javascript, images...), code completion, and many other features, but I think his comparison with gophernotes needs to be updated.
Serious question: what's the point? Is there anything in the realm of interactive data analysis-type programming that isn't already better with Python and instead needs to be done in Go? I might be missing something. If so, what?
Jupyter has uses outside of data science; I find it to be an extremely useful programming scratchpad for web development with Python and Node. It was renamed from IPython to Jupyter to reflect its broader scope, Python was the original kernel but there are many now:
One thing I was wondering how did you manage to build it? There's a lot of code there and I was wondering if I could make my own version for a different programming language.
> I don't get why you want to use this over an IDE. All you get is cached intermediate variables and being able to print them, am I right?
You get more than that from Jupyter. Especially with Python, but for Go also.
Jupyter is for exploration and prototyping. Think of Jupyter as being a massive improvement on the command line shell, rather than an IDE downgrade. If you're developing large structured programs, yes the IDE is definitely the way to go. But if you're doing interactive coding and you want to run the code line by line or in small groups, you'd use the shell, or better yet Jupyter. Jupyter is between an IDE and a shell.
Jupyter gives you inline images & plots. It's made with visual results and interactive plotting in mind. Most IDE's don't have anything like that. You can also compare Jupyter to Maple or Matlab or Mathematica.
Jupyter lets you add readable formatted markdown between code blocks, and you can export your notebook in presentation formats like PDF or HTML.
Notebooks show the results of the program run, so someone you share with can see what happened before they run the code.
And I don't know about the Go kernel, but with Python at least, Jupyter most often installs a sandbox environment that includes several powerful Python libraries. What this means in practice is that sharing notebooks is vastly easier for the recipient of a notebook.
1. That's very misleading, because what PyCharm did was add support for jupyter notebooks. Without iPython/Jupyter there would be no "interactivity and cell based execution" in PyCharm.
2. And AFAIK it's just Python, does it support the few dozens other jupyter kernels?
1. No, it's not. In regular python files you can create cells using #%% and run them separately. Yes, there is also notebook support but it requires Jupyter.
Jupyter serves a different purpose than IDEs and text editors. The main use cases are experimentation and presentation.
If you want to try a library it can be more flexible than a REPL and more interactive than a program in its own file.
Jupyter is also used to produce documents that mix code, prose and visualizations. A typical use case is to write a small amount of code that processes some data and produce a chart, providing explanation formatted in Markdown along the way. These documents can be exported in a variety of formats, including printable documents, presentation slides and blog posts. There's a very vibrant and high-quality ecosystem of projects developed around Jupyter which ranges from making interactive documents to running distributed notebook servers.
In general you wouldn't develop a full software project in Jupyter.
Note that Jupyter is not the only project that provides this type of notebook interface. You'll also find Zeppelin coming from the Java community, Observable for JavaScript and the very powerful Nextjournal SasS product.
I love to use notebook style IDEs to prototype, test and benchmark "smaller" things. And with smaller I don't mean things that aren't complex or difficult, but things that don't need thousands of lines of code or ellaborate software architecture.
They are great to try new algorithms or evaluate their performance. If something is not quite right the first time, just change a line and evaluate that block again, without having to run another, potentially expensive block, again.
You get to see the changes to your latest block immediately, without having to go through all the previous steps again.
They are also great if you want to put together some graphs or statistic displays.
I'm still hoping for something like Mathematica for javascript, including ways to print graphs, meshes, point clouds, etc. to you notebook.
well in the context of python it makes some sense to use it for teaching and presentation purposes, even tho it's a bad habit to program in it in my opinion, as
- it's a pain to write abstractions, as you have to remember to reevaluate the cell
- you can't use the debugger
- you can't jump to the definition of externally defined stuff, which is critical if you want to become a really good developer, as it's teaching you to read and understand code
@mschuetz there are decorators for everything, to time functions, to cache results, to cache results, to cache results for 1h and get fresh data after one hour, to cache results given certain parameters
but in the context of go I don't see the point, as nobody sane will ever do an experimental data analysis with it
go is excellent for high performance data processing, but you definitely don't want to use it for data analysis.. I'm not even sure if a go based dataframe implementation would be that much faster than pandas as it's more or less a wrapper for c functions
Do you do any data analysis? Have you used any of the tools for Go like gonum? Notebooks are great for presentation of the data, and how you got where you did. They display the plots and graphs in line for you. You can add markdown for extra documentation/explanation.
Sure you aren't making production grade models with the notebook, but they are great for rapid prototyping or presentation about what your models are doing.
haha yeah I do, in python only tho.. I use plotly for visualisations. It renders html and invokes the default handler for .html to display it. Knowing plotly it's trivial to learn dash, which let's you build interactive data dashboards (https://dash-stock-tickers.plot.ly).
> Have you used any of the tools for Go like gonum?
well as far as I now, there are barely any data analysis libraries for go. gonum is an advanced math library, not really a data analysis tool. then you got gota, which is unmaintained (at least they aren't closing the prs) and a couple of none standardized learning algorithms.
not really the stack I want to use for data analysis..
Fair enough. I do think this is a step in the right direction. I know on the slack there are plenty of people who want to make Go into a language you can use for Data Analysis/Science. It might just take a little more time for it to happen since it is relatively new to Go.
I believe it's intended for teaching and guides, not development. For example, I'm currently working through a data science course which uses Jupyter notebooks to allow interactive code to be embedded within the course materials, instead of being kept in separate files. The student can read the material and experiment with the code without switching to another window.
I just don't see Go ever becoming a good language for numerical computing. There was such a push with implementing [multidimensional arrays in Go](https://github.com/golang/go/issues/6282) but it looks like it's been shot down more than once.
When using Python and R interactively, the syntactic sugar and functions that allow for rapid analysis (subsetting, indexing, merging data objects etc etc) more than make up for the slowness of the language. If someone wanted to implement that workflow in Go for repeated analysis of very large datasets, I doubt they would need it to be interactive.
But it looks like Julia might become the goto numerical language that is compiled and fast.
If this were possible, it would require the blackest of black magic, and mapping anything other than primitive types across the void would be essentially impossible, e.g. anything useful like a DataFrame.
One could implement a kernel that forwards calls to other kernels depending on a prefix and explicit push-and-pull functions to move variables around. That last part would require an extension of the protocol or a parallel channel, but the forwarding kernel is actually not that difficult to implement.
Just playing with the code in the mybinder.org it feels responsive enough. As well as for data analysis, this will be a great way to do those quick checks that I like to do in python to confirm how an api works. I've been meaning to start using golang more and this might be how I do it.