Hacker News new | past | comments | ask | show | jobs | submit login
REPL-Driven Development (2017) (github.com/matthiasn)
164 points by kimi on Feb 6, 2018 | hide | past | favorite | 72 comments



In Smalltalk, the class browser (editor) has always been the REPL. You can highlight any snippet of code you browse, including code you're currently editing, and evaluate it in the context of whatever class you're browsing.

For example, if you're editing a method called "fooBar", you can just type (in the same input field) "self new fooBar" ("self" is bound to the class being browsed) or, if it's a class method, "self fooBar", highlight it, and then type the appropriate shortcut (like Cmd+d in Pharo) to evaluate it.

This is useful for documentation. You can have a method begin with a comment that shows examples, which a person browsing can highlight and evaluate, and also experiment with.


But that's not really a REPL. It lacks the 'L' and often also the 'P'. Running code from a text buffer in an editor is one way to interact (many Lisp tools do that), but a REPL is usually a tool and its purpose is to read expressions, evaluate them and print the results. I would be surprised, if Smalltalk did not have that somewhere, too. The REPL has a visible I/O history and is not conflated with other text input fields (like a function being edited).


> But that's not really a REPL. It lacks the 'L' and often also the 'P'.

This objection is subtle. The "P" in REPL of course does not simply mean to emit some printed representation of the result of the last evaluation, but to emit a specific representation that a corresponding operation, (read), can comprehend.

In that sense, no non-Lisp, including Pharo, has a true "P" and a true REPL. (Though many objects in Pharo do have such printed representations).

> Running code from a text buffer in an editor is one way to interact (many Lisp tools do that),

Like Emacs and SLIME. But having used both, I prefer the Smalltalk browser-based IDE approach. The browser makes code navigation much easier, and it saves you from having to type out a lot of class and method creation boilerplate code like the kind you end up writing when using Emacs and CLOS. Maybe LispWorks (which I've never used) is that kind of IDE, but for CL?

> I would be surprised, if Smalltalk did not have that somewhere, too. The REPL has a visible I/O history and is not conflated with other text input fields (like a function being edited).

Repeatedly print-evaluating (Cmd+P in Pharo) code in a workspace/playground is actually the closest I think most Smalltalks come. A command prompt-style REPL isn't as useful in a fully graphical environment. You could easily create one if you wanted, though, and text-based Smalltalks like Gnu and Amber have one.

However, the inspector in Pharo can function as a nice graphical REPL. You can enter code in an inspector pane, evaluate it with Cmd+G, and the result will appear in a new inspector pane to the right of the current, and you can cycle back and forth through evaluation panes.


> but to emit a specific representation that a corresponding operation, (read), can comprehend.

On the Symbolics Lisp Machine they then added a facility where READ can read arbitrary things, since each thing you PRINT has a printed representation/view, which stays connected with the original object. Thus you can enter non-text things or non-printable things in a REPL.

> Maybe LispWorks (which I've never used) is that kind of IDE, but for CL?

Not really, it is not browser based in the sense of Smalltalk. It has browsers for things like classes, but you do no in-place editing.

Apple Dylan or Interlisp-D were that way. Interlisp-D is a similar managed source system like Smalltalk, but with Lisp representations and structure editing.

> However, the inspector in Pharo can function as a nice graphical REPL. You can enter code in an inspector pane, evaluate it with Cmd+G, and the result will appear in a new inspector pane to the right of the current, and you can cycle back and forth through evaluation panes.

LispWorks has something where you have a REPL and an inspector in a window. Each evaluation result in the REPL goes into the inspector and you can cycle between inspected values back and forth.

One can also link windows in LispWorks, for example a REPL with an inspector.


The 'P' is just another entry on the Smalltalk menu options.

One can choose to evaluate, or evaluate and print.

The 'L' is the whole Smalltalk environment, as anything that changes the environment is kept in memory or saved along the image on exit.

A Lisp like REPL is available as Transcript windows.


The output goes into the text field where I'm editing. That's not a REPL. Usually printing output to a random text field makes no sense. It can make sense to execute commented code snippets in code, or evaluate code and have the result being inserted.

A transcript window is usually a text output window, more like a console output. I'd say the Workspace window is slightly similar to a REPL. But it still does not work like the usual Lisp REPL, where you type to the tool usually textually at the bottom and get the result displayed there. With Smalltalk I usually type to an editor Workspace window and REPL I/O does not need to have a sequential order.


You're right Workspace window not Transcript, I always mix the name of them.

Still, it looks pretty much to the Lisp Machine demos I know of.


Not really, for example the Symbolics Listener works completely different.


Programming Clojure applications with CIDER[0] is a really slick experience. By connecting through an nREPL port to a running environment you can capture variable values like function inputs, change and re-evaluate function definitions on the fly, and test hypotheses by calling funcs directly with whatever input you want. Not to mention the speed of prototype iteration. This has been invaluable when trying to quickly track down production bugs.

I've lately been working in Python, and even with IPython/Juptyer the workflow isn't quite as magical as Clojure's.

[0]: https://github.com/clojure-emacs/cider


If you're not into Emacs, IntelliJ has a sick integration that works through a nREPL called Cursive that is a joy to use https://cursive-ide.com/


Fellow Cursive user here. Being able to hit a key, in my case Shift + CMD + P while the cursor is on some text and to see it live in the REPL is pretty incredible.

It’s also kind of weird that a lot of other languages do not have this


As he rarely asks for donations, I feel it only appropriate to mention the author of Cider (& Rubocop for Ruby people) has a patreon if anyone is feeling generous & feels the above (free) tools adds value in their lives: https://www.patreon.com/bbatsov

Sadly, it's only at $31 per month


But does it allow you to examine stack and local values when an exception occurs, I wasn't able to do so last time I tried a few months ago.

Also the cider environment is kind of brittle with seemingly lots of components, all of whose versions needs up to align for the magic to work.

And another issue with clojure and leiningen is the almost impossiblity of using local jars as dependencies, without setting up local maven repositories. Too much work for what I was doing for fun.


I've been working with Clojure a lot and some years already, but only a couple of months ago really started working with CIDER. It makes everything so much nicer and it feels like the way Clojure is meant to be developed. I had some grudges about leiningen and the startup speed, but having a CIDER connection, tests ready to run immediately and a REPL where I can test my code in a buffer, writing Clojure is such a joy.


Might be worth mentioning gumshoe[0] as well, which is fantastic. I use it to capture bindings without thinking about it (using the automatic plugin for development mode - no overhead for production!).

Often times I run a long-ish/effectful function (web request, network request, etc.) and wonder what the value was somewhere in the function, but unfortunately I didn't have the foresight to do a (def -my-value value) ahead of time.

With gumshoe, it's all automatically captured, so I can pop the values into my repl and play with them to understand what happened.

A few examples:

  user> (deft add-ten [num]
          (+ num 10))
  #'user/add-ten
  user> (add-ten 2)
  12
  user> -add-ten-num
  2*

  user> (deft destructure-example [{:keys [a b] :as args}]
          a)
  #'user/destructure-example
  user> -destructure-example-a
  1
  user> -destructure-example-b
  nil
  user> -destructure-example-args
  {:a 1, :c 2}
Anyway, it's quite surprising how often this saves my bacon. And it pairs very nicely with cider indeed!

[0] https://github.com/datodev/gumshoe#gumshoe


Re Python, PyCharm is one of the most slick experiences, supporting everything you mentioned.

That said I find IPython supports everything you’d want to do in Python save multithreaded profiling for which Yuppi is useful.


Just a reminder to younger crowd Visual Studio had all this before (packaged in single app hence well-integrated and easy to install) probably for 15 years or longer.


Are you referring to the VS debugger and msvsmon? The VS debugger is an excellent tool, but stopping the world each step is different than how the Clojure REPL setup described by the parent works.

Did/do people really develop applications interactively in VS other than in F# Interactive? I've been using VS a long time for VB6 and C#, only in edit/compile/execute-debug mode.


Interactive Lisp development with Emacs has been around longer than that. The first commit for SLIME, which is a (third-generation, I think) Lisp development environment for emacs, is from September 2003: https://github.com/slime/slime/commit/2fb4c4e53b59ed7b6e5f55... (and the comment implies that there was development before then).

Visual Studio has been around since 1997 (which is pretty cool!). Emacs has been around since 1976, although I don't know how far back it got a decent Lisp development environment. Early 80s, maybe?


Interactive REPL in Lisp: early 60s.

Interlisp development environment: early 70s.

Interlisp-D: Early 80s.

MIT Lisp Machine + Zmacs: end 70s.

Macintosh Common Lisp, LispWorks, Allegro CL, and many others ...: mid-end 80s.

GNU Emacs based Lisp development: end 80s.


Are you trying to say the "younger crowd" is on Emacs and doesn't know about Visual Studio?


Anecdata: Am in my twenties, use Emacs, haven't used VS. :p


Given the phenomenon of Lisp Cycles[0], it's entirely possible.

--

[0] - https://xkcd.com/297/


For a language with a very good REPL environment, try Factor. :)

But there are a lot of kinks with REPL:s that the talk doesn't detail. Most of them have to do with code reloading and all the corner cases that can lead you into. For example, suppose you have started a thread which calls the function "foo" from within a while loop. Then you recompile "foo." Now, what should happen with the thread? Should it switch to the new foo or keep calling the old one? What if you remove the foo function from the module in which it was defined? What if you have a class and you change its definition. What happens with old instances? What if you recompile the REPL? Recompile the compiler?

Solving issues like the above is really, really hard. Especially if the solution has to be both efficient and also work as the user would expect.


Clojure has dealt with this problem actually. If you are curious check out mount, component, and this blog post: http://thinkrelevance.com/blog/2013/06/04/clojure-workflow-r...


As a Python and JavaScript developer I live in Jupyter and the Firefox/Chrome devtools (and I'm starting to get into Observable). I genuinely don't know how to be productive in languages that don't have a great REPL.

I think REPLs might be why I've never really got excited about IDEs. When you're working in a REPL you can answer questions about your code instantly without needing smart autocomplete or context-sensitive help.


I do a good combination of data science and development.

REPLs are nice, don't get me wrong, but they are by nature single entrance, single exit. You are basically riding shotgun on a single unit test. You get a great view along the way, but the problem solving doesn't scale, nor can it handle situations where you need to consider multiple paths through the code. Writing a good test suite, and knowing just where and how to place debug trace statements are two of my best problem solving skills.


F# hits a really nice sweet spot for this: scripts with an integrated interactive scripting environment, and complete access to a plethora of unit testing frameworks.

That lets you work line-for-line through your code, inspect and analyze each step as needed, edit parameters as wanted, restart and do clean runs through calculations using custom and third party libraries, and also interactively execute any tests/specs you have defined. It's a very tight feedback loop that doesn't compromise any of the long term artifacts you might want in a serious app.

In conjunction with .Nets extension mechanisms and some F# language features the setup lets you fire up a basic script, replace/extend anything from your object graph with new code, write accordant tests, and execute any or all parts of the test as desired in an interactive REPL-like session for problem solving and debugging: all in the same file :)

This flows really nicely with the single pass compiler, keeping things nice and clean so they integrate into the main app once exploratory development is done. Semi-serious typing, functional composition, and dynamic exploration -- it ain't bad.


Once you achieve the right result from the function you are writing, you just add that REPL invocation to your test suite. Then you just re-run your tests from time to time as you change the code further.

A REPL makes writing both code and tests easier.


This is exactly my flow. Get the small snippet/function working in the REPL, copy it into the source....move on.


>You are basically riding on a single unit test. You get a great view along the way, but the problem solving doesn't scale, nor can it handle situations where you need to consider multiple paths through the code.

Where a REPL starts to fail, I find integration tests start to shine and vice versa.


I see it along the lines of REPLs being helpful for prototyping code. TDD, strong static type systems and proper separation of concerns let you build larger maintainable systems.


A good REPL keeps a history. It happens frequently enough that dumping that history will give you a reasonable basis for a unit test. :)


Indeed, that's partly what my own testing framework (that I wrote and use on my own private projects) takes advantage of, except in interactive use.

The Common Lisp REPL automatically binds the last expression evaluated to the variable "+". It also automatically binds the value returned from the evaluation of the last expression to the variable "*".

I write code and develop interactively at the REPL with the package under development loaded/running. When the value I expect gets returned from the function or form I'm developing at the REPL, i just evaluate the "test" function (or a variant thereof if extra specificity or features are wanted such as names, doc-strings or tags, etc.), and the forms and expected values are packaged into a test and added to the projects' test repository.

Pretty much all the benefits of TDD, without writing tests up-front or wanting to do red/green (i did allow for the option, but i practically never do in practice), because they're an almost costless artifact of just developing now...


Yes, IPython supports something like that with the %save magic command, IIRC.


REPL are obviously a poor substitute for unit tests; but a unit test is a poor substitute for a REPL.


I happily use both. REPLs are for exploratory development. Unit tests are for ensuring code works (and continues to work in the future).


I disagree.

I have my IDE setup with some scripts to copy a database from production. Set some breakpoints and press the play button and I am debugging.

With A REPL first I need to do a load of import statements. Then populate my objects. Then get started with the actual debugging. I personally don't find it anywhere near as efficient as an IDE that has been setup. Plus with the IDE I can stop at a certain point in the code, have numerous variables populated then run "evaluate" in PyCharm, which gives most of the benefits of a REPL as far as I can tell.

Admittedly the IDE does take a fair bit more effort to get to a state where it is set up to be useful, but dealing with the whole system as opposed to a small part of it is incredibly beneficial for development. Though I guess a lot depends on the type of problem you are solving.


>With A REPL first I need to do a load of import statements. Then populate my objects. Then get started with the actual debugging.

No, you absolutely do not. You just need to click around until your code hits the point in the code (or, ideally, run an integration test that gets you to that point) where you want the ability to run methods, inspect objects, etc.

Just make sure you've shoved an import IPython ; IPython.embed() in there and you're good to go. No additional imports necessary.

I agree that having your environment set up with production data (or as close an approximation as you can get) is a good idea, but I'd rather script that with a test framework where it can be used for regression testing rather than bodge it together using an IDE.


"You just need to click around until your code hits the point in the code (or, ideally, run an integration test that gets you to that point) where you want the ability to run methods, inspect objects, etc."

This is generally a lot less effort than populating objects in the REPL to reproduce the same set of conditions.


That's basically an admission that you're not writing enough realistic tests or your dev environment is set up wrong.

For me, it's way way easier the other way around.


Ahh the classic "you're doing it wrong" reply with no advice on why it is wrong.


Do you need advice on why creating a code base not surrounded by realistic & reliable tests is a suboptimal approach?

Or did you think that once they're there it would still be a lot of effort to use them to populate objects?


Doesn't mean that its going to catch the bugs that are present. An IDE when properly setup is closer to a production system than a load of unit tests and as such will be able to fix far more bugs in my opinion.

But you are welcome to do what works for you. Maybe you work on something more suited to your approach. My approach works better for what I do.


I write integration tests, as I mentioned in the first post and realistic ones as mentioned in the second. I try not to write unit tests because, as you mentioned, they are unrealistic (among other reasons), and unrealistic tests tend not to catch bugs.

That said, a high level 'unit test' (e.g. one that manufactures a request and checks its response) is often realistic enough and works pretty well with IPython.embed().


> I have my IDE setup with some scripts to copy a database from production.

Can't you just run those scripts as the first thing in your REPL session?


That just copies the data, its doesn't do imports or initial fetching of data - which running the application in and IDE does for me.

Having an IDE setup is pretty essential to my job, so I am going to need that for debugging certain errors anyway. Once that is setup, I rarely see the need to start scripting things for a REPL as a debugger feels far more useful to me.


The "Open Debug Prompt" in Pycharm is the best of both. Breakpoint at the point of interest, enter the REPL, and continue writing at the prompt. Also good in Matlab. I hope I can see this feature in Rust someday, although it seems like a tall order for a non-interpreted language.


I have used python javascript and clojure and let me tell you that repl driven development is not exactly what you get with python and javascript. The most frustraiting thing after having used a lisp is the fact that you can not connect to your program while it's executing. In node for example i wish i could just connect to the running server and test some functions with the current program state, like database conections etc. In clojure you can do that from your editor and that is awesome it's really powerfull. You can just point to a function and execute it inside the running whole.


Node has a built in debugger that allows you to attach to a running process. You can even use chrome dev tools for debugging it.


In Python the built-in debugger can't attach to a running process, but it's just a matter of installing one.


Why Jupyter? Do you mean iPython? Jupyter isn't a REPL. I find doing actual development in Jupyter to be impossible. It's OK to write the top-level orchestration script, but for writing functions (which is what using a REPL is all about) it's just horrible.


I think you're confusing the old and new naming scheme. The old iPython project split back in 2015 into Jupyter and iPython. Jupyter is the name of the language agnostic part of project that develops the notebook, web apps, repl and messaging protocol parts. iPython is the kernel that provides python support for Jupyter. However since python is by far the most popular language used by Jupyter, people generally just say Jupyter when just say Jupyter/iPython.


No, the REPL is still part of iPython. Jupyter is the language-agnostic notebook.


I was pretty sure all the 'GUI' components ended up under the Jupyter umbrella, not just the web notebook. I'll admit I could be wrong.

That being said, why do you find iPython so much better than Jupyter? I write probably 80% of my python code in Jupyter and cannot imagine coding without it.


One reason is that I can use iPython with a decent text editor like emacs. I've tried to make Jupyter work, but it seems like you're supposed to use the web based editor, which is horrible.

But it's also just not a nice workflow. Projects are not linear. I tend to work on a function and get it working in the REPL. Then build up the program by combining together higher level functions. How am I suppose to test functions in Jupyter? Write the test then keep evaluating it and then delete it later?


Jupyter is not linear. You can edit and evaluate any cell at any time. If you don't want test interspersed with the code, you can keep tests at the bottom by adding cells in the middle.


Jupyter is mostly useful for diplay of preexisting work.

I use it to make very pretty PDFs for bosses who want executive summaries, and I've started investigating using it to give lab talks so we can hack on the data together if i have to.

I don't think it's a particularly useful development tool.


Yeah this is my feeling too. I like it for presenting work that's already been done. But I won't actually do that work in Jupyter.


In an IDE, the REPL is often called the "Immediate" panel or similar. It's available whenever you're stopped at a breakpoint.

I can't think of any good ide (dating back to vb6 20 years ago) that doesn't have one.


Those things are specifically not called a REPL because they aren't a REPL. However, there is now a thing called the C# REPL [1] that shipped with VS 2015 Update 1, but I've never used it so I'm not sure if it's true to the name.

[1] https://msdn.microsoft.com/en-us/magazine/mt614271.aspx


Your link has a section titled "The Visual Studio C# Interactive Window" explaining that that's where to find all this REPL functionality in VS.NET.

But, as I said above, it's not explicitly labeled as such.


I got "Interactive" confused with the "Immediate" window. Things like the Immediate Window have been around for decades in most version of VS, but both "Interactive" and "C# REPL" are new to VS 2015.


> So Clojure has dynamic classloaders, but it does not have a replacement for classloaders. And so classloader confusion is something that cannot be solved at this level.

> But I could imagine if you were coming from Common Lisp, or something like that, and saying "that seems like a wart." And it is a wart that we lovingly embrace, for the mighty power of having access to the whole JVM ecosystem.

There's Armed Bear Common Lisp[0], which is a Common Lisp on Java, which gives access to the whole JVM ecosystem. Dunno how it handles classloaders though.

[0] https://common-lisp.net/project/armedbear/


Clojure is a Lisp designed from the ground up for the modern era. For new applications, it's a better fit than Common Lisp.


Since I've mainly used REPLs in a text-only environment, I never thought about the idea of creating little GUI 'sidecars' from the REPL, potentially with the aid of a little utility lib to simplify things such as "make a window containing the value of this var", "graph this var over time", or "make a button that executes this snippet of code". Very cool idea.


anecdotally, i tried adding up how many ipython sessions I currently have open for various reasons on my workstation.

I stopped at about 15. REPLs are incredibly useful.


A while ago I wrote a Vim plugin to send Python code to a running interpreter, redefining and debugging on the fly. Unfortunately that’s not a good fit for Python. Code reloading on classes , data descriptors etc requires adhoc semantics. All in all, hard to reason about.

I even find Jupyter notebooks get this a bit, when people eval cells out of order..


I was hoping for an article advocating REPL-driven development versus test driven development, I was a bit disappointed!


The alternative to REPL is a DEBUGGER with a great IDE. Don't get me wrong, REPL is nice, but IDEs are even better.


A "debugger" is a REPL.


It's a REPL with the ability to arbitrarily jump to chosen points in your application's execution, as opposed to attempting to populate the scope with copy-paste or dummy variables.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: