Hacker News new | past | comments | ask | show | jobs | submit login
Scrapscript: A functional, content-addressable programming language (github.com/tekknolagi)
193 points by luu 68 days ago | hide | past | favorite | 40 comments



> Scrapscript solves the software sharability problem.

> it’s JSON with types and functions and hashed references

> it’s tiny Haskell with extreme syntactic consistency

Content-addressability is one of the main features of Unison [1]. How do the languages compare? Why did you feel the need to create a new language (especially since both have a strong Haskell flavor).

I'm guessing lots of people will have these questions so it could be good to mention Unison somewhere in your materials. The language seems to have big goals, so it doesn't seem that you are working on it "just for fun" :-).

Are there other "content-addressable" languages out there other than these two?

--

1: https://www.unison-lang.org/


Great questions and observations!

--

> Why did you feel the need to create a new language?

Well when I started on this language in 2017-2019(?), I definitely wasn't aware of Unison.

I remember talking to Richard Feldman about scrapscript when he first started working on Roc (fun fact: I was the first person Feldman added to the private repo!).

I spent a long time on the initial Scrapscript implementations, trying to compete in some similar applications as Roc, and Feldman just absolutely crushed it. I felt (and still feel) that Roc is doing a great job at the low-level "platform" experience I yearn for.

[1] https://www.roc-lang.org/

Eventually I met some Unison folks at Strange Loop (2019?), and we had very different inspirations and goals. That seems to still be the case.

There are some hard-to-articulate things I want out of the web, and Unison's architecture doesn't seem suited for it.

Anyway, Scrapscript only got a bunch of attention a year or two when I started working on it in public a bit more. I think defeating alcoholism also made me much more productive haha

--

> How do the languages compare?

From what I understand, Unison has very little in common with Scrapscript.

"Content-addressibilty" is really more of a mechanism than a "feature". Kinda like how AI is not really a "feature". You can't just sprinkle SHA on something and go raise venture capital haha

I think one major difference is that Unison opted for a more traditional git-based paradigm and then built some stellar dev tools on top of that. So far, Scrapscript is a bit more ambitious in its plans.

So go try out Unison! And come back and play with Scrapscript too, when it's more mature :)


> From what I understand, Unison has very little in common with Scrapscript.

I think you should read up on Unison a bit more, I see a lot of overlap.

Both identify code with hashes, and implement code sharing through hash based syncing. Builtin serialization is also present in both, and both are very Haskell inspired.

One difference I see is the sharing mechanism, where Unison has a local sqlite repository and tooling for uploading, while Scrapscript seems to be built more around IPFS. But that seems like an implementation detail rather than a fundemantal difference.

Unison supports algebraic effects, aka abilities with handlers, which is a much more powerful and general concept than the platforms in Roc.


I think ergonomics and practicality could use more consideration when discussing new languages. A language exists in an ecosystem.

At a glance to me, Scrapscript looks very traditional and easy to grok from a regular programmers perspective, there are build steps, you can use an IDE or editor etc.

Unison feels more like "Smalltalk Squeak" and Scrapscript feels more like "GNU Smalltalk". Come live in our new world, or integrate new stuff into your existing world kind of vibe.


Congrats on all the impressive progress!


Can't sprinkle SHA on something and raise venture capital? Ahem, behold Bitcoin. :)


Bitcoin was not VC funded


... but a bunch of companies were, and a lot of the claims about what web3 was going to be were really vague


> I think defeating alcoholism also made me much more productive haha

Just out of curiosity and because we all have different limits - how much do you think you used to drink, generally, and what helped you quit?


Before I quit, it was ~6 drinks per day minimum.

Here's the essay I wrote when I decided to quit:

[1] https://taylor.town/1000-weekends

Here was my 8-week progress report:

[2] https://taylor.town/8-weekends

Right now I'm 96 weeks in! Going great!

If you need any support/encouragement, always feel free to email me at hello@taylor.town :)


I've had an unnamed content-addressable language in my private repos for awhile now. The language was an exploration of what it would look like if an entire language was transactional, and any failures (power loss, etc) could be mitigated by simply restarting execution at the last line of code executed. Content addressability helped solve the issue of ensuring state was consistent even in the presence of restarted I/O transactions.



The web site appears to claim that this solves interoperability between systems. I don't understand how. What enables one system to understand content coming out of another? For example, let's say you have two medical devices sharing patient data. Will scrapscript make it possible for them to create a mapping between the information models of the two systems?


Hi! It's me, I'm one of the interpreter and compiler authors. Happy to answer questions


What is the runtime thing I run on a machine in order to support shipping code around? How does that work?

Unison makes reference to a node I also could find scant details of rust.


I'm not 100% sure what you're asking. The package manager equivalent is in development but currently looks like a tiny wrapper around a hash->serialized_program database.


I think you need Python to be running the script on each machine.


You can also compile a scrap to C (and then probably from there to native, or apparently with pnut you can now compile to shell...)


Hello! This project looks very interesting. How would you compare it to Dhall, which seems to have similar aims? Is scrapscript Turing complete, or are there totality guarantees?

https://dhall-lang.org/


What exactly does it mean to "address content", and how do I do it in the syntax of scrapscript?


This might be a stupid question, but by "content-addressable" do you mean that namespaces are eliminated? This thing where you store each individual version of a function indexed by hash, that's content addressability?


Because when I think of content-addressability, I mean you can find things intensionally and not just extensionally, that is, you can know the shape of what you're after and find something that matches that shape. More or less what vector databases are doing these days. But in this context, it seems to mean that all the transactions (? maybe?) are stored and indexed by a unique key - but that still means I need that key to get something, right?


So, (and this is five minutes playing here no deep dive), content-addressable is taken Seriously here.

Each expression is a hash of (its ancestors - not too sure) but the implication is sort of the git dag is built into the language. Looks like that has deepmimplications for releases, deployments, inspecting code chnages

There is an implication that my-labcorp is using scrapscript. Which is interesting as it seems to be a health based platform - tracking and dealing with test results, X-ray images etc etc

So I love committing to one of the most basic CS concepts (cryptographic hashes)

And I love the implications for code management. What I am not sure is why that helps with things like labcorp platform - I also love those implications (encrypt and store and just shuffle that data) because cannot any language use the concepts ?


I don't think I fully understand the way versioning and encapsulation happen by hashing in this, but I agree that conceptually it sounds very cool. Maybe the first new language I'd take a stab at in a long time just for the novelty of it.


Another niche that ScrapScript might fit nicely in is NixOS/Guix type specification languages (e.g. nix)


Related:

Some Tricks from the Scrapscript Compiler - https://news.ycombinator.com/item?id=40933250 - July 2024 (6 comments)

Scrapscript - https://news.ycombinator.com/item?id=40613756 - June 2024 (1 comment)

A Baseline Scrapscript Compiler - https://news.ycombinator.com/item?id=40551583 - June 2024 (4 comments)

scrapscript.py - https://news.ycombinator.com/item?id=39104504 - Jan 2024 (75 comments)

Show HN: Scrapscript “guide”, proposals, and community chat - https://news.ycombinator.com/item?id=36519579 - June 2023 (2 comments)

Show HN: ScrapScript – A tiny functional language for sharable software - https://news.ycombinator.com/item?id=35712163 - April 2023 (135 comments)


I am interested in this as a replacement for bash scripts

The killer feature would be pulling in other scripts as libraries (they may be published anywhere) but I can lock down the hash of these dependencies inside of my script file.

Is this possible?


Yep! In scrapscript, you can use the hashes directly to lock down the versions of any code directly.

There's a few reasons why this might be annoying to deal with in practice, so we're working on a few different ways of managing versions throughout your script


Deno can do this, and some other languages but Deno is the nicest I've used for this purpose so far.

There's a great list here

https://dbohdan.com/scripts-with-dependencies

but be aware that a lot of those are through third party tools and therefore don't have proper IDE support etc.


FWIW we can do this* with bash in the nix ecosystem (not that this knowledge should deter you from checking out scrapscript).

* you said locking the hash in the script, and in the nix approach these are generally locked in either a lockfile or in the nix expression.

Can elaborate if you're curious.


Multiple files adds just a tiny bit of friction when moving things around, putting on Gist, sending on Slack, etc.

I would love for the Nix lock to somehow be embedded in the script.


You can inline the script within nix and get them all in one file (but that implies buy-in on nix, or maintaining 2 copies of the script).

At the risk of overstepping (it sounds like you might be a little interested) here are a few links in to some basics just-in-case.

For context, I extracted my bashrc into its own repo, and it in turn sources bash libraries for my shell history and git status plugins. It's also leaning on some libraries that are pulled in through those two. (To be clear, this is just a general example of the flake pattern with a separate lockfile. I can find or synthesize something with the inline script pattern if you ask.)

- Source statements: https://github.com/abathur/bashrc.nix/blob/e7c437578e2a623f4...

- Nix expression responsible for resolving and ~linking these two dependencies in to the final script: https://github.com/abathur/bashrc.nix/blob/e7c437578e2a623f4...

- Flake inputs for the two dependencies. Note that these can be specified as just the repo or a branch and updated by updating the lock through Nix, or they can be ~pinned to a specific rev: https://github.com/abathur/bashrc.nix/blob/e7c437578e2a623f4...

- A gist with examples of the entire chain of scripts that get resolved together here: https://gist.github.com/abathur/09f06f1f34b003376017d7a5c031...

These all get handled separately, so they can technically have divergent implementations of the same external command (for example, one library could depend on libressl and one could depend on openssl--and each resolved library would independently point to the correct one.


As the author of another language with uses content addressable code for all source code (also to aid in shareable code - https://yazz.com/app/homepage.html) I feel that it is really good to see that projects such as ScrapScript use this technique. Well done guys!


Lots of amazing ideas here! One more that would be great is 'sandboxing' if a program can use network/filesystem/etc, then I would feel safe to run any random scrap if I knew I would get some sort of oauth-looking screen that said what it wanted to access.


Is there a way to search the large scrap yard?


How does Scrapscript compare to Dhall?


i’ve liked this from the start but never dove too deep.


Same, good that op posted it as a reminder


It compiles to python?


Compiler is written in python. It compiles to native code




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: