scrapscript.org
I started casually working on scrapscript in 2015. I built a few compilers over the years to test out various ideas/implementations, and I think I'm finally happy with the overall design.
The code is not public yet. Email me at hello@taylor.town if you're interested in joining the core team later this year.
Let me know if you have any questions or feedback :)
That's wildly cool.
This and Val Town [0] feel like they were made for each other. If you added them together with a nice front-end like mmm.page [1] I bet a million devs would sign up.
I'm working on something in a similar space that I'm hoping mihht develop into that [2]
It makes sense that the next step for the web is a more low-level social platform that works more like an agnostic API than a walled garden.
This is pretty much my plan for the next few years, with some added surprises :)
I really want to rekindle the days of geocities and flash. I call this "the cheap web" haha
I've also been working on open standards for authentication and payments within the ecosystem, but I'm not quite happy with the results yet. I don't like this world where we have to use companies like Auth0 and Stripe to build something worthwhile. I'm open to ideas!
[0] https://mastodon.ar.al/@aral
[1] https://mastodon.laurakalbag.com/@laura
Or maybe you can come up with something similar to a cryptocurrency that integrates with your data structure and has core things like digital signatures for secure transactions and public distributed ledger(s). It seems like there could be a way to adapt some underlying features of ScrapScript for this stuff. Maybe as part of the "lab" thing.
Just don't call it a cryptocurrency. Lol.
Excited to see where this project goes.
The more I dig into mmm.page, the more I'm loving what you've built! I can't wait to play around with it next week. The design is gorgeous so far.
Somebody is giving a talk on "Print Inspired People-Computing" at my tech conference this August. You two would probably get along pretty well!
I'd love to have you in our community, so feel free to email me at taylor@outland.sh and I'll send you a free ticket :)
Side note: I was just watching the val.town video and at the end he mentions that "you are an object" and other people can access your properties. It immediately made me think of playing hackmud. Not sure if that's a good thing or bad!
I'm grown out of it.
That being said, I have my thoughts on nostalgia-mining [1], and in short, believe that a more interesting visual culture today requires a more contemporary approach.
For example, one of my favorite pages is a page of hand-drawn comics combined with GIFs made by a kid [2], another is a cleverly-designed page on "soft tech" [3]. Here is an illustrative thread comparing link-in-bios and mmm.page sites made by the same person -- it's surprising how much tools can limit our self-expression [4].
Personally, I get excited when I see these examples -- a departure from the grid-confines of contemporary web design. In theory, nothing about "letting people create pages with as much freedom as possible" intrinsically = "90s web with blinking GIFs all over".
Let me know if you have any other questions.
[1] https://twitter.com/xhfloz/status/1631746051295117313
[2] https://littlebookfair.mmm.page/oneoneone
> pre { overflow-x: scroll; }
`overflow: scroll` is very poorly named, and practically never the right tool. You want `overflow: auto` instead, which only renders scrollbars if they’re needed.
(Long ago, there were two potentially valid scenarios for it: for avoiding reflow when changing the document’s length, and for unbreaking viewport units so that 100vw includes the viewport’s vertical scrollbar. The former was only exceedingly rarely useful, and can be achieved better now on all but Safari with the scrollbar-gutter property. The latter only Firefox ever implemented, and no one else wanted to, so they eventually removed it from the spec and from Firefox, so now viewport units are just stupidly broken by design. I myself have not found a single place I wanted to use `overflow: scroll` or where it would have been reasonable in the past decade. Stylesheet linters should include a default rule that complains about `overflow: scroll` because it’s so temptingly-named but so thoroughly not what you want.)
By default, using a name like `janedoe91/fibonacci` pulls the latest version of that particular scrap from the scrapyard. But all of its references are baked-in unless the author updates them.
The way around this is that consumers are empowered to use whatever forcing upgrade strategies they want. All scraps are transparent, so you can pipe everything through an "upgrader" scrap before execution, which replaces dependencies all the way down the stack with their latest versions. I'm sure many of these upgraders will exist depending on what types of tradeoffs you're looking for.
MVS is a nice and easy algo: https://research.swtch.com/vgo-mvs
user/my-scrap@99
user/my-scrap
All scraps are transparent, so you can create your own tools to manually tweak/replace dependencies before execution:
$ cat my-scrap.ss \
| scrap apply my-dependency-helper \
| scrap evalWhy opt out of proper dependency management?
user/my-scrap@414
Remember that the language itself works like a merkle tree and every scrap can be cached anywhere. To keep the language and tooling extremely simple, I think this works well enough.
> What if something breaks contract in the middle, something I do not control?
Everything is strongly typed, so there are certain guarantees about how things can break and update. You can easily make tooling that never makes breaking changes.
People can always introduce implementation bugs into their code though, but that's inevitable even with dependency management.
> That sounds like a huge burden on developers with lots of risk.
I will try my hardest to reduce that burden. I think I can put some sane defaults in the tooling surrounding the language to obviate the need for traditional dependency management.
1. My code depends on dep A and B, which also depends on A.
2. A has some issue, so I update my reference, but B does not.
3. Given dependencies are bundled, B still contains the bad A.
So how can I updated the A used by B? (Without effectively forking B) This is more a problem with bundled or censored dependencies than the lack of dependency management. Dep Mgmt aims to solve this issue by having only one copy of a dependency
Also the ideas of immutable deps mean this property of “duplicated” deps comes naturally.
I think many bugs can come from what you propose. A large dev tree would need everyone in the tree to coordinate to upgrade all at once. That seems basically impossible for any dep tree. Especially in a system where the deps are small code snippets like this one. Even small programs can have 1000s of deps.
Why are you implying that this model is “normal” dep management?
If the “old” A has a severe vulnerability and the “new” A has breaking changes - then there is no solution other than to avoid/replace/fork B until B is upgraded.
As I see it most of the more unique ideas could be implemented in form of some tooling on top of some existing languages.
What do I overlook here?
I deeply hope that's true, because it'll save me a lot of work :)
Software doesn't seem to be getting better. In fact, it seems to be getting worse in many areas.
Personally, I don't think there's "one big thing" that harms the ecosystem. I think it's friction in 1000 different places due to systems that don't quite work together seamlessly.
Scrapscript is an attempt to vertically integrate all the lessons we've learned in software in the past 80 years.
import foo/bar
And foo's bar library will just work in your program without you having to download it then bundle it. It will just look for foo’s bar library online when it runs (and I guess it’ll be running on an online platform, not necessarily on your own computer).
Type-safety guarantees that you won't get any side-effects where the types don't fit.
And if you really want to run untrusted code, scraps are designed to be very easy to inspect :)
Do equivalent programs produce the same hash?
Ie: a = 1 + 1 Return a
Vs
a = 2 Return a
What if local internal variable names change?
Ie
b = 2 Return b
Right now, all three programs would result in a different hash. I decided that the scrapyard publication process should do as little "magic" as possible.
As the tooling evolves, we can provide a canonicalizer scrap that renames variables and simplifies the code. I think eventually we would make this the default process to encourage more reuse.
I didn't read it in-depth yet, but I think I had a previous implementation that worked very similarly to theirs.
The problem with variable names is that they actually matter for some of the collaborative editor features I hope to implement in the future. There are certain assumptions I can't make quite yet, and so am keeping my options open :)
I could define that a familyname and a givenname are strings, and then a function argument of type familyname would be differentiated from a function argument of type givenname, but still the argument names themselves could be dropped from the hash.
A-If a program is a thing that produces a result for a given input, these programs are the same.
B-You can’t generally know if two programs are the same without solving halting problems, reducing polynomial algos to linear time or having infinite time and space computing resources, etc.
C-The best you can do is ask if two compilers produce the same output, or a linting/refactoring/canonicalization/formatting operation results in same source.
D-If a program is a thing that interactive debuggers use to create explorations next to source code while computing, these programs are different.
I agree with your approach of starting with these programs as being different, and allowing custom scraps to become a higher layer hash resolver. The reason is that compilers change over time, and it is only under the lens of a compiler(or linter, etc) that comparisons can be made. To bake a holy compiler in today would be arbitrary and quickly outdated.
n = 1
while sum(d for d in range(1,n) if n % d == 0) != n:
n += 2I'd be impressed if you gave me a program for which it provably cannot be proved that (edit: if) it halts. Or a bound on its size.
For every program that halts, it's obviously possible to prove that it halts. The point is that there is no algorithm for deciding if an arbitrary given program halts.
But can you prove that statement if I change it into:
there is no algorithm for deciding if an arbitrary given program with size less than N halts; for some suitable definition of size.
This looks really interesting to me.
There is a feature I would like that I have never seen in a scripting language, and I wonder if Scrap might support it:
> Monadic bind points and applicative join points in the script language
Hopefully an example can convey what I mean.
Bind:
do {
let! x = some_value_in_a_monad
x + 1
}
do {
let! x = some_value_in_a_monad
and! y = another_value_in_a_monad
(x + y) * y
}
Is this possible in Scrap?
Juggling lots of monads can be annoying. But to keep scrapscript small like json, this is something I would try to solve with editor tools rather than the language itself.
My hunch is that if you do not put the syntax into the language, then it will be hard for the community to standardize on one preprocessing tool.
OCaml (which I am a fan of!) struggles with fragmentation here.
I suspect that everybody will settle on a reusable `task` type, but each platform can determine its own async scheduling system.
Consider this example:
| "/home" -> q -> res:success <| "<p>howdy " ++ name ++ "</p>"
, name = q |> dict/get "name" |> maybe/default "partner"
| "/contact" -> _ -> res:success "<a href="mailto:hello@example.com">email</a>"
| _ -> _ -> res:notfound "<p>not found</p>"
. res = : success text : notfound text
text => query => res
. res = : success text : notfound text
But I 100% agree that community standardization is incredibly important! I just don't have a solution yet :) I will try my best to get everybody to work together though haha
Also how does this not create either an enormous tangle of circular references or a huge pile of unused... scrap?
The repository/scrap pile of content could be stored in lots of ways that you could choose. I imagine if everyone pushed directly to a global “scrap pile” like IPFS it would end up as a huge pile of unused scrap. Like unison, they’d need to provide some kind of default development environment, but anyone could build additional tooling (kind of like anyone could write a git clone).
For instance, suppose "foo" produces a structure and "bar" consumes it - what happens if the user passes the output of "foo version 10" to "bar version 1"?
If breaking changes occur, the type system will let you know :) Then you can either pin the function to an older version or fix the breaking changes directly.
The language provides enough guarantees to make tooling that can handle most of this for the user automatically.
I wish there was an example of a moderately-complex scrapbook, so that I could see what the code looks like when you start mixing several scraps and need a minimum of scaffolding.
I mention it here[1], but security updates can be performed by both authors and consumers.
What I like is you can flag "breaking" vs "enhancement" vs "no api change". And "no api change" is usually always what you get with a security patch.
Really you only need 2 versions in semantic version (my opinion): breaking.non-braking
"next versions just increment by 1" might be good where you have a non-breaking change, and that would allow security patches to flow down.
You can send things around without converting everything into strings and objects that don't quite fit your use-case. Plus you have the added benefit of strong typing AND the ability to send those types to other systems.
Most of the other features exist to make it nicer to use as a full programming language.
Thanks to the author, this is a very exciting thing.
Of course, ligatures without the "disable ligature under a cursor" extension are slightly weird, but it even adds charm, we're on a site "for hackers" (laughing).
So, well, what kind of a ChatGPT can come up with something like that? :)
Not every idea deserves to have an esoteric language attached to it to work.
EDIT: I just copied and pasted a code example into a text window and the raw typed text is exposed. Maybe something like a suggestion to do that could be included for anyone flummoxed by the symbols?
I had to copy-paste the "weird-looking play-symbol" (|>) into a text editor to see if it really was what I thought it was [1], after noticing that selecting it with the mouse split it in half. That is highly distracting from the message, at least to me.
[1]: It was.
Seriously, fonts that coders tend to use have included ligatures for quite a few years. For me, it’s less visual noise and a cleaner look.
I use Vim/Neovim and WezTerm but there are many combinations of editors and terminal emulators that support ligatures.
And many programmers fonts: https://github.com/ryanoasis/nerd-fonts
EDIT: the author claims that it will be open sourced soon :)
Email me at hello@taylor.town if you’re interested in joining the team
I think first steps with the core team would be to source feedback on some of my design and architecture decisions. It'll be much easier to build everything if everybody shares the same vision and we're aware of all the tradeoffs.
The bootstrapper was written in Elm, because its parsing combinator library is by far the nicest to use.
I wrote a rust bootstrapper in ~2019, but it was much too heavy for my experimentation phase. I think rust would be a good candidate going forward for the bootstrapper going forward though.
I'm open to suggestions!
I don't have experience with writing compilers, so I can't suggest anything. I just know that Haskell is promoted as a language suitable for this kind of task. Have you looked at it?
I only chose Elm because I was experimenting with some visual debugging stuff, and boy, it's really a joyful experience. The problem with Elm is that it's impossible to write a serious compiler in it hahaha
Maybe check out the Joy programming language? It seems well-suited to this domain.
I tried to make a concatenate sister language to scrapscript called "sewerscript", but there's only so much time in the world.
- the language is like JSON, so you can send programs (and custom types) between computers
- you can store code in public "scrapyards", and send references to the scraps for others to download later and reuse
Code is executed on your machine, just like python, JS, etc.
I first wanted to see if there was any community interest before dumping a ton of work into this project. I honestly didn't expect this much excitement :)
If each scrap carries its own versions of its dependencies, then even the "same type" could actually be a "different type" because of version mismatches?
I’m planning on publishing more of these kinds of details in the next month or two
It makes me feel like some hacker dweebs in the future would be communicating through this platform.
Looks interesting though. Would like play with this when it's released.
> # Optimized for AI & Autocomplete
> ```
> f a b
> . f = | x -> y -> x \* y
> . a = 1
> . b = 2
> ```
> Scrapscript encourages wishful thinking[0].
>
> Declare your goal up-front, and let your tooling make educated guesses about how to get there.
I feel like I took crazy pills. Or maybe this is what having a stroke feels like.
For this specific example, many languages have constructs like this:
let x = 1 in f x
f x . x = 1
For AI/autocomplete, tools like Github Copilot currently have to guess what you want from all the parts (e.g. variable names) you put on the table. By putting the variables last, the editor tooling can create boilerplate for your variables as you type.
Let me know if there's anything else that might need extra explanation!
f a b where
f = \x -> \y -> x * y -- or f = (*)
a = 1
b = 2 > . f = | x -> y -> x \* y
Scrapscript:
| x -> y -> x * y
x => y => x * y
| true -> true -> 2
| true -> false -> 1
| false -> true -> 1
| false -> false -> 0With regards to AI, I think focusing on the fancy auto-complete isn't really a great idea. This seems designed around AI as it is now, not where it will be. But, I see your point, especially where your approach solves other issues. I think the page would be better served if you mentioned AI/auto-complete more as a side-effect than intent.
---
A small nit before I dive into my main point: the page is a pain to copy and paste from. Things like title capitalization should not be lost when doing so.
---
My biggest issue with this page is that it says it solves several issues but handwaves critical details and doesn't demonstrate the value proposition of scrapscript when claiming to solve them. You make a very big claim at the beginning that you are solving all of these problems:
> APIs diverge, packages crumble, configs ossify, serialization corrupts, git tangles, dependencies break, documentation dies, vulnerabilities surface, etc.
but the page is very lacking in the details of what parts of those problems scrapscript are solving (many of these have multiple vectors), and in places where it does, it often leaves out critical details, uses very simplistic examples that feel trivially solvable in many languages, or provides only example code that is difficult for newcomers to relate to its section.
For instance, I assume the APIs diverge claim is addressed under the section `Typecheck Across Network Bounds`:
> The scrapscript compiler tells you when remote APIs differ from the code. And if the API changes while the code is running, scrapscript offers a series of graceful handling options.
What are these graceful handling options? (Separately, is the compiler reaching out to these scrapyards every time it runs?)
I'm also not convinced some of these problems are solved. For example, documentation dies. I am guessing this is addressed by the `Self-Documenting Typed Configs` section. From that, I'm guessing the documentation referenced is auto-generated. But, the most important part of documentation is intent, which is usually not captured by code. In your example:
my-org:my-config
{ name = "my-server-001"
, cpus = :4
, mem = :16
}
. my-org = : my-config
{ name = text
, cpus = : 1 : 2 : 4 : 8
, mem = : 1 : 2 : 4 : 8 : 16 : 32
}
Overall, I think you should be very explicit about which presented issue each section of document is addressing. More details about how they are being solved need to be shared. And better examples should be used to demonstrate why scrapscript is the superior choice to solving these issues rather than other languages and their libraries.
All in all, there's a lot of cool stuff in here, and I bet this is just a result of you not having the time to spend on this page. But right now, this page comes across as a lot of empty marketing (and there are a lot of buzzwords).
I do have other, language-specific concerns (e.g. time != versioning, scrapscript assurances that all programs are safe to send around, `echo "'err red'"`), but I don't feel like there's enough information on this page for me to intelligently raise them.
Good luck!
My best friend gave me much of the same advice before I published the page, but I became too impatient.
I've bookmarked your comment, and will make sure to include deeper explanations in the near future
Thank you :)
Nope; anyone who has some basic programming knowledge and perhaps a high level understanding about how different programming languages work. And the ability to connect some dots together. I'm old enough to have learned BASIC programming on an Apple II and took Pascal in college, which is the extent of my formal programming training.
> “it’s JSON with types and functions and hashed references”
Another way to say this: it's executable YAML, which is a strict superset of JSON [1]. I like YAML, so that's what came to mind when looking at ScrapScript code. I also like clean, minimal syntax when I can get it, like YAML, CoffeeScript or Haml.
> “it’s a language with a weird IPFS thing”
I’ve been playing with IPFS [2] since its early days in 2015, though not much recently, though this will probably get me back into it. Content addressing solves a lot of problems that I won’t get into here but it’s certainly not that hard to grasp. IPFS is available for pretty much every browser these days and is integrated into the Brave browser [3].
> all programs are data
This concept has been around since the creation of Lisp in the 1950's. This enables all kinds of cool features the computer science types get excited about. I've known about this concept since the 80's when I used to teach kids Logo (which is a Lisp). The term used nowadays is homoiconicity [4] but that term wasn't in widespread use until fairly recently.
Bottom line: ScrapScript sounds very interesting and I'm looking forward to checking it out.
[1]: https://yaml.org/spec/1.2.2/#12-yaml-history
[2]: https://ipfs.tech