A JavaScript parser and interpreter written in Go | Hacker News Reader

A JavaScript parser and interpreter written in Go

github.com

232 pointsscapbi67 comments

A couple years ago for a programming languages course, we wrote a bytecode compiler and interpreter for a JavaScript-like language we were using in the class (objects, prototype-based inheritance, higher-order functions, etc), and we initially started building it in Go, but the biggest thing that made us switch to C++ at that time was the fact that Go didn't have a straightforward union type.

It looks like this interpreter is using tagged unions for values, and using the empty interface to emulate a union type. I seem to remember that we may have read something at the time that recommended using the empty interface instead of unions, though I don't remember for sure. Nice to see some interpretation efforts finally being realized in Go!

Sum Types in Go - http://www.jerf.org/iri/post/2917

It's less convenient than writing a compiler in Haskell, but then, what isn't? It does give you reasonable type safety, though. (Again, don't say that where a Haskell programmer is listening, but it's at least decent.)

Thanks. That was really interesting.

Excuse the silly question, but why was a union type so important?

Interpreters typically use polymorphic locations to hold values for the interpreted language. Technically almost any type of polymorphism is enough, but the fewer indirections you can get away with, the faster you can make a simple interpreter. So that would usually mean a sum type in a functional programming language with algebraic data types, or a simple object in an OO language, or a tagged union in C and Pascal.

But for better performance, approaches like tagged pointers (often used in Lisp implementations) are useful, as it eliminates a level of indirection for most integer operations. If you can afford to use 64 bits for your values, you might consider using doubles everywhere, with invalid exponents for stuffing a shortened pointer inside the float (I believe luajit uses, or used to use, this technique; it would probably be a good match for JS as well, as JS doesn't have integers).

I was wondering myself why you needed a union type.

Is it because JS has dynamic types ? var x = 5; x = "John Doe";

It allows you to create a single, compact structure that can then be utilized in a number of ways without having to re-cast it, replace it, or otherwise reallocate it.

For example, you can have a union between a 64-bit pointer and a 32-bit type identifier plus 32-bit value. This means you can store 32-bit integers in the same space as a pointer.

The question is what about a JS interpreter makes this necessary.

As I answered above to another commenter with a similar question, JavaScript is a dynamically typed language and representing dynamic values in a statically typed language requires a bit of thinking, and unions are a common way of doing this.

The alternative is a data structure which has N-1 empty, yet allocated, slots (N is the number of possible types.) instead you can have one field indicating the type of the variable and then your code can switch which field it accedes based off that indication. Total size is one int plus size of largest type represented.

You could potentially also have an interface that defined methods for all the basic types `interface { AsString() (string, error), etc. }` and then each basic type implements this interface and returns an error if it can't/shouldn't be represented as the requested type. A type-switch could give you the same info you get from the tag and you don't pay for type-assertions

Yes. JavaScript has dynamic types, and representing these in a statically typed language requires some finagling.

Casting to/from empty interfaces is kind of doing the same thing for void* pointers in C, right?

No, for once, Go doesn't have casting, it has type conversions, but they are type safe. With void * you can do anything, with interface{} you can only use the dynamic types inside the interface.

That being said, using interface{} for unions is extremely unfortunate. The bright side is that after 4 years of using Go almost exclusively, I only had to abuse interface{} only once, with compiler parsers (like the parent says). Every other time there was a better design which did not require using interface{}.

This is the most Hacker News sounding title I've ever heard.

No, there's not enough Redis here. It needs more Redis.

Haskel!

Its actually really easy to get involved in hacking on GHC, the main Haskell compiler. Seriously, its as easy as taking some time to just try to build current head and report any build problems! https://ghc.haskell.org/trac/ghc/wiki/Building

I'm actually really excited by all the people starting to jump in and try to learn/help. We've even got 2 really smart high schoolers doing some amazing contributions to GHC recently.

seriously: its easy to get involved in hacking on interesting open source projects. Just choose one you care about and stay excited by, and dig in!

I think the title has been changed. Can you say what the original was?

I could be wrong, but I think they're just poking fun at all the "An X interpreter written in Y" where X and Y are the current trendy languages. Usually one of X or Y are JavaScript or Go, so seeing both in the same headline is like some sort of Hacker News headline jackpot.

I'm subscribed to HN's RSS feed using a feed reader app on my phone, and when I saw this item the title was truncated at around "written." Somehow I just knew it would be Go.

I entertained the idea that maybe it would have been written in JavaScript, but I think a JS interpreter written in JS has already been done at least once.

The comment was also probably poking fun. Yours too?

We use this for the sync function interpreter in Sync Gateway. Robert has been very helpful and responsive with pull requests etc. thanks!

Edit to add link to example code using Otto https://github.com/couchbase/sync_gateway/blob/master/src/gi...

Thank you for being one of the few gophers that put newlines between standard, third party, and local imports. I wish gofmt forced that standard on people.

It is a useful convention in C and C++ to group includes, but I don't see the benefit in Go.

Since gofmt sorts imports, the grouping is soon undone.

In any case, it is quite easy to distinguish the three kinds of imports if local imports share a common prefix or set of prefixes.

Am I missing something here?

Note that gofmt sorts within groups. So if you leave spaces between groups of imports, it'll sort them all separately. Presumably, this sort of grouping is the reason it does it this way.

It's possible that if you have packages that understand flags (config package, with flags in init(), for example) that if you include it after the testing package, or before. One of them won't accept flags. (because flag.Parse() has already been called)

You can't rely on the order of imports initialization anyway. See http://golang.org/ref/spec#Program_execution .. I believe that's also the reason, you shouldn't call `flag.Parse()` in `init()`

Alright, 90 points, most comments being meta about the title so I'll be the brave one and ask: What is this actually good for?

I can't think of any reasonable use case. Grab little NPM ditties and incorporate them into your Go binary - Javascript to Go becomes as Lua is to C? Somebody enlighten me.

Edit: Not that this needs a use case per say, just that the intent behind it is underspecified enough for me to wonder about it.

We have distributed crawler written in go configurable with xpath and javascript for advanced crawling using javascriptcore-go[1] and also our knowledge engine serves various form of formatted information card scripted in javascript.

Duckduckgo goodies[2] also a good example of what we've been doing.

1) https://bitbucket.org/rj/javascriptcore-go

2) https://duckduckgo.com/goodies

In the case of web application, you can have the same templating engine running in the browser and in the server. This means rendering the full HTML at the server level if needed or part of it or just at the browser level. You get a bit of freedom. It is painful to manage two different templating engines between the server and browser sides.

I don't get it. Are you saying - okay let's replace the javascript run time in today's browser to this version in Go?

I don't think he's saying that. I think he's saying your Go server could execute similar JavaScript as the client executes.

Imagine a highly dynamic web page, beyond the initial page load when you interact with the page javascript executes the user's actions then rewrites large chunks of the page. As a developer, you need to write code on your backend that knows how to render the initial HTML for the page, but then you have to duplicate this functionality in JavaScript since the client needs to be able to render any chunk of the page that changes in response to a request.

You have a few options as a developer here. You can live with maintaining two code paths in different languages that do essentially the same thing. Or you can get rid of the backend rendering entirely, making your page less friendly to no-script users and web crawlers (and sometimes making the site flicker a bit as content gets loaded initially for all users.)

Or you can use shared code on the backend and the client for rendering HTML by having a backend that speaks JavaScript.

I distribute a binary that can be extended via Javascript/Lua/... API by a customer. This is great. I don't want the customer to mess with the Go source but I give him the chance to mess with the program by a well defined API.

Not everything needs a use case. But if you really want one, here's one: I'm a student and I want to write my own interpreter. Well, here's a good example of how to write your interpreter.

That's not a use case.

You nonetheless understood what I meant, and knew by reading my comment that "use case" was a mis-use of language. That's nit-picking.

We use it because we have user programmable callbacks, and JavaScript provides a well-known sandboxed language. Otto makes it really easy to expose native functionality to JS.

If it was fully functional it could be useful for automated testing. Write your tests in javascript, execute them at the point of integration in Go.

Reminds me of the early 2000s argument "why do we need another browser? IE is good enough".

This project may not be the fastest JS engine or the one with the most features right now, but if nothing else, it's a really nice project for learning Go and writing interpreters.

Maybe some new ideas will be explored in this engine first, because it's faster to implement them in Go than in the (huge) V8 or other JS engines.

Having new alternatives to established software is always a good thing.

I wasn't making any argument, simply asking.

You can scale npm now.

Isn't there also a Go parser and interpreter written in JavaScript?

Paradox!!!! Infinite loop!!!

Oh, wait causality doesn't work that way. Never mind. :P

I have no idea how to really ask this, but does it uses continuation-passing style [0] to execute expressions? I tried to search for "cps" or "continuation" in the repo, but no luck. I also don't really have the time right now to go dig through the source.

[0]: http://en.wikipedia.org/wiki/Continuation-passing_style

Most languages don't have reliable enough tail call elimination to implement CPS without trampoline techniques to get rid of the excess stack frames, which in turn has a fairly hefty performance impact unless you're using it for something high-level like async callbacks.

It would be nice to have some rough performance numbers to compare with existing JS engines.

Someone please bind the Go Net and IO and there you have a Node.js-esque-Go-Hybrid.

Not serious. haha.

Feed from node to go channels ... and then fly it on top of gevent async overload!

Can someone explain to me whats wrong with using V8 in Go? I mean what's the point in building js interpreter in Go, when you already have a better one made.

What's the point of starting anything new, when something like it already exists? Why become a doctor when there are already good doctors? Why write a book when there are already good books? Why sing a song when somebody else already sings it better?

By putting in the context the amount of money and number of one of the best programmers in the world building a V8, building a Go V8 alternative is probable not a best business choice unless you're a huge company with clear goals

I suppose this could be useful when generating a statistically compiled binary

Well, maybe they will come with a better Google Closure in Go

Because Google Closure Compiler is unbelievably slow

At least two concrete reasons:

1. No cgo required. 2. Portable to any platform Go runs on that V8 doesn't.

Now someone must make a Go interpreter written in JavaScript, so you can JavaScript while you Go while you JavaScript while you ...

Does this aim to compete with v8 implementations for high level products usable by non go developer (I think qtwebkit, here), or is it just a mean to have js integration in go for small low level scripting ?

Obviously the latter. Otherwise it would have to be a JIT compiling VM, not a simple interpreter.

the perfect hacker news title

Show HN: How a JavaScript parser and interpreter written in Go allowed my startup to succeed and Why your company should switch to Go

Show HN: A JavaScript parser and interpreter written in 30 lines of Go

you forgot to mention someframework.js, and Edward Snowden... :)

And that's how we learned not to use MongoDB.

...we accept Bitcoin too!

This is not a good thing nor will it lead to good things.

You say that, and yet here we are commenting on a forum? written in a Arc, a language implementation that wasn't strictly required.

Care to elaborate?

Reply on news.ycombinator.com