Proposal for a Go Interpreter | Hacker News Reader

Proposal for a Go Interpreter

docs.google.com

114 pointsmhfs47 comments

> To provide an interpreter that can embed inside a Go application, in order to provide scriptability to the Go ecosystem at large.

Can we move away from scripting engines at some point. They are big black boxes that you throw characters into and then "things" happen that you have no control over.

* What if I don't want the security?

* What if I want more security?

* What if I want actual pointers?

* What if I want to do one of a million things that don't fall into the original assumptions about what a Go interpreter would do?

Compiler-as-a-service is the future. A high quality parse tree, with a built-in interpreter or possibly JITter. Refactoring tools and tons of other cool things can be built instead of the same old black box.

If it's just going to be another plain-old interpreter, it would be best to instead focus on integrating V8 because a Go interpreter would bring far less to the table (notably interpreters are typically slow and one of the big benefits of Go is that it can be fast).

+1 for the idea of integrating V8.

I tried using V8 from Go, but it turned out to be a performance-disaster mainly because the garbage collection strategy of Go requires objects in Go to be movable, hence all interactions between Go and V8 have to go through a lookup-table.

interesting, it's true having a toolbox for creating go interpreters would be much more powerful.

I am not sure I completely get what a compiler-as-a-service really is. could you expand please?

(wrt JITing, yes, we considered it and it's on our radar. we know some version of it worked at some point, at least at the proof-of-concept level: https://github.com/nelhage/gojit)

thanks for your input!

I suppose CAAS[1] doesn't make much sense outside of the .Net community :). The idea is that you very aggressively break down the compiler into the core constituent parts.

A less Microsoft-centric project would be Clang. Clang is quite accessible and despite that it's not as loosely coupled as Roslyn (e.g. I don't think you can refactor using it), you can still solve some real problems with it (such as autocomplete and static analysis).

Basically, CAAS is the front-end equivalent of LLVM. You turn the whole compiler into a very tidy API: following from that it should be significantly simpler to create something such as the interpreter that you are aiming for.

[1]: http://www.infoworld.com/article/2621132/microsoft-net/micro...

ah, didn't know about CAAS. of course LLVM/CLang was also on our radar. we are still pondering on whether we should target LLVM bitcode so we could easily build upon LLVM toolchain (and cross-pollinate with other LLVM-based interpreters. e.g. seamlessly interoperate with IJulia and CLing interpreters, sharing code and values at runtime thru LLVM.) to me, the main issue with LLVM is that it's a real pain to develop against, I know this for a fact having worked a little bit with CLing, the CLang-based C++ interpreter (the edit-compile-test cycle is that of a set of large C++ libraries, gophers tend to be spoiled in that department)

tangential: actually, you can refactor with the CLang libraries: http://llvm.org/releases/3.7.0/tools/clang/docs/LibTooling.h... (well, to some extent)

V8 is large and not actually that easy to embed. There is a pure Go implementation of Lua for scripting already [1]

[1] https://github.com/Shopify/go-lua

There's also GopherLua [1], which is more active and supports features like coroutines and channels.

[1] https://github.com/yuin/gopher-lua

Compiler-as-a-service is the future.

Lisp. You want Lisp.

FYI: To view it without the 'suggestions' just go to the right and change your own mode to view. Then the stupid changes other are suggesting wont show.

This link should take you there directly: https://docs.google.com/document/d/1Hvxf6NMPaCUd-1iqm_968SuH...

Ah, yeah. I think given the amount of abuse on the doc, the post should be updated to have this as the link.

Go is a great language. But why on Earth would you want to script with it?

I'm not sure either. So these are just guesses:

• You already know it.

• You want to use Go libraries.

• You want to extend a Go codebase at runtime, without integrating another language like Lua.

Weirder possibilities:

• You want to be able to use it as the basis for a Go "live image" system, where you can hot-reload modules on editor-save by re-parsing their source.

• You want to try building a Go version of Ruby's Pry (http://pryrepl.org)

• You want to enable your Go program to do stupid things involving generating strings of Go code and eval()ing them.

• You want to take some first steps toward some utopian dream of pushing live Goroutines over the network as frozen state + closure source, similar to what could be achieved on Mosix or the Erlang VM.

1) play.golang.org is a popular feature of Go. Why not have better versions ? (e.g. an IPython/Jupyter kernel)

2) Golang depends on code generation for a lot of it's features. Executing generated code immediately could be helpful.

3) Very quickly execute small tests - one of my favorite python and java/eclipse features that I find somewhat lacking in Go and very much lacking in C++ development : Unit tests that execute before you've fully processed the fact that you pressed the test button in the IDE.

4) Security isolation of parts of programs would be useful

5) Same with various other isolation principles - memory usage, cpu usage, ...

6) I've always thought that Go's best feature is that it's pretty good at compiling different processes into one binary that then work pretty independently. But as I've been making bigger programs I find this less and less true.

Truth is, we need a programming language that is an operating system. Sort of like Erlang, but euhm, somewhat more usable. Go isn't that language. A language where you can start up a thread with $megabytes memory available, $cpu cpu allocation, kill if it goes over it for more than 2-3 seconds and restart after waiting 500ms. Something like that. It needs to be a language where you can really do separation-of-concerns - optimization if compiled into the same binary - minimal code changes to move a specific service to a version running on a different machine.

actually, somebody already wrote a kernel for (at the time) IPython, using "my" github.com/sbinet/go-eval go interpreter: https://github.com/takluyver/igo

my plan for this new go interpreter proposal, is to be able to use for scientific exploratory work (like we do with IPython right now when in a python-based ecosystem or CLing when in C++ land).

#3 in particular would be a huge win, not just for tests, but for exploring bits of your program in a repl as you develop it.

2) Golang depends on code generation for a lot of it's features. Executing generated code immediately could be helpful.

That's horrifying.

Let's pre-emptively define a .ghp file format !

That would be totally amazing. I played with the idea of letting clients run custom code on my servers but that didn't materialize so far due to the amount of work required to safely allow that. Being able to do that easily would certainly make online tools more configurable for power-users - or let an application-economy flourish where users can share small snippets that customizes a service in given ways.

For reference, I run a website analyzing service that crawls websites every week or so and reports errors, mainly broken links. As all the document sources are available at TCP level it would be trivial to extend it with custom error reporting.

Thank you zupa-hu, this is exactly the type of use-case the proposal is intended for.

> Wow, this file is really popular! Some tools might be unavailable until the crowd clears. [Try again] [Dismiss]

Tangential question: Is it documented anywhere how many simultaneous users a Google Doc can have?

I would be curious to know as well. Also if Google Apps have a higher or same limit. I've come across this limit with things that were only mildly popular and I've wondered if there are any large companies who use this that could possibly run into the limit internally and if so if there is a way to raise the limit.

Given how fast go code compiles, I wonder how many use-cases for interpreted Go code would be solved by a good plugin system? Certainly for the use-cases where I would use this proposal, I'd be just as happy (if not happier) with a plugin system - which is probably a bunch less effort to implement, too.

There was originally a proposal for a fairly promising plugin architecture for 1.5 - it didn't make the cut, in the end, but there might be some PoC bits out there if anyone's interested.

I used to use Lua a bit for scripting, so I was thinking about that as a scripting language for golang too.

After some discussion with a friend, we decide that a golang interpreter would be the best solution though. It could allow you to use channels to communicate between the interpreted and compiled sides of the app. Spawning goroutines on either side would also be handy.

However, that was yet one more project on my list of side projects that I haven't gotten to... So it is nice to see some interest in this idea.

Wonderful ansible, thank you for supporting the idea. We'll have to work-out how to use channels to communicate between interpreted and compiled sides of the app.

I was surprised the GoRe REPL [1] was not in the list of existing tools [2]?

[1] https://github.com/motemen/gore

[2] https://github.com/sbinet/notes-and-todos/blob/master/README...

that's my personal bias. GoRe's REPL recompiles and re-runs the resulting binary after each line entered, accumulating side effects:

  $ gore
  gore version 0.2.5  :help for help
  gore> var i = 41
  gore> f := func() { i++; fmt.Printf("universe=%d\n", i) }
  (func())(0x4012b0)
  gore> f()
  universe=42
  gore> f()
  universe=42
  universe=43

when Go was released back in 2009 there has been a string of such REPLs, which don't match "my" definition of a REPL, at least, not the one I want to be able to use for exploratory work (the kind you get from a python or IPython prompt)

again, that's my personal bias. that said, GoRe is a nice project (gocode-completion, especially, is nice)

Go is one of the fastest compilers around. I think compile-speed was one of the design-goals. Hence, what additional benefit would an interpreter have?

state. as I said elsewhere in this HN thread, many of the first REPLs that appeared when Go was first released used this rather effective scheme of recompiling on the fly a single main.go file after each new input the user entered and re-running the whole thing.

nothing's wrong with that except that you accumulate side-effects:

  gorepl> f := func() { launchMissiles(); fmt.Printf(">>> missiles launched\n") }
  gorepl> f()
  >>> missiles launched
  gorepl> var a = 2
  >>> missiles launched
  gorepl> fmt.Printf("oops!\n")
  >>> missiles launched
  oops!

that's not the kind of REPL I want, and that's not what "my" REPL does (https://github.com/sbinet/igo) nor the one built on top of LLVM (llgoi: http://llvm.org/viewvc/llvm-project/llgo/trunk/cmd/llgoi/)

Please stop with the vandalism.

Can you explain what you mean?

OP posted to a document that was/is modifiable by anyone who can access it - so jerks started messing with it.

Tought so, but now it says it's "too popular" and some features have been disabled, so it should be "solved" -- I definitely didn't see anything weird in the document.

I didn't see anything odd either.

That's good, when I saw it the whole document had been defaced.

Yes, now read-only.

Yeah, when I had replied it was unreadable as all of the text and been crossed out, random edits introduced, etc.

If it's not 'too popular', it defaults to 'suggestions' mode, which will show all the vandalism -- switch to 'viewing' mode, and you'll only see the original doc.

Why would there be "legal issues may restrict use of an interpreter in iOS applications"?

Because Go for iOS is a thing now, and iOS is the only platform currently that would prohibit the use of run-time interpreted code.

Interpreted code is now allowed on iOS. You just can't run code that has been downloaded over a network with the exception of content loaded in a UIWebView.

As long as the interpreter doesn't execute any prohibited system calls, the iOS store reviewers won't care and won't have a point to argue. OpenGL shaders are 'interpreted' code (though explicitly allowed). Protobuf messages can be 'interpreted' code. You probably can't get away with shipping Go code as a resource (or fetching it in cleartext over the network), but you probably should secure the input to your interpreter anyways.

There's not much difference between downloading and parsing several hundred kb of JSON and downloading and running a script. I mean, there is, but the iOS store peeps don't need to know ;)

That was true like 5 years ago. The author is just misinformed.

Many thanks, I'll remove that line.

Here is a potentially more urgent idea: Create a transpiler Javascript to Go. Now THAT would get my attention.

Or if you're feeling really adventurous, create a JavaScript -> HaXe transpiler and add a Go output for Haxe.

The compiler team built a non-general transpiler from C to Go in order to port the compiler.

Perhaps it could be hacked to read js?

Reply on news.ycombinator.com