Neugram – Scripting in Go
neugram.io
neugram.io
I appreciate the sentiment – that there's a lot of work left to do – but isn't the assertion about syntax size false? I thought Go's syntax was quite small compared to other popular languages? It only has 25 reserved words (Java has 50+) and was designed in part to be uniform and easy to read.
With more words: I have written a parser and type checker for an ML-style language, with parametric types and several other neat tricks in it, and I've now written a parser and type checker for a good subset of Go. The latter has been far more work. I am not entirely sure how to explain the work. Go has lots of syntactic and type conveniences that are easy to use and read, but quite difficult to implement.
As there are few implementers and many users, I think the original designers of Go picked well when they put the work on us implementers.
First is the fact that the top level is statements, not declarations. That would mean heavy modification to the go/parser package to make the inner-function statement parser the top level.
Second is the fact I need to parse incrementally, to implement a REPL. That would not require serious modification to go/parser, but it would to go/types.
Third is I wanted to experiment with new syntax. It was quicker to get to that experimentation by working from scratch rather than adopting go/parser and go/types.
Fourth is there are some fundamental things I would like to do differently, particularly around how comments are handled. I haven't got there yet.
In retrospect, if the new Go parser inside cmd/compile existed when I started this, I probably would have started from there. (It does many of the things I wanted to do to go/parser.) I suspect I still would have ended up with my own type checker though, if for no other reason than incremental type checking.
I'll have to try it out.
Languages that have simpler syntax tend to have far more complex semantics (to infer what's missing, etc.).
Eg. python's semantics is horribly complex: "hello".upper() is something like str.__dict__['upper']("hello")) -- which is all run-time. Whereas say a C++ version amounts to quite a simple run-time function call on some bytes.
In particular note the difference between
const hello = "Hello, 世界"
const typedHello string = "Hello, 世界"
As a user of Go I find untyped constants to be extremely useful. They almost always do what I want without surprise. Implementing them however, is not trivial.
A tricker example is embedding. It adds a lot of complexity to the type system. As a user, it can be a bit surprising, but I admit it is very useful.
Let's look at some of Java's keywords which have no analogue in the Go specification's list of keywords
1. boolean: go has the `bool` type, but it's a builtin, not a keyword, so it doesn't count against it.
2. byte: same thing
3. null: Go's `nil` is again not counted as a keyword in their list of 25
....
And hey, that accounts for almost the entire difference if you count all the primitive types.
Go, on the other hand, does not consider those "keywords", which leads to absolutely silly behaviour.
A programmer can accidentally redefine `len` to be a variable, and then `len(x)` will be an error later.
Because go has a ton of important builtins which aren't reserved keywords, you can write incredibly confusing code:
https://play.golang.org/p/-xChCblMEh
For a nice list, check out https://play.golang.org/p/yjkxqVY4eo
... yeah, Go's small number of keywords is often touted by Gophers, but really it's because the go team made this incredibly stupid decision.
If you counted the same sorts of things as keywords that java did, Go would have almost exactly the same number... and more problems would be caught at compile-time (like `len := 1`) .
Whenever a difference is found between the two Go compilers, gccgo and gc, the spec is consulted to see which is right. If the spec is not clear on which compiler is right, then the spec is changed so it is.
When the third frontend was written (go/parser and go/types in the standard library), the spec played a similar role. Though far fewer changes were made to the spec, suggesting it is getting pretty good.
Java has 53 keywords, whereas Go has 25 keywords and 39 special identifiers, totalling 64, a little more than Java's but less than C#'s 78.
May be worth checking it out, at least for ideas/inspiration.
It's premature for feature requests I know. But I'd love to be able to view the Stack Trace of a live running goroutine. Typically something I log to disk using the "runtime" calls. To be able to probe that info from a command shell would be neat, don't you think?
I put a tiny link at the bottom of to the home page: https://neugram.io. I'll do something more elaborate when I'm done with this other bug I'm working on. Thanks.
Of course, I must put a disclaimer: I am incredibly biased as I am of Nim's core developers. But to help convince you, Nim has:
* Shebang support
* Error handling via exceptions
* A somewhat limited REPL
* Operator overloading
While I applaud the author's efforts, I can't help but wonder if Nim would fit their needs (or be easier to adjust to their needs).
I kept trying to write a "Why Neugram" post and kept getting stuck, which is why I went with this incomplete enumeration of differences from Go. But I get a bit more into my motivation in section 4 of this post today: https://neugram.io/blog/design-principles
Basically, I write Go all the time. I occasionally write bash, perl, or python. Occasionally enough, that those languages fall out of my head. What I really want is a scripting language that reuses all my day-to-day knowledge from Go, only with some more scripting language friendliness.
I suspect if I programmed in Nim every day, I would use it as a scripting language. Nim is neat.
https://schd.ws/hosted_files/osseu17/84/Replace%20UEFI%20wit...
I used that package in an interpreter of my own and it works wonderfully to provide `readline` style keyboard interaction in a pure Go program.
I fear I may hit a wall with it eventually. For example, several interesting approaches to https://github.com/neugram/ng/issues/49 would step outside what liner can do. But I'm going to use it as long as I can.
Definitely something to try.
Any ideas how it could relate to the go toolchain? could I stick my Neugram files in a repo and go get them?
You certainly could put your Neugram files in the same git repository as your Go code, and then "go get" would get them.
I think there is something to be said for something like:
import "github.com/crawshaw/foo/mypkg.ng"
(Note that a Neugram package is limited to a single file, unlike a Go package. This is for a couple of practical reasons, notably init order, and one philosophical reason: which is Neugram packages shouldn't get as big as Go packages.)
Go has an awesome ecosystem for writing servers and web applications but it's obvious that the libraries weren't built for scripting purposes and the language is more explicit (for good reasons).
Python, on the other hand has an ecosystem that's great for scripting and language features that are more compact and terse.
Possibly you don't because you use both languages more than me. My scripting needs are approximately once every other week, sometimes even longer. That's enough time for me to forget what python package has the date/time functions I need.
I also wouldn't even begin to compare something in its infancy like Neugram with something as well-established as Python's huge collection of libraries. There are many years of work between here and there.
It depends what the client or project demands. I've worked in Python, Javascript, PHP, VBA, Java, Go, and C (plus a few dialects of SQL). Spending months writing mostly one or two of those at a time. The only ones I enjoy enough to use in my hobby projects are Python, Javascript, and Go.
But in my experience the hardest part of toggling between languages like that is the ecosystem (libraries and tools).
In this case it seems like you're adding a new ecosystem which means I'd still have to stop and figure out "what's the right package to use for this?"
For the most part, language is easy. It's the idioms, libraries, and tools that I find myself googling the most.
import "github.com/pkg/errors"
Very much my goal is when scripting, to use the os.Open and ioutil.ReadFile I know, along with filepath.Join, time.Now, and friends.
It's a novel idea but I don't really see it taking off. Is there really that much benefit to having a shebang at the top of the file versus executing "go run" and is it really worth having a REPL for a fairly verbose language? I certainly don't see myself wishing for a Java REPL, and I don't particular see myself reaching for a Go REPL either.
Another thing I'm interested in is operator overloading, for compact/pleasant matrix/table types.