Gomacro: Interactive Go interpreter and debugger with generics and macros | Hacker News Reader

Gomacro: Interactive Go interpreter and debugger with generics and macros

github.com

189 pointsingve85 comments

C++ style templates...? What's wrong with ripping TypeScript's generic implementation? It's simple, clean and direct

    function firstOf<T>(anArray: T[]): T {
         return anArray[0]
    }

    interface thing<T, Y> {
        key: T
        anotherOne: Y
    }

    const things: thing<string, string> = [
        { key: 'hello', anotherOne: 'world'},
        { key: 'value', anotherOne: 'thing'}
    ]

    const result = firstOf<thing>(things)

Typescript, as I know, uses type erasure to implement generics, which can cause significant overhead in cases like JSON decoder.

> Also, Go is currently lacking generics (read: C++-like templates) because of the rationale "we do not yet know how to do them right, and once you do them wrong everybody is stuck with them"

To be honest C++-like templates are probably the worst way to do generics. C++ "did them wrong" by picking C++-like templates and now everybody in the C++ community is paying the price being stuck with horrible unreadable error messages, abysmal compile times, an explosion of complexity (SFINAE, CRTP, metaprogramming using templates, template specialization, etc.) and a myriad of other problems.

I sincerely hope Go doesn't adopt C++-like templates. Fortunately C++-like templates are not the only game in town, and there are other, a lot more saner and principled flavours of parametric polymorphism out there.

> To be honest C++-like templates are probably the worst way to do generics.

I don't think this is fair. Any approach to parametric types has to make some very hard trade-offs. C++ sacrificed:

* Fast, modular compilation

* Straightforward error messages

* Low cognitive overhead

In return:

* Templates perform as fast as hand-instantiated types. This is something some other languages like Java definitely cannot claim and is a key requirement for C++.

* Template functions can have complex constraints on their template parameters. Deferring type checking until after instantiation means you can do all sorts of interesting things with the template parameter and as long as all of those operations happen to work with the instantiated type, it's allowed. Most other languages place much more stringent restrictions on the expressiveness of generics. For example, in Java, you can't do "new T()". In C#, you can, but you can't do "new T(...)" with any kind of argument list.

Those are both really valuable features, especially in a language where performance is so critical.

Rust's generics don't have overhead and the error messages are at least as sane as Java's. I think the problem is with how C++ implemented its templates.

C++'s templates are more expressive than Rust's generics. Rust is adding stuff to try to catch up (specialization, const generics). You might see the extra power as an anti-feature, and that's fine, but it's undeniably useful to lots of folks and is undeniably a trade off.

But doesn't the fact that Rust is managing to add these features mean the lack thereof is not a limitation of classical parametric polymorphism?

Not that they're necessarily trivial to integrate, especially when the people working on it would rather try for the stars of genericity (e.g. dependent types versus const generics).

I would be surprised if Rust's type system winds up supporting everything C++ templates do. But maybe I'll be wrong, or the only things not covered will be niche and easily worked around. I don't know.

I feel like part of the reason that I don't run into issues with Rust having less expressive generics than C++ is that Rust has a ton of other useful meta-programming features; a lot of the crazy things I've seen solved by C++ templates can be solved in Rust via things like proc macros.

Just wondering if it's possible to implement something like expression template used in Eigen3 in Rust. C++'s template can be used in more than generics.

Sure, but those are different tradeoffs than those listed above. And some of the warts with C++ templates might not be inherent to supporting the features that C++ templates offer over Rust generics--for example, what features are enabled by error messages that point into the expansion of a template rather than a type error at the template itself?

They aren't different. The last bullet point in munificient's comment is a point about expressiveness.

I honestly don't know what point you're trying to make here. This really shouldn't be controversial.

Fair point, I missed the last bullet. To me, munificent's comment reads like a false dichotomy--you either have the Java's problems or C++'s problems. I was pointing out that there are other systems that largely have neither--namely that you can have template-based generics that perform better than Java and are friendlier than C++. He may well not have meant it that way, but that's the context for understanding my posts.

C++'s templates are more expressive than Rust's generics

80/20 rule applies here. 80% of the boilerplate can be eliminated with 20% of the expressiveness.

Yes. I agree. Some might even argue that this applies to Go itself: its builtin generics get you a lot without bringing in the full baggage of user-defined generics.

There's really nothing in my comment that suggests I think otherwise, so I'm a bit confused by your comment.

I'm expanding on your comment about trade-offs. I tend to like nuanced views of cost-benefit trade-offs in language design. It is a complicated subject, after all.

Yeah, but there are some limitations with Rust's generics around numeric types. You can't declare generics with arrays of a fixed size(which is really common in performance sensitive code) so there are trade-offs.

I understand that it's being worked on. I won't disagree that C++ templates are sharp, pointy tools but there are some things that they enable which you can't get in other languages.

Yeah, my point was not "Rust generics are strictly better than C++ templates in all cases", but rather that the GP made it sound like you have to choose between C++ templates and type erasure and Rust proves that's a false dichotomy. Sorry if I was confusing.

I think it's how C++ is compiled. Until recently, there was no way to forward declare templates and that required all sorts of hoop jumping or concessions to overcome.

Similarly, the existence of header files at all is probably an anti-feature.

> and the error messages are at least as sane as Java's.

Last time I made a mistake stringing Tokio promises together, the error messages were straight from the C++ hell.

...and for that we have C++.

For me Go is a replacement for Java. It is my "workhorse language" for server applications and for writing stuff that other people have to deal with and understand. For this type of software Java and Go are fast enough. Go has the advantage of coming with a lighter cognitive load than Java because it isn't dominated by huge frameworks. Nor is the language itself complex. These are important reasons why we chose Go.

If I were in the gaming industry or writing software that depends on wringing every last bit of performance out of the CPU, C++ would probably be relevant for me.

If Go were to get C++ style templates it would greatly reduce the fitness for purpose due to the increased cognitive load and the likelihood that developers will resort to "cleverness". Keeping "cleverness" out of your code is of the utmost importance when developing code that people have to understand.

I very occasionally do write code where every instruction counts too. Mostly on embedded platforms. Even there C++ is a bit troublesome because it is so easy for people to make mistakes. I'm not terribly fond of C++ there, but I can tolerate it. (Mostly I prefer to use C for embedded development and I really wish languages like Rust would be more ready for prime-time in embedded development)

You might like this presentation on the Curiosity rover's computer that goes into the OS, code (C++), and system design. https://media.ccc.de/v/35c3-9783-the_mars_rover_on-board_com...

Besides pointing that typeclass based generics (Rust, Haskell) have all those pros and none of the cons, none of the gains you claim are inherently lost on Java-style generics either. It's just that Java has funny language limitations, that do not apply if you remove the incompatible requisites from the language.

generics in Haskell are done via type erasure, so they definitely do not perform nearly as well as c++ templates, especially for things like numerics. I don't know how rust implements parametric polymorphism to comment.

I'm pretty sure there are mechanisms you can use in Haskell to reduce boxing despite erased generics. Specialize pragmas, for instance.

> typeclass based generics (Rust, Haskell) have all those pros and none of the cons

My understanding is that typeclasses incur a runtime penalty on invocations since the typeclass essentially carries around its V-table with it at runtime.

> none of the gains you claim are inherently lost on Java-style generics either. It's just that Java has funny language limitations, that do not apply if you remove the incompatible requisites from the language.

I don't understand how the last sentence of that does not invalidate the first. If you remove those limitations, what you get is not Java and would lose many of the other benefits that being Java has.

In Rust, type classes do not have to carrry that overhead; when you use them as a trait bound, you get monomirphized code. When you use them as a trait object, you do carry the vtable around.

For other languages, you may be correct in all cases.

And it's worth mentioning that the "trait object" form of generics is relatively rare in Rust code; there's a fair amount of friction to using them that makes them somewhat of a beginner hurdle considering how easy they are to reach for (which the future `dyn` keyword intends to rectify).

I disagree.

Yes, any approach to parametric polymorphism is going to make some tradeoffs, but the benefits of C++-like generics which you've enumerated (templates perform as fast as hand-instantiated types and the ability to have complex constraints) do not require you to simultaneously have the drawbacks which the C++-like generics have.

For example, ML-style parametric polymorphism has shown us that it is possible to mostly have our cake and eat it too for 90% of cases where you actually need to use generics; unfortunately only very recently (e.g. with Rust, even if its flavour of generics is still limited compared to what you can find in functional languages) ML-style polymorphism has actually appeared in a more mainstream language.

Well, ML doesn't have subtyping or C compatibility, so it's in a very different space. C++ had to deal with adding generics while also preserving all of the important other properties that make C++ C++. You can argue that those properties also aren't important, but they obviously are for C++ users.

I thought that most MLs did type erasure, like Java, instead of completely reified types, like C++. Although I know that F#, which is an ML that can reify types at runtime using the CLR JIT.

What is it about "parametric types" that wreak such havoc on error messages being intuitive? Is is it just the levels of indirection involved? If so is this also the reason for the slower/modular compilation? Thanks.

With C++ in particular, it's because type-checking happens only after a template has been instantiated, so the error messages you get are based on the instantiated types and not the original template.

I apologize if this is a naive question but could or could have C++ used type erasure at run time similar to what Java does? Or is there some technical reason it's not feasible in the C++ run time?

C++ sacrificed:

    -Fast, modular compilation
    -Straightforward error messages
    -Low cognitive overhead

That's precisely the wrong way to go on every count! That's a perfect example of designing for the machine, not for the humans! (I say this as someone who wrangles C++ in my day job.)

Do you truly think that no code should be optimized to run as quickly as possible over more human considerations? I think game engine developers might like to have a word with you...

Do you truly think that no code should be optimized to run as quickly as possible over more human considerations?

No. Of course not. It's mentally lazy to just push the lever all the way over to that side. By the same token, it's also mentally lazy to just push the lever all the way over to the other side. In some kind of design space over many variables, there are going to be some maxima.

A bit more general than making the common case fast, is making the common case fast enough, debuggable, and straightforward, with the option of making something faster when and where you need it.

I think game engine developers might like to have a word with you...

I would like to think so. I'm developing a cloud resident game engine for MMOs. (Just a hobby project, admittedly.)

All that said, those 3 decisions for C++ were 3 particularly egregious ones.

I went far down the rabbit hole of C++ template metaprogramming path when I built a game engine years ago. Besides enjoying the challenge, I was able to make some APIs much cleaner and more flexible. The problem arose when I realized over time that I was the only engineer in the entire company capable of maintaining and extending my "clever" design.

Later on I built our server backend in Go. As I built up the team I was continually impressed at how the enforced simplicity of the language made it easy to skill up new members and be able to understand code written months or years before.

With that said, Go absolutely needs generics or at the bare minimum variant types. Too often I was forced to sacrifice type safety or add a ton of code duplication where very simple generic support would've been the solution.

I'm content with the implementation of generics in C#. Copy it, move on, problem solved.

I'd love to see C#'s generics in Go. There is, however, at least one plausible reason to not do it as @ianlancetaylor puts it very well in the main GitHub Issue for Go generics:

> It is an essential feature of Go's interface types that you can define a non-interface type T and later define an interface type I such that T implements I. See https://golang.org/doc/faq#implements_interface . It would be inconsistent if Go implemented a form of generics for which a generic type G could only be used with a type T that explicitly said "I can be used to implement G."

The whole Issue is worth a read. Here's direct link to the above comment: https://github.com/golang/go/issues/15292#issuecomment-21008...

I find it a very weird reasoning, as inconsistency is everywhere in Go's type system.

Every standard type literally has it's own type system, as do more than a few functions. Structs, arrays, maps, slices, ints, ..., make, new, range, delete, ...

I'm not sure I understand why types would need to declare the interfaces they implement to function with generics. If my generic requires interface I and T implements it implicitly why wouldn't that just work?

Does Go's empty interface type count as a bare minimum variant type? For example, it is the parameter type for fmt.print().

>"To be honest C++-like templates are probably the worst way to do generics."

Could you or someone else elaborate on C++'s implementation and how it contributes to the negative effects which you've enumerated? Might you or anyone else recommend some links or literature on this? Lastly this might be a silly question but are generics and C++ templates strictly equivalent? Or are there some subtle conceptual differences?

To be honest C++-like templates are probably the worst way to do generics.

C++ templates can provide you with the joy of debugging exceptions for which no source code exists! Been there. Done it for my job.

It exists, just temporarily and it's machine-written.

C++ templates suck. They could have been tolerable but the way they are designed forces them to be awful.

The people designing C++ either don't write code every day (too far away) or they do nothing but C++ every day (too close) and have become blind to better, non over-designed language feature implementations.

It reminds me of the nonsense Java does that's clearly demonstrated in FizzBuzzEnterpriseEdition; a bunch of things that get in the way because one or more people were blinded by the language.

It exists, just temporarily and it's machine-written.

The relevant point is that it may not be available for you at debug time. I know empirically that it can not be available, and the programmer will have to infer it by looking at the involved templates.

I dont think it's fair to say "Java does" because that can all be done in any OOP language. If anything it's more about people who over engineer.

Or to be more succinct: If anything it's more about people. Extreme Programming took this into account. Left to their own devices, most coders tend to over engineer to impress their friends and coworkers.

"... there are other, a lot more saner and principled flavours of parametric polymorphism out there."

Could you point to some of those?

OCaml, F#, Haskell, Rust, Ada, Scala.

Haven't used them much (or C++'s at all), but (not speaking as a PL-designer-or-implementer), D's templates seem to be not bad. I could understand how to use them for some simple cases, at least, without much studying of the subject.

And I liked what Walter Bright (D creator) said in (IIRC, in a DDJ article): something to the effect that, after working on the design of D templates for a while, a sudden insight he had, was that templates were (just [1]) a way of parameterizing a function not just by value parameters, but by type parameters too.

[1] Not really "just", of course - there will be a lot to the implementation, but conceptually, that simple.

Edit: Links:

https://dlang.org/spec/template.html

https://tour.dlang.org/tour/en/basics/templates

https://dlang.org/articles/templates-revisited.html

> Not really "just", of course

For C++. Tons of languages do the type paramaterization and it solves more than 80% of the common problems. 80% of the remaining 20% can probably be covered with compile-time code generation (or runtime code generation), for which templates aren't the only solution (D, C# and Lisp have different examples of how to solve this). The ultimate remainder of problems are likely really obscure, and are probably indicative of code smells (e.g. VectorF<3>).

I knew this day would come. While this is a great start into the realm of Generics, to me, it just doesn't have that easily grok-able feeling that the rest of Go has. The syntax is kind of clunky and while I applaud the effort, I'm sure Rob Pike and Co. have been tooling around with this idea since the beginning of the language. It'll be great to see what shakes out possibly in Go 2.

It's really useful to have access to an eval() function when paused in a debugger. That alone makes this useful -- it'd be great to have IDE's like Goland take advantage of this to provide this functionality.

Goland's debugger can evaluate expressions when paused.

https://www.jetbrains.com/help/go/debugging-code.html#924cf9...

It must be some subset of expressions (or I must have set mine up wrong), because anything that calls a function (vs a simple conditional statement) appears to throw an error for me indicating that it cannot execute.

Some people that are complaining about C++ templates (and suggesting generics from other languages as alternatives) are missing the code generation aspect of templates.

Generics operate entirely at the Type level. For instance (using java)

  List<Integer> list = new ArrayList<Integer>();
  list.add(1);
  Integer i = list.get(0);

is under the hood equivalent to

  List<Object> list = new ArrayList<Object>();
  list.add(new Integer(1));
  Integer i = (Integer)list.get(0);

The differences between the two snippets are entirely at the type level - they compile to the exact same bytecode. Templates are strictly more powerful - they allow you to generate different code for each instantiation. The most obvious place where this is useful is when dealing with primitives that take up different sizes in memory. This is why you can't use generics in Java with primitives - you need Objects, because pointers to objects are always the same size. But if you want performance, so you want to be able to represent collections of primitives generically, then you need the power of templates because you need your code to compile to different things depending on the template parameters.

Generics don't have to be implemented with type erasure at the bytecode level. The CLR doesn't, for example.

http://www.jprl.com/Blog/archive/development/2007/Aug-31.htm...

Very neat project! Not sure what I'd use this for in practice, but it's a nice proof of concept for several things.

Definitely nice for learners to have an interpreter!

Bravo!

In my quick read, it wasn't clear if it would be possible to open a channel in regular Go code, and pass that into the interpreted side.

Just being able to call functions in normal compiled Go code would be enough I suppose, but direct support for cross-domain channels would be cool.

Imo, the generics are a downside, not a selling point.

Can I use this to interact with my Go projects' source code? I've tried other Go REPLs in the past and they didn't play nice with my setup (didn't like my symlinked GOPATH.) So I use a self-hosted playground during development sometimes.

stop infecting the clean simple language with that complicated and confusing mess. Go program in C++ or Rust, please. If someone needs to generate something, they can use Go generate tools or simple old m4.

Why the MPL license? Have you thought about releasing under the same license as Go instead/in addition?

Any time you see something like this. Similar to typescript or es6 transpilers it's a signal something with the original language is off.

Perhaps it's the lack of Generics in golang.

You are so right -- thats why in better languages like C# and Java you need an IDE to write half your code for you, and a multi-million line optimizing runtime to make it run fast.

I find you sarcasm insulting. Every language has flaws and I never said C# or Java were better. I'm saying that the fact this exists points to something needed in the language.

Besides the fact that generics is a up and coming feature in go 2 basically proves my point.

The project has several objectives--to bring a REPL for Go, to provide a macro system for Go, to facilitate experimentation with generics, etc. None of these are meaningfully served by "having generics".

Experimentation serves a purpose and that purpose is to see whether they can fill a gap missing in go.

Yep, that's the point :)

Comparing apples and oranges. Java and C# are verbose because of libraries and frameworks in the ecosystem not because the languages are inherently flawed.

Not sure what your point about the runtimes is? We shouldn't use or build highly optimized runtimes? Go's original runtime sucked and there's been a lot of work put into making it better too, I guess they should have just stuck with it?

>"Go's original runtime sucked and there's been a lot of work put into making it better too, I guess they should have just stuck with it?"

Could you elaborate on what the shortcomings of the original runtime were? Might you have any link regarding this? Thanks.

I think he's remarking about how the first version of anything is almost always suboptimal compared to later versions. I.e., it "sucked" compared to later versions, almost by definition.

The original runtime had far, far worse GC pauses. They concentrated on making sure they could make a good one later rather than getting a perfect one out the door immediately. There was also a series of crashing bugs of varying severity in the first couple of versions. Even then you had to go a bit out of your way to find one, but they're a lot harder to come by now.

Nothing that surprising, really.

If you look at Go's release note history: https://golang.org/doc/devel/release.html look towards the first few releases for the bugs they mention. For instance, here's a 1.1 fix: https://github.com/golang/go/commit/d72c550f1c7e13c323f4507b...

One of the advantages of not mutating the language too much is that there aren't very many things like this in the last few releases. I would expect the first 2.0 release to have a few rough edges even in the first non-beta release.

You're conflating "optimized" with "optimizing". The observation is that Java and C# depend on a _very_ sophisticated runtime (including a JIT compiler) to perform comparably to Go which has a comparably simple runtime. I don't know that the point is a strong one; I'm just trying to clarify the conversation.

You mean outperforming Go, actually.

https://www.techempower.com/benchmarks/#section=data-r17&hw=...

Not to contradict your point but be careful on putting too much weight on microbenchmarks for they are implementation sensitive. For example, here you can see Go on par, if not faster, than other stacks: https://www.techempower.com/benchmarks/#section=data-r17&hw=...

I consider a good number of modern platforms fast enough for most uses. When tight requirements arise, which is rare, I resort to PoC implementations rather than microbenchmarks. There's a lot to more to consider than raw performance.

I don't, actually. One microbenchmark isn't conclusive (never mind that Go is outperforming C# according to The Fine Microbenchmark).

Go looses against Java in all of them, which was also mentioned by you.

And there are plenty of variations to chose from.

https://www.quora.com/Is-he-looses-an-arrow-grammatically-co...

Thanks.

There are not "plenty of variations"; they all measure roughly the same things--web framework, Postgres driver, JSON library. Go's postgres and JSON libraries are probably not well-optimized. Even if this isn't the case, at best it means that Java has a higher ceiling--it doesn't tell us about whether average Java code is faster than average Go code, nor does it tell us about how difficult it is to optimize code in each language such that it approaches its respective ceiling.

Here are some more microbenchmarks that show Go and Java at about the same performance, and the rules of the game prohibit idiomatic Go optimizations (hence the poor performance on binary-tree; regex-redux is slow because Go's regex library is famously unoptimized).

For real-world programs, Go and Java are in the same ballpark.

This is literally just a REPL for Go. Are you saying that Go is "off" because the compiled language doesn't include an interactive mode by default? By that logic, there is also something "off" about Rust...

To be fair, it's a REPL that adds macros and generics.

Reply on news.ycombinator.com