Learning Go by porting a medium-sized web back end from Python
benhoyt.com
benhoyt.com
Go really feels opinionated around the wrong things, in order to claim "simplicity" as a feature:
- No generics just mean you're going to be handing interface{} all the way in your stack, it makes things more complex and less readable for no reason (and less safe)
- error handling which is essentially string-based, in 2017? Give me a type system, please
- gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features, saying "running it in an automated process is not supported", despite all github projects already having it in their automated CI suite
- go test is good, but in the end it is found lacking
- dep, well, if you don't mind dumping the code for all your deps in the source tree, it's probably ok, but it solves a deficiency that the language should not even have (see next point)
- stability: the language itself is fine on that, but the ecosystem really isn't. The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.
Depends on what you do. I've written a good chunk of go code and interface{} is the rare exception rather than the rule, usually employed where a user might supply arbitrary types (ie a unmarshaljson like function)
But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.
>- error handling which is essentially string-based, in 2017? Give me a type system, please
You can put anything behind an error:
type IntErr uint64
func (i IntErr) Error() string {
return strvonc.Atoi(i)
}
Additionally, nothing in Go forces you to actually use the error interface (sans stdlib).
You can invent your own Error interface that uses ints. (Won't work with most external libraries but nothing is stopping you)
Panic and Recover can to my knowledge both handle things that aren't errors; recover returns a interface{} that you check for value and type. You can put anything in there.
>stability: the language itself is fine on that, but the ecosystem really isn't.
A lot of projects focus on stability. If the API is breaking a lot they usually use Gopherpit or gopkg.in to ensure compatibility in the future.
In my experience, breaking the API is not something well used libraries do lightly (libraries with no or little use do break it sometimes since they lack the usage to refine it)
If you need more, you can use vendoring.
That's not really true. If you want an SQL database to back your website, then you'll deal with interface{} with Go's sql package. If you want to gzip your output, then you'll use interface{} with your writer.
To be fair, I think interface{} gets an almost unfair amount of hate. While it is fscking annoying when passing around core types (I hate having to use switch clauses just to inspect the type of an interface{}!!!) I do love how interface{} is used for complex structures. In fact there are a few areas of Go's standard library which I wish used interface{} more in that respect (eg Go's `file` struct should be an interface{} so I can create custom methods for os.Stdin/out/err)
No, they are interfaces, but they are not interface{}s, the interface that is met by having no methods and is therefore met by everything. Nobody is complaining about the use of interfaces that declare a useful set of methods, and are meant to be used when the set of methods is all the target code cares about. They may complain about some of the edges around them (no support for contra- or covariance), but that's not the complaint that is being discussed here.
The complaint is specifically around interface{}, which is a type that means nothing, and so everything fits into it. It means that if you are a function that is receiving an interface{}, you know nothing about the argument to start with, and will have to either pass it along to something else, or examine it via type assertions or in the worst case, the reflect library.
As a multi-year Go programmer, I agree that A: the "true" interface{} that is the meaningless type appears less often than people think, and that if you program is shot through with it, you are either working in a domain where Go is not suitable (complicated math processing, IMHO, for instance), or you are programming some other language in Go, and need to learn how to use Go properly, but also, B: it does happen that things that would be well-typed in other languages will have interface{} show up, which means you're going to sacrifice runtime safety and some performance, though the exact impact depends on what you are doing.
(I say the "true" interface{} because there's also some places where interface{} shows up, but it doesn't hurt you. For instance, the standard JSON library accepts an interface{} to say what data structure to parse the JSON into, and the code that implements that is somewhat complicated, but for the most part, that interface{} doesn't bother you, the Go user. In theory you can encounter run time errors if you pass in something that doesn't work, but in practice, if you pass in the same type every time, which is the natural case that is going to result if you just program normally, it's not a problem. In general, `interface{}` showing up in the incoming parameters of a library is much less concerning than an interface{} showing up in the return parameters for a library. In the former case, even though the language does not enforce type safety, you can enforce type safety by how you use the method; in the latter, you are stuck with an interface{} in your code with all that implies.)
That appears to be a definition unique to you that I have never seen from anyone else before, including the Go designers. io.Reader is an interface, no braces. Its full expansion would be interface { Read(p []byte) (n int, err error) }, and it is incorrect to substitute that for interface{} as that is a very different thing. Structs that implement the interface are, well, structs that implement the interface. interface{} is specifically reserved for discussion about the empty interface.
For instance, do a browser find for "interface" in https://golang.org/doc/effective_go.html ; you will find the documentation frequently referring to "interfaces" and that interface{} is reserved for the empty interface.
For the final proof of this, note how confusing it is for you to be reading the Go criticism as being about the use of interfaces-in-general in Go. Why would that provoke such a reaction? The answer is that people aren't talking about the feature, they are specifically referring to the number of places interface{} appears, the "Any" type, the "I don't know what's in there at all" type. It's not the use of interfaces in general, it's the way that suddenly one goes from a decent enough statically-typed language to a not-all-that-great dynamically-typed language when interface{} appears. (I disagree with the HN gestalt about the severity of this problem, but I understand it and still agree it's a non-zero problem.)
With this statement, you've shown you have close to 0 Go experience.
* interace{} - a place-holder for a value of "any type". It means "a value that satisfies the empty interface", i.e. an interface with no methods. All types implement it. To do anything with the actual data it encapsulates, you have to do a type assertion to unbox the value. The type assertion will panic or return an error if the type assertion is incorrect. This is something like C's void , or Java's Object.
io.Reader, io.Writer - these are interface types that specify a set of methods that must be implemented by another type in order to satisfy that interface. These are akin to C#/Java's interfaces, but are less restrictive in the sense that any type in Go can implement them, not just "classes". E.g., http.HandlerFunc - a function type that implements the http.Handler interface.
True. But for most real world scenarios I only ever use interfaces at the boundaries between APIs and they quickly get asserted into a type after being passed.
Conceptually they're a bit like passing marshalled data between APIs in that sense.
That's IMHO a great way to think about when it's Good™ to use them. Yes, there are times when you just have to use the empty interface type but it's really not as bad as everyone makes it out to be.
I love Go because it is explicit and because explicit types make everything easier in the long run. I get tired of seeing Go written like Python/Ruby/$otherDynamicLang with overuse of interface{} and/or reflect.
How do you deal with the lack of sets and trees? I believe those two data structures are frequently almost indispensable - even in the most boring mostly-CRUD web projects. And it's either type assertions on {}interfaces or go-generate based specialized implementations. Both feel odd to me. (Am I missing something?)
I haven't found a need for trees in most of my projects and where I did I implemented it when necessary.
You can however, if you need to, separate the code and data of a tree by using a data adapter. The tree only stores a unique ID for an object which you can receive using the adapter, which would be a simple map or slice.
But as mentioned, I very very rarely needed them and not yet in any CRUD web project.
Python packages are installed via a separate tool so I'm not sure what you're getting at here.
Including github in the import path is a good idea. It makes it so you always know where the source code for any package is. This consistent approach empowers sites like godoc.org. I don't know how many times I've tried fruitlessly to find documentation for libraries in other languages, but in Go the developer who created the library gets it for free. I wish more languages did this.
Do you remember the left-pad fiasco in npm? Out of the box you'll have similar problems in Go, but at least you'll get them when you work locally, not when you spin up a new node. And to fix the problems you end up having to do the same thing regardless. Either you vendor your dependencies or you setup some sort of caching middle-man proxy.
Go's dependency headaches aren't the fault of the language or the ecosystem - Go actually has a really well crafted module system, specifically designed to fix major performance issues in languages like C++ and Scala. Rather they're a reaction to the problems latent in the "easy" solutions and simply switching is just trading one set of problems for another.
Where is a useful question to answer, and nice to have consistent across an ecosystem. But What is also a very important question to answer, and tying What to be mutable and dependent on When is... highly unfortunate.
This could be one reason why that is the case: https://xkcd.com/927/
That Go solves some problems for programmers outside Google is a side effect of being a general purpose programming language. It is not a primary business goal except in so far as it raises Google's bottom line, creates a hiring pipeline, and keeps its employee moral off the floor.
It's not necessarily all about peak efficiency for specific enterprise applications.
A Golang noob here. Can you (or anyone else) expand more on this? Are there any links that I can read more about this?
> No Generics > Error handling > gofmt > go test > dep / dependency stabilityI mean, if you lose an arm you probably stop trying to pick stuff up with your missing hand after a while too, doesn't mean it's not better to have two arms.
One thing you did not mention is very varied quality of the standard library. I’m still baffled about ”standard date” crazyness. There are huge warts in the sql interface etc
My collegues used to joke golang is a great solution to google’s problems (for others, not as good as it arguably could have been to put it mildly)
- error is an interface. If you are returning strings most of the time, you're probably doing it wrong.
- Wasn't gofmt originally executed during compile time? I'm not sure why it changed, but it suggests that such automation fell apart in practice.
The fact that people can claim that C#, Java, Scala, Kotlin, Haskell, Rust, Swift have "ridiculous" implementations of generics sounds like a very bad excuse.
The Go team has, until recently, been quite open about not being confident enough in their own abilities to implement it well in Go. Maybe to the point of absurdity, but they were explicitly warned by designers of other popular languages (specifically some of those on your list, I believe) to not be cavalier in how they add generics.
That doesn't mean nobody on earth can implement generics well. I'm sure if someone from the C#, Java, Scala, etc. teams, with the necessary expertise, wanted to step in and contribute generics for Go, it would have been a welcome addition. At very least for the community who would have jumped all over a fork with generics. However, I do not see that code anywhere. You cannot force people to do things they do not willingly want to do.
Now that the Go team has had enough time to put for the effort to learn more about generics and the research related to them, they feel more confident about being able to add them in a somewhat reasonable way and have officially announced that they will be added in the near-ish future.
That's the problem, I see the following often suggested instead of Go:
Java - maybe if you use Spring Boot you can get around some of the boilerplate, but there is a ton of baggage in the ecosystem.
Clojure - This is really popular where I work but LISP + Java Ecosystem turn a lot of people off. Also, some people really want static typing (maybe Spec is good enough?)
Node/TypeScript - terrible concurrency, good for throwing something together that works rather quickly, good for cutting corners. Terrible memory, cpu profiling. As a professional node developer in the day job I could go on and on, but at least the type system is better than Go (too bad the concurrency story is so abysmal)
Elixir - not popular, but at least it's on BEAM VM and has lots of libraries. Not statically typed (yes you can use dialyzer). Never done anything with it, but apparently deployment is a pain.
I personally write JavaScript mostly because its ubiquity. It's good enough, until it's not.
The alternatives vary according to the issue you want to tackle, so I would be hard pressed to recommend a particular one. And go does fill some vacant spots in language space.
Using interface{} instead of generics is a bad idea, you lose type safety. It's messy, but code generation is your other alternative, sort of like your own templating system. I've used go's built in templating code to generate type safe Go code from things like SQL table schemas. I'd highly recommend that route over naked interface{}
Error handling is string or type based, that's up to you. I declare specific error types for my functions, like FooError or BarError, the caller can then switch on err.(type) and respond appropriately. Alternately, you can define concrete error objects and return those, and make comparisons like "if err == FooError".
I'm with you that dependency management is terrible, vendoring is a poor workaround.
Google operates on a mono-repo basis, as I understand it (from speaking to ex-googlers.) All libraries etc. etc. all under one big repo. If you think about how that would appear to end tooling consuming it, the approach taken with imports makes sense. It's lousy for the rest of us outside of the google mono-repo ecosystem, but it does make sense.
Only one story comes to mind where a programmer had a legitimate problem using a high-level language and solved it with a lower-level language, and that was Armin Roacher, who solved a significant performance issue using Rust: https://blog.sentry.io/2016/10/19/fixing-python-performance-...
Can you think of other stories? Please, share.
I encourage all to be more skeptical about potential gains and realistic about who is actually gaining.
The story I see over and over again as a startup CEO consultant.
Developers learn a new language that is faster to develop, safer and faster to run than existing solutions (which work but are of increased technical debt) and successfully & quickly port the existing solution to the new tool.
The company doesn't really benefit from it apparently because they never scale enough or because they lacked a target so they had their developers working on sth small instead on working on the company's products.
This skill is highly valued and the developers leave for an other company while the original one struggles to continue.
Do you think that it may be the case that the problem in all these is not within the developers or tool? Furthermore, shouldn't the CEOs and the CTOs know better and to a better job?
Most often only in the eye of the developers with no proper evaluation.
"Furthermore, shouldn't the CEOs and the CTOs know better and to a better job?"
Yes.
Would that it was that simple. Most of the time I've seen this it's been more like “writing new code is fun, fixing a bad design in our current code is too much like work” without factoring in the cost of rewriting, testing, optimizing all of the code which wasn't a problem relative to simply replacing the hotspot which was.
Yes, sometimes a codebase is so bad that there's nothing of value to be preserved but that's pretty rare and unless it's some turd from an acquisition or consultants the odds are extremely high that the broken business culture responsible for the first bad codebase will cause just as many problems the next time around.
That does happen but in my experience it's far less common than fanboyism or poor architectural/analytic skills — things like spending months rewriting a Python program in Go for a single-digit percentage gain because someone on HN said the magic pixie dust would make it faster and nobody checked whether it was I/O bound, spending most of its time in OpenSSL, etc. I've seen people propose rewriting code in C rather than optimize a database query because Real Programmers use hard things like C whereas SQL is beneath them. (I wish I was making that up)
To be clear: I'm not saying that Go isn't a perfectly fine choice, only that our field is more prone to fads than we like to admit and most of the things which cause bugs or make people wait are decisions higher in the stack.
There are more than enough Real Programmers or lacking soft eng who are not experienced or knowledgeable enough to know when to use C/python/go/sql/etc but again for me the problem is either whover hired them and whoever is managing them.
I also more than agree with the last sentence.
As far as I can tell this engineer just wants to stick "built stuff in X" on his resume. He doesn't have a good reason to not use our existing stack. If I were the manager I'd demand more technical justification.
I'm just learning Go and I'm itching to find a project to use it on. It's super hard to justify it though when C# or Python would work just as well and those are languages that we already use.
If working as a programmer means sticking to C# or Visual Basic or Python codebase and just working on adding new features or new business logic to your application for years and years, many engineers will not find that fulfilling. The best engineers will always want to learn about new tech and approaches (dev ops, containers, PaaS, automation, new languages and tools, AI / machine learning etc).
You cannot blame them for leaving. What you should instead do is provide interesting challenges for them, if they want to use some new technology, work on finding a justifiable use case that management will approve for it.
I understand that for a company that treats tech as a cost centre this approach makes sense but then they can't wonder when best engineers keep leaving to proper tech companies that do actual R&D and where they can become best engineers they can be.
You need to allow your best programmers to learn new tech and progress in their career or else they'll eventually leave for greener pastures and you'll be left with mediocre people who are content and don't want to learn new things.
It's like that analogy I have heard over and over again.
CFO asking CTO: What happens if we invest in developing / training our people and they leave?
CEO replies: What happens if we don't and they stay?
The JVM as far as I know is still the runtime king, faster then Go.
The question to be answered is why are you porting it? Fun and giggles? Hoping to achieve better performance? Scale? Portability? Correctness? Simplicity? Support?
I'd say most of the time its for fun and giggles. Which I think is as fine a reason as all the others, but when its costing someone's elses money, maybe you should be more considerate.
People want to switch away from python because of all the pain they feel from it.
I mean, I get it, it's easy to pick up and use. Coroutines and channels are useful, and make easy to write async stuff.
But god damn, it's so dull and boring, there's no joy in it. We're programmers, we're supposed to extract abstractions and express them in the code we write. And IMO your language should definitely try to help with that.
Instead, Go wants you to do err, value = ..., if err == nil { ... } on almost every single line. You can't adequately compose stuff. Well, you can try, but you quickly run into its ugly, verbose syntax for anonymous functions or lack of generics.
Maybe I'm not getting something, maybe it's just not for me, I dunno. But next time I need high performance compiled language, I'll use Swift or maybe even Rust. Also I think Scala Native is somewhat stable already, gotta look into that.
This example actually seems to be the opposite of an optimization. There's 50% more code to maintain and there appears to be almost no appreciable benefit to compensate for that.
In any case, switching to a compiled language and static types should bring some benefit, in addition to the gain in speed.
Zero mention was made of bugs caught or customer-noticeable speed improvements so I'm inclined to think that there was actually no benefit to having static typing.
Given that Go is similarly verbose to Python, probably 100% more code.
Premature optimization would be something that makes the code more complicated and more difficult to follow but produce better performance.
Writing the same code in a fast compiled language is just the default thing that you should be doing when you are not doing premature optimization.
Writing in a slow interpreted language is more like premature "deoptimization" (if that's a word).
If the fast compiled language is equally expressive and equally fast to write then no.
Those things are rarely (if ever) the case, though. It's certainly not true in this case. Go appears to be about 30% less expressive.
What I am talking about is the static type system. You can't express that in Python, despite how much more convenient some of its features are.
pessimization?
But the code doesn't have to become more complicated for it to be premature optimization. It literally just means improving something (generally performance) before there's a clear need.
> Writing the same code in a fast compiled language is just the default thing that you should be doing when you are not doing premature optimization.
I assume you mean "rewriting" here, since we're talking about porting existing code. Sometimes it's the best use of time, sometimes it's premature.
The reason premature optimization is evil is that it consumes a lot of time and makes the coder more difficult to read and reason about.
If there's no clear benefit to this, then all the work done to optimize it resulted in _negative_ value. There was no gain, only time wasting and more complicated code.
An example of where performance doesn't matter: a piece of code is only executed once every 10 seconds and it finishes executing in 5ms. Spending time to reduce its execution time to 1ms does not produce any tangible benefits.
An example of where performance does matter: a piece of code is running all the time and it takes 500ms. If you can reduce it to 20ms, it's totally worth it.
IME thinking about language performance at all is premature optimization; language performance almost never makes the difference, other between-language differences will swamp any gains from language performance.
> (...)
> why were you thinking of writing it in Python at all?
They simply have no awareness of how slow it is, or how important performance is.
> IME thinking about language performance at all is premature optimization
Are you serious? Python code is about 10 times slower than equivalent code in Go.
And 98% of the time it doesn't matter. For most business problems that you'd want to solve with a computer, for modern hardware, Python's performance is more than adequate. It's really not worth worrying about.
If you have a server side application that you hope to be successful then the code will be running all the time and you will pay the price literally because cloud hosting services charge for CPU and RAM usage, which will be off the charts if your server is in Python.
Not to mention all the time you will spend firefighting the performance issues that will inevitably arise. Which also literally costs money in terms of developer salaries and opportunity cost (you can't spend that time to develop new features).
Your hosting fees are a tiny fraction of your expenses. Development time is worth a lot more.
> Not to mention all the time you will spend firefighting the performance issues that will inevitably arise. Which also literally costs money in terms of developer salaries and opportunity cost (you can't spend that time to develop new features).
Scaling is a nice problem to have. If you get to the point where you actually need to improve performance, you'll be able to afford to spend time on it.
Only when your requirements are low.
Things get very expensive when you start requiring real performance.
> Scaling is a nice problem to have. If you get to the point where you actually need to improve performance, you'll be able to afford to spend time on it.
You don't know that. If you're a startup, it's very likely that you're still not profitable at that point.
There are often mitigating factors, though.
And switching from an existing language to use a new one that is possibly more optimized carries costs that have to be considered. If the new implementation shaves a few percent off some time or other resource scale, and the sum of that savings over the life of the program is more than the cost to do development and maintenance it's worth it. But that often isn't the case.
But why not get all the gain? Well, sure, if performance is the only thing you care about. But it usually isn't. You might also care about networking, or multithreading. Would I rather write that code in C, or in Go? Is it enough better in Go to be worth that last 10% of performance? Arguably, yes.
When someone says "X matters", it doesn't mean that they are saying "X is the only thing that matters". It is not correct to conclude that they should, for consistency, be saying "turn the X knob as far as you can". Instead, interpret it as saying "given your other constraints, look for a sweet spot where X is somewhat toward this end of its range."
That's why saying a compiled performant language should be default is not logic I understand. Default should be whatever you need for your requirements. If you don't need anything special, just go with what you already know well.
And, as I said, "Why not use C all the time by default then?" does not actually follow as the logical endpoint of the argument that your trying to answer, so making that your response really doesn't contribute much to the advancement of the conversation.
But if you're writing server code, I think choosing an interpreted language like Python/Ruby is the single worst mistake you can do.
Once you write a well designed, well tested, and feature complete piece of software, you can deploy it and forget about it unless the server it's running on breaks.
Go isn't special in that regard, unless there's something that makes Go easier to write, test, or deploy, which might be the case, but you haven't supported that.
My technical motivations were more about moving away from Hibernate. Arguably there were plenty of reasons for leaving it as it was, though -- if I had known enough to be able to solve certain data access complexities back then, Grails was in many ways a better fit for the problem. The experience in a (more-or-less) functional language wouldn't have hurt any either.
The business motivation was simple: Java/SpringMVC programmers are (or were then) far more plentiful, so adding/replacing tech resources was made much simpler.
Overall Go feels to me like some version compiled PHP with better concurrency support.
...and simpler syntax and without the quirks. But that's one of the best descriptions of Go I've seen so far.
- simple (so not Scala, Haskell, Perl 6, etc.; no Rust either, unfortunately)
- clean (nice syntax, preferably Python inspired, but consistent)
- modern (generics, some functional features, string interpolation, etc.)
- decent concurrency/parallelism story
- good IDE, preferably supported by the core dev team
- compilation to a (possibly static) native binary
- decently big ecosystem
- and a big enough community of contributors or commercial backing from at least 1 stable entity
- also preferably didn't have a lot of historical baggage
From what I gather C# might actually be close to this, 2-3 years from now, if the .Net Core transition happens smoothly and the OSS community adopts it... Who knows, maybe even Kotlin 2019 or so?
Go is showing itself to be a good compromise especially with the simplified nature of deployment and runtime stuff. It’s like a safer, higher level C, with a decent runtime library. That has so much mileage.
I’ve learned one thing as well over the last 15 years of using .net as well. An IDE is a monolithic brain damager. If you have to do something outside the scope of it, you’re shot. Give me an extensible text editor and shell.
Now I’m going to duck for cover!
Crazy attracts crazy, now you know.
Mono has always supported "compilation to a (possibly static) native binary".
Windows 8 introduced "compilation to a dynamic native binary" on the Windows Store via the Bartok compiler used in Singularity.
Windows 10 adopted "compilation to a static native binary" with .NET Native, based on Midori compiler toolchain.
The .NET Core team has CoreRT for the same purpose.
C# used to be that but I don't think that's the direction anymore.
No language ships a cross-platform GUI toolkit in its standard library.
In a couple years, .NET Core will be even more stable and will surely have convinced the stragglers in the community to adopt it. (I'm seeing this is already mostly the case, because everyone likes not being locked into Windows.)
I'm a pretty big fan of .NET Core (coming from Node.JS) and I really do expect it to grow a lot in the next few years as people discover how easy and sane it is.
I expect they want a native binary (possibly even statically linked) so they can trivially distribute the artefact without requiring that the VM be installed or bundled which is a PITA, in which case "compiles to bytecode" is useless.
Just looked at wikipedia examples and the code is full of special chars and shortened keywords. Doesn't look nice or simple to me, tbh.
As it stands, I would rather bet on OCaml.
All commercial JDKs had this option in one form or the other, and many of them are still around.
If you want to try this for free on open source projects, Excelsior JET is one option.
Well, there was GCJ, but I think that hurt more than it helped.
- much simpler than Scala while still being much more expressive than Java
- clean syntax, with a higher consistency than e.g. Java
- Generics, Higher Order Functions, Multiline-Strings, Destructuring, ADTs, Pattern Matching
- Coroutines (experimental but production-ready)
- IntelliJ Community Edition (or Ultimate)
- not yet but will be possible with Kotlin Native
- has its own ecosystem but also has great interoperability with Java
- stable for quite some time, big community
- well, a little bit for reasons of Java-interoperability; on the other hand
Most of the stuff in Kotlin looks more like less boiler plate (data classes) than "more expressive".
Little things that are a apin in Java and elsewhere are quick and natural in Kotlin:
- nullable types are smoother than Options - null checks are relatively painless - the stdlib works with you, not against - named parameters and optional parameters means no need for builders - etc. Etc.
"nullable types are smoother than Options - null checks are relatively painless"
Might be the case, but Options are more expressive in that I can combine them and restrict them with other type classes to express larger constraints. Reusabilty is higher e.g. with Monad transformers like OptionT.
So nullable types are less code, but also less expressive as building blocks of larger constructs.
They might be also less expressive because they conflate two concepts: Not initialized and not-there, whereas Option expresses not-there and _ (in Scala) expresses not-initialized.
"named parameters and optional parameters"
Not sure how this more expressive?
* Top level declarations
* Sealed classes
* Coroutines (in Java this is currenty only possible with Bytecode manipulation)
* Inlined Functions and Reified Generics
* Covariant Collections
* Tail Recursion
What of these would you call "more expressive"? I'd agree with sealed classes, they express a limited set of possible classes. Perhaps tail recursion as I can express problems in way of recursion without stack overflows. I don't think coroutines are more expressive than Futures/get, only a runtime optimization for many concurrent executions.
callbacks -> futures/promises -> coroutines
Because callbacks are a completely different style of code, your async looks totally different. Futures and promises are better, but still make it difficult to ever break out of the async context, coroutines make the difference between async and normal code nonexistant, and allow clean escape from an async context.
Interesting. I thought the lack of proper tail recursion is a JVM limitation. Hence why Clojure uses loop/recur to handle it by translating it to a loop rather then as an actual tail call. Which means things like mutually recursive functions are not possible in Clojure without blowing the stack.
How does Kotlin handle that? Is it internally rewriting to a loop or trampoline, or did they find some way to actually make proper tail calls?
Yes. You add a `tailrec` modifier to your function. The compiler then checks if the last op in the function is a call to itself. If it is, it gets replaced with a loop. If not, you get a warning that `tailrec` is ignored.
It's implemented with some bytecode hackery and hence limited to inlined functions [0], which makes it significantly less useful.
Funny how a misguided design decision from the Java 1.5 days (running generic 1.6 code with a 1.5 jre) still haunts us today...
[0] https://kotlinlang.org/docs/reference/inline-functions.html#...
It's as fun as writing Python (with a similar syntax), but it has essentially all the goodies you want from Lisp when you need them, runs as fast as equally optimized C, produces standalone native binaries, _or_ standalone JavaScript if that's your thing.
It should happen soon (TM), but from what I understand it will be just another version, just this time renamed to 1.0 - this basically means you could start using it more seriously even now.
> except the ecosystem/commercial backing
If this is not the most important feature on that list, here is one more vote for Nim. Give it a try, it might (pleasantly) surprise you!
Nim itself does not come with an IDE, but does include nimgrep and nimsuggest which underlie intelligence and other features in every IDE setup.
There’s also c2nim which will generate the bindings for you (though it might need some hand holding if the header files make extreme use of macros)
Once you get used to it, though, you'll probably want to write idiomatic F#.
- arrays with slices (that could also be used for strings) and Cilk Plus array notation [1]
- maybe actual string type with separate concatenation operator (possibly space as awk uses and C for literal strings) it could probably be folded to improved arrays
- standard SIMD vector types so that most use for operator overload disappears
- cleaned up standard library
- actual macros so the remaining use for operator overload disappears
- module system (so you would generally not need make and could easily compile to static binary)
- already you can build with static musl libc [2]
- something like libdill [3] or libmill [4] in stdlib
- there are multiple good IDEs, debuggers, analyzers
- one of biggest ecosystems and communities
- historical baggage mostly cleaned up
basically go is missing from your list is generics and string interpolation.
What about Java? Its easy, modern in the way you describe, has a good concurrency story, multiple great IDEs, can compile to a static binary with commercial JVMs, or experimentally in OpenJDK9. Really big ecosystem. Huge commercial backing. It just maybe fails at historical baggage and syntax.
Kotlin fixes those last two. And Java 10 will also fix a good chunk of the syntax complaints.
In all fairness though, I think you're focused on nitpicks that have no practical benefits. You're asking for a language with a better user experience. Which is great, as it can make programming more pleasant, and I'm all for it.
That said, I'd focus on language changes that provide functional value. Haskell adds a powerful correctness layer at a low expressiveness cost. Go adds a useful CSP concurrency mechanism with green threads, as well as being friendly to containerisation. Erlang and co add a great deal of reliability features and massive parallelism with first class actors. Clojure gives you powerful metaprogramming, state of the art immutability and a similar CSP layer as Go, while maintaining the access to Java's large ecosystem. Kotlin adds null type checking, and better functional primitives. Rust gives you safe memory access without the performance overhead of a GC. Chapel allows for super parallel computing. Scala gives you OOP with the safety layer of Haskell on it. Nim gives you faster python, with type safety. Crystal gives you faster Ruby with type safety. Etc.
Those are the languages that challenge the status quo, and have a chance at bringing real value in terms of the average output they result in when used to program with.
A slightly better UX for Java or Python is a hard sell. Its like the famous Unix quote: "the most dangerous enemy of a better solution is an existing codebase that is just good enough." - Eric S. Raymond
Java, C#, Python, Ruby, C++ are all good enough, even UX wise, to be displaced by something that doesn't offer compeling functional value. At least, in my opinion.
I love that Go is a language that can be mastered in a matter of months; that it has such a simple "get up and running" story; that its tools mostly do the right thing by default; and that almost everything is straightforward. In most other languages, I have to figure out what build system to use, how to script it, how to specify and download dependencies, what testing library to use, how to actually run the tests, how (if it's possible at all) to get a static binary, etc--and then I usually have to learn a complex language on top of all of that. For the most part, Go just works out of the box.
I think folks also underappreciate Go's runtime--it has a fast garbage collector and an awesome goroutine scheduler that all get statically compiled into your application binary. Mostly, I want a functional language that compiles to Go so I can still leverage Go's awesome tooling, runtime, and deployment model, but with things like generics and algebraic data types. I've started prototyping it too, and so far it's coming along smoothly (modulo all the things I'm learning about building a programming language). On that note, there is some effort going on in the Go community to build a VM for Go, which would probably make an even more attractive compilation target, at least from a toolchain perspective. Even if my project doesn't take off (and it likely won't), I think this is a low-effort way to address Go's most significant complaints.
[^1]: Well, "most" judging by the significance many of Go's critics place on semantically versioned dependencies
If you have to test file io or DB interaction, you can either do the adapter pattern or, as I prefer, just actually use the filesystem and DB but with some test infrastructure around it.
Example:
> type Bar interface{
> Walk() (int, error)
> }
>
> func Visit(b Bar) (string, error) { ... some code ...}
Hope that helps.
One extra thing that is the major motivation for me to go to Go is the ability to deploy as a single God damn binary and not deal with dependancies and pip. pip and counting on OS repositories may be the right thing to do, but it is just too cumbersome. I have an air gapped setup at work, and there is no easy way for me to deploy Python applications - because all projects just say pip instal this and pip install that. It would make my life so much easier building something on my laptop, scp the binary to servers and expect them to "just work".
[1] https://conda.io/docs/user-guide/tasks/manage-environments.h...
I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one.
Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?
EDIT: Yeah, just tried it again, it rewrites the shell files so everything works again. It even uses the same python version as you used.
conda create -n myenv python=3.6.3
conda install -n myenv scipy
conda install -n myenv numpy
conda install -n myenv pandas
source activate myenv # To make use of the environment.
source deactivate
conda create --name myclone --clone myenv. # Clones the environment.
du -sh /Users/asampath/anaconda3/envs/myclone
du -sh /Users/asampath/anaconda3/envs/myenv
813M /Users/asampath/anaconda3/envs/myenv
Fat environments. But, this works out just fine in our NFS mounted storage in cluster.
If you use packages from conda channels (which are repositories of packages), then it should work flawlessly. Usually basically everything is available from either the official channel (anaconda) or some community channel (like conda-forge). At least in the data-science domain. I'm not really familiar with the python web-development scene, so not sure about that.
Edit: actually once I had to copy an environment from one machine to another (because of connection problems), and I seem to recall that worked too, but I don't think this method is officially supported.
Snatching defeat from the jaws of victory :)
But I'm not sure who to believe, the other reply to the parent comment says that copying things around is actually possible.
The philosophy is to think of Python as a compiled language and then everything works.
The CI system make the "image" and that gets pushed. In our case we make a virtualenv but you could also use pyfreeze.
Works like a charm.
Some languages have even more powerful tools for that sort of deployment. One of the best examples I've seen is distillery [0], an Elixir package which allows for hot upgrades. That means you can just copy a single a file and upgrade without having to take anything down.
Like Turbo Pascal 4.0 in 1987 or Modula-2 in 1978, as two possible examples. :)
The Delve debugger still doesn't let you run functions and I miss all the autocomplete fun with ipdb. I hope they can make go a little more friendly to debugging in general.
>However, I’d seriously consider using Go for larger projects (static typing makes refactoring easier)
Assuming a fully tested codebase and a relatively sane runtime type system, I'd much, much rather maintain the shorter codebase than the statically typed one.
Having less code makes refactoring easier.
I agree that brevity sometimes aids refactoring, but there are many parameters that have to be weighed in, and often I find that the benefit of static typing outweigh whatever ignoring the type or checking it at run-time might offer.
I've not found this to be true, and tests that do type checking are a waste of time in any language.
Bugs that are caught by static type systems are almost always caught by a combination of behavior tests and a sprinkling of sanity/type checks on border code (e.g. a module's publicly facing API).
>you'll find out at compile time what you did wrong
In a fully tested code base that is regularly tested the distinction between compile time and runtime does not matter a lot.
By contrast n code bases in dire need of tests (which is more common than generally assumed), the distinction is absolutely vital.
Then you go asyncStuff.Request on it, and it blocks. There are some quirks but I remember that it worked all right.