Why We Switched from Python to Go (2021)
softwareengineeringdaily.com
softwareengineeringdaily.com
"I just wanted to learn a new language"
Now, the points they've made are valid, Go concurrency beats the pants off python, and with gofmt, there is less of a debate about style.
Te bit about finding a team is pure horse cock. As they point out there are a bunch of C++/C programmers about. There are even more python programmers too. Go is a tiny pond compared to the big three (c, java, python).
Your assumption is wrong, we did not pick a different language just for fun or because we wanted to learn something new :)
Python was just not the language we needed for our APIs, we tried hard to make it work but eventually we looked at alternatives that would allow us to keep things simple and performant. End result was better latencies, a much simpler architecture (less async/MQ work) and processes that would eat a fraction of RAM and CPU compared to Python.
If you can program in any language, you can program in Go. There's so much less to learn than even Python (perhaps especially Python when you consider all of the different packaging tools, testing frameworks, sync vs async, decorators, metaprogramming, etc).
> "I just wanted to learn a new language"
Strong disagree. If the actual purpose was learning new things, they'd have chosen a language which does things a little differently than all of the mainstream. At least Haskell or some Lisp...
The fact they chose Go shows they care primarily about the practical side.
People want to learn a new language for reasons: perhaps they want to learn new concepts, perhaps they want a language that's faster, perhaps they want a more mature ecosystem, perhaps they want to increase their employability. There aren't people who "want to learn a new language" as their ultimate end goal.
Quite possibly! That would be consistent with my point that they chose Go for practical reasons, rather than to learn for the sake of learning, as KaiserPro was suggesting.
If you want to learn for the sake of learning, you'll learn a different paradigm, rather than rehash the same thing with a slightly different flavour. Also probably your productivity will tank.
It doesn't have to be a single variable equation. I can learn something both to learn it and for practical reasons.
The original comment I was replying to said their reason was "I just wanted to learn a new language", which is what I disagreed with.
and
> finding a team
Are connected. More people out there would be willing to work in Go than Python, even if more people know python. First, a large portion of python writers aren't backend devs, whereas pretty much all Go users are. Second, Python has a very bad rap as a backend language, and for pretty good reason. Third, if someone doesn't know Go, then can still get up to speed and use the language extremely proficiently within a few months, unlike any of the other languages you listed.
Which combined with a new code base is very stressful. but this avoids the main point, which is
> Python has a very bad rap as a backend language,
its not a front end language really(discounting the transpiled JS web stuff). I know people dont like it, and that's fair enough, but its as good as virtually any other language for "backend stuff". Its perfectly possible to write low latency high scaling code in python, just as it is in go and rust.
Despite what people say, code is just a logic brush, how you hold it, and how toy paint with it determines how pretty the picture is.
Yes there are inherent defects that each language has, but once again part of being proficient is know how not to blow your feet off. If Goethe can make wonderful prose in german, you too can make good scalable backend code in python.
Depends on what you mean. A tiny and simple HTTP handler brought up on lambda, sure, Python will be as good as anything else. Anything larger than that, and no, it isn't. Python just doesn't have the semantics to do what Go and Rust can at scale. I run a service that uses 4 threads in Python and it's horribly slow and unreliable. I worked on a service in Go that used hundreds of concurrent go routines and it was reliable as hell.
> Despite what people say, code is just a logic brush, how you hold it, and how toy paint with it determines how pretty the picture is.
That's a great analogy when you consider painters use different paint brushes for different parts of the job.
> Yes there are inherent defects that each language has, but once again part of being proficient is know how not to blow your feet off. If Goethe can make wonderful prose in german, you too can make good scalable backend code in python.
Sometimes yes. But accomplishing the same in Go is so much easier I don't know why you'd bother.
That is very anecdotal to the point where I’m not sure it is all that useful. Go is supposedly good at parallelism, yet it has plenty of warts to the point where it is not at all more stable than any other mainstream language out there, so other than its pragmatism on channels, it really is not better than what you can do in Java or C#.
If parallelism is truly important for you than the actor model will be hard to beat in terms of reliability.
Also, go accomplishes the same with much less verbosity because it's the default operating mode.
So yes, you can do everything with Java that you can with Go (not true with Python) but it's a bit like using a semi-truck to commute.
Theoretically, sure. If you _really_ know your stuff.
However, you can naively pack hundreds of thousands of go routines into a single thread in a go routine after a week or two of using it.
I'm not saying go is better, I'm just saying Java copied this abstraction for a reason: doing it right is hard
Think of it this way, if you could all the python code on github and ran it though autopep8, what fraction of the files would be changed?
Generally in the go community, contributions, patches, code, etc that's not formatted with gofmt is considered wrong/broken.
How do humans that disagree with the One Format deal with it? Do they self-select away from Go?
> Think of it this way, if you could all the python code on github and ran it though autopep8, what fraction of the files would be changed?
How does the format evolve over time as the community discovers improvements (if it does), and what are the implications for code on github?
Like much of go, there's the "go" way to do it, and the language simplicity reduces the chance of multiple programmers taking multiple approaches that are all different enough to require wrappers to interoperate. Sure your pet feature from some other language may be missing.
The language (and gofmt) have evolved. In particular since the parser is externally usable (by gofmt, IDEs, and other tools) you can actually use that tool to rename variables, functions, etc without having to worry about getting a REGEXP right. So when there's the occasional incompatible change gofmt can parse code into a tree, make a transformation, and then back info code ... including comments.
It's relatively common (in my experience anyways) to have commit hooks in git run gofmt on go code. Generally go's been very compatible, none of the issues like Python2 vs Python3. I'm sure some avoid go for these reasons, but I lump them in with folks that reject python because of whitespace/indenting.
This pretty much sums it up. Gofmt isn't perfect, but everyone uses it and deals with it. It's included with the language so there's no point in trying to create a "competitor".
I can’t imagine ever picking a language based on this feature and seems weird to even mention it.
This has never caused me a minute of headache and the only time I care is when conflicting styles are used in the same file.
I always thought it was kind of dumb of Go to care about this but don’t care enough to affect my language choice. I don’t use Go but am open if it’s a fit for a project.
I like that Python has that variability and let’s developer make decisions as they see fit.
I don’t want to talk to people about tabs and spaces unless we’re already friends and having a few beers.
You get far better tooling, excellent performance, the ability to write extremely concise code (as long as you don't apply mainstream coding standards/style) and a better range of libraries for most situations than Go. More chance for re-use too (i.e. same library for mobile apps, websites via web assembly, backend and apps for all major PC platforms).
Also - the leaders of both Python and C# are coworkers - they both work at Microsoft.
That said, I want to learn Go properly.
I don't think there is any difference between Go and Python when it comes to debugging. Support for both is widespread and high-quality.
Being able to evaluate code while in the middle of a breakpoint is extremely useful.
You don't need to say it doesn't matter. Your language has features Python doesn't have, there can be tradeoffs
Everything you said above plus BIG community. C# is the top #4 language on StackOverflow, it's even more popular than PHP and HTML.
By comparison, Go is not even among the top #50. I'm not saying Go is a bad language, just bare numbers. Every problem you're going to bump into with C# - has been arleady discussed and resolved already.
P.S. latest .NET-core beats C++ in some benchmarks I love that
I'm interested in this aspect!
I discovered that I was able to write short scripts in PowerShell in a style I find readable, or I can add extra stuff (parameter annotations, mostly, IIRC) and it can stretch to look like the C# I see in Microsoft API documentation. I find that the idea just gets buried.
So I need to revisit C#, to find out how much can be effectively elided.
Not all code will be though - we have a lot of Java-minded people in the community (who port Java libraries/frameworks) and they tend to be... ugly..
Pulling third party packages would be instantly flagged and be put under high scrutiny.
Meanwhile with C# and .NET’s ungodly framework size you can just cruise along because it’s made by the creators of the language.
Both languages are great in this regard-- you can do a whole lot without 3rd parties.
It’s all going through an SA&A process that will check your c# framework as well as your random package downloaded from the interweb.
(based on the decent amount of audited government work done using tons of third party libraries. Just try doing anything in data science without third party packages)
> Revel, Iris, Echo, Macaron and Buffalo seem to be the leading contenders.
If you're talking about MVC-era frameworks these are fine. However a lot of companies are using Go for microservices so I would highly recommend looking at https://goa.design/ and https://gokit.io/
This was written 4 ½ years ago.
This article is dated "Wednesday, March 3 2021" but appears to be identical to https://getstream.io/blog/switched-python-go/ dated Oct 17, 2017 and updated either May 14th 2019 or Sep 15, 2020.
Both say "Early this year, we switched Stream’s primary programming language from Python to Go", and there was HN discussion about it on Oct 17, 2017 at https://news.ycombinator.com/item?id=15495562 (397 comments).
Perhaps the author took something he had written before, and really did update it before republishing?
For example, you cannot now "Swap out True and False". That was changed in Python 3, and while many were still using Python 2 in 2019, it wasn't so true in 2021.
I also wonder how numba does with the "simple_gauss(time)*popularity" expression.
IMO modern languages like Go and Java are pretty easy to get into and you can use them for a first implementation without really sacrificing development time relative to Python, as long as you have invested the time to learn those languages and the associated tools. (C++ is not like that unless you have made a very serious investment in setting it all up.)
I’m not trying to say that any of these languages are better/worse, just that they are differently suited for particular situations (program size, team experience, etc)
In mye experience Python programs are not more difficult to reason about than equivalent Java programs. To the contrary an over-reliance on certain design patterns and ubiquitous, inescapable OOP complicates Java code bases, while the static typing is so weak it affords little safety compared to e.g. Python.
Worth keeping in mind a that a Python program will be about half the LOC of a Java program doing the same thing. (See the reference section here [1].) In other words you can get further with Python before passing the complexity threshold.
Bugs are also proportional to lines of code [2], which is another element that favours Python over more verbose languages like Java.
1. https://svese.dev/comparing-java-and-python-is-java-10x-more...
2. https://amartester.blogspot.com/2007/04/bugs-per-lines-of-co...
> Python is not maintanable above 50k to 100k lines of code and because of that people consider this code bases very large
Note that this is a speculation, not a conclusion, the article is not very thorough, and LOC is mostly used because it is convenient to measure, not because it is what we are trying to measure. I'd personally consider 50k a "medium" code base, and 10k is "small".
> Worth keeping in mind a that a Python program will be about half the LOC of a Java program doing the same thing.
LOC is a confounding variable.
> Bugs are also proportional to lines of code...
Not supported by the citation. The citation measures bugs per line of code, and finds that for 500 kloc of code, the average number of bugs will be somewhere between zero and 25,000. That's a very wide range.
To be clear, I'm not really trying to fight against Python or for Java here. I'm just giving my reasoning for why I might personally choose one or the other. I think that the idea that you would switch languages because Python is slow is actually far more situational. You might have ten reasons to choose Python or Java, runtime performance may only be one of those factors, it may not be heavily weighted, and in some cases, Python runtime performance can be extremely fast (I do a lot of NumPy stuff... it's great).
The relationship between programming language and code quality is, at best, a difficult relationship to study. It's hard to make any kind of direct statement like "using language X results in more bugs than using language Y" and back it up by evidence, even though we believe it to be true. Individual statistics which relate some variable to LOC is not useful in isolation.
See: https://arxiv.org/abs/1901.10220 (a reproduction of an earlier result, that invalidates many of the conclusions)
> while the static typing is so weak
Both of these are straight up false. Java can be written with static functions and single-depth inheritance trees just fine. Hell, the direction the language took for quite a few years now are pretty much this with records, pattern matching, etc.
And while Java is not Haskell, it has a moderately strong type system with quite a good generic implementation making even some more advanced functional patterns expressible.
Your claim on LOCs is also false (as well as most of this kind of claim) Java has at most a tiny bit of constant overhead, it won’t make it anywhere close to 2x.
I think you're spot on. Is some ways, it is a tooling problem, as type annotations require mypy (or some other type checker), and enforcement via e.g. CI.
Crucially, retrofitting them to a codebase is a difficult and tedious problem. On the other hand, it's relatively easy to do for new projects, but so is choosing a different language.
In my experience, this is because a lot of Python programmers are trying to write Java code in Python, carrying Java's awful and unreadable paradigms into a language that has no reason for them.
The other issue with Python is that it uses indentation for scoping. Combine that with the fact that it is super easy to mess up indentation when you are moving code around via copy/paste and it is super easy to change the meaning (ie accidentally move a statement out of an if block).
Using indentation for scoping is great for small projects and for beginners. However, once you get to medium or large projects, having the extra redundancy of curly braces is reassuring.
Python is a great scripting language, and it's a great language for hackers that work on their own personal projects. It's not a great language for enterprise microservices on teams of like 20 people. The growing pains are real.
Copy/pasting large blocks, yeah, you can mess it up. But any editor worth using will let you select several lines of text and hit Tab or Shift-Tab to add/remove an indentation level, so fixing it only takes a couple seconds.
...or has equivalent functionality that doesn't require the mouse ;)
The ability to select text in this way has existed in Windows for at least 20 years.
Professionally, I'm at the whim of whatever environment my employer uses. I've used Linux (CentOS), Windows, and Macs. If I'm doing Python development, then it's pretty OS agnostic unless I'm using a Python module that compiles to a native binary, in which case Windows is certainly a nightmare.
Not using a programming editor? I select the text and hit Tab or Shift+Tab, not exactly rocket surgery. The lack of redundant noise characters pays off every single day, where as odd indentation problems are a once-a-decade issue.
If you're doing data science things, it's hard to beat pandas/numpy. I get that those are popular in that community because the barrier to entry with Python is low. People who are just looking for a tool to solve immediate problems would do well to reach for Python.
The problems start when you try to write more complex things. Then you run into the weaknesses in tooling, testing, and performance. People would do well to skip Python and go right to another solution.
I say this having spent half of the last decade working on Python projects. If not for data science and academia keeping it alive, it'd be disappearing along with Perl.
That's quite a statement. Go on, for most of the other languages, show me a web framework that's better than Django.
Python absolutely has deployment challenges, but the performance is Good Enough™ and the speed of modelling and maintaining and accessing databases in Django's ORM is so much better than anything else I've found. Its Admin interface too is superb for same-day turnarounds on little CRUD projects. Nothing comes close.
And I've written quite complex systems around Django. Ones that are multi-headed websites, APIs to physical hardware IO, ANPR, card readers, ID printers. Python hasn't let us down.
I think that’s subjective, so take this with a grain of salt. I do have a different opinion though. Also, my point isn’t to argue, but to encourage consideration.
I’ve used Django and Python quite extensively in the past and I would say that, for me, Phoenix/Elixir and Ruby on Rails are both better web frameworks than Django.
The reason I feel this way is that Phoenix/Elixir, for example, has much better dependency management and tooling than Python/Django (e.g., Mix is vastly superior to Pip, in my opinion).
Also, when I was using Django in the past, even the core team suggested using a different directory layout than is generated by default. This meant that every Django application I worked on had a completely different project layout. Both Phoenix and Rails are more opinionated regarding their project layouts. Some may consider this bad, but the benefit is I can go to any Phoenix or Rails project and instantly be productive because I know where all my controllers, models, views, templates, and contexts are going to be. I also like that Phoenix is much faster, more scalable, and makes better use of hardware resources than either Python or Ruby.
I think Python and Django are excellent, but I’d place them at #3 in my top-three list of web frameworks. That’s just my opinion though.
What I took issue with was the suggestion that "most other languages" have a better web framework. I'd wager most other languages don't even have a web framework. And even amongst those that do, there's some real dirt out there, and much worse examples of packaging, tooling, etc. Two Python frameworks, Django and FastAPI would be in my as-objective-as-possible top 10.
It was a silly thing to say.
90% of the performance gains I've achieved are by optimizing/reducing large DB queries, or adding an index to the DB.
That's just like, your opinion, man.
> Tooling sucks.
Not in my experience. Lot of tools could be better, but I would not say there is a lack of un-sucky tooling at this point. Poetry and pytest in particular are largely excellent to work with.
> Dependency conflicts are common.
This has been a problem historically but it's leagues better today, especially with lockfiles.
> There's no test framework/runner worth a damn.
Um, what? Unittest (stdlib), Pytest, tox, nose, hypothesis, schemathesis, and a few other lesser known ones.
> Web frameworks are inferior to those in most other languages.
FastAPI is pretty amazing. Flask is well regarded as a good intro framework. Django I'm not a fan of personally, but many love it.
> If not for data science and academia keeping it alive, it'd be disappearing along with Perl.
And machine learning. And web dev. And sysadmin scripting.
The built-in unittest module is essentially a clone of JUnit, so it can't be much worse, and most of the remaining limitations are fixed by pytest which people seem pretty happy with. What test frameworks are you comparing them to?
The thing I want is a good monorepo solution for Python. There's at least 3 for JavaScript.
As much hate as JS gets, its package managers are also much better than Pip.
In order to manage complexity of Python as you do your big projects, you really need to do "enterprisey" patterns like DI. Even though it's rare for Python programmers to write like that.
Re: test runner: pytest is pretty good
Then if you need to do something like build a distributable archive for your app (say, to deploy it to lambda), you can use Bazel's dependency graph to get only the packages that the app transitively depends on.
-r package_a/requirements.txt
-r package_b/requirements.txt
Is bazel also a good choice for managing the virtual environments of my developers? Do you have any examples?
At my place of work we use some rules [2] to build virtual envs from Bazel dependencies. It's been a great stop-gap for us and allows us to use traditional Python tooling (e.g. PyCharm) with Bazel-managed dependences.
[1] https://github.com/aspect-build/rules_py [2] https://github.com/cedarai/rules_pyvenv
Okay, so what solution? What do you recommend? I'm guessing if you're qualified to make statements like your first paragraph, you can bring your wealth of skill and experience to the table and show us what we should be using.
Is it really?
Unskilled developers can be very productive very quickly. Skilled developers can write some really beautiful code with it.
Has issues with the ecosystem - it's a headache with M1 and far too many Python projects are just broken. The latter is because it's used by scientists / students so is more a function of the community than the language.
I use it as a scripting language, which it excels in as long as you don't have many dependencies.
Go: fast and cgo is really powerful. The language itself is a little verbose for my tastes though - it makes the simple verbose. The package management story is weird (import the source code from github). Better than having the dependency hell of npm or Python though.
Python is extremely convenient when you are trying to be productive. It is a good general purpose tool, especially if you're doing anything web related. My project starts in seconds and requires no compilation. Stack traces and tooling for the language is far beyond most other popular languages IMO. The standard library is excellent, but also the quality of popular python libraries is unmatched by other languages in my experience.
Python is not super fast at doing some stuff. If you're doing that stuff all the time it may not be a good choice. But for most of what I do, python is really good enough. A user will not notice a few 10s of Ms for that web request, and usually that isn't even the scale anyway for this type of stuff.
It's not true for two reasons:
1. In practice writing a prototype in one language, then completely scrapping it and rewriting it in another language almost never happens for fairly obvious reasons.
2. The "slow bits" of a Python program are rarely neatly concentrated in a few parts of the program that you can rewrite in a fast language. The whole program is slow, so there's no small part that you can rewrite to make it all faster.
3. There are other languages that are just as easy to use as Python (easier even!) but are also fast enough that you don't need to rewrite anything in most cases. Typescript is probably the best option at the moment but there's also Dart, Lua and probably others I've forgotten.
Python is popular because it's seen as very beginner friendly so there's a mountain of content out there teaching people Python. Also the REPL makes it good for beginners and for science.
The very words such as bottleneck/hotspot indicate that it is common to get the most benefit by optimizing small parts of the code.
I had enough of the rewrite in C experience that since then, if the language doesn't come with a compiler on the box, either JIT or AOT, I only see it as scripting language for sysadmin work or learning purposes.
Also Vignette, remember it?
When we dediced to move away from TCL/C, after exhausting the performance options, J2EE was going to be the next version.
Due to a twist of fate of being Microsoft partners, we got invited to take part into .NET launch product line, and that Java rewrite became a .NET one instead.
The original founders, after going through these experiences, went out and created OutSystems based on the learnings, naturally in .NET and later a Java backend was added as well.
It's had its deficiencies as well, exacerbated due to lack of investment, but that is changing. These are being knocked out one by one, every release.
Adding in more bullshit just gives me more code I have to read that I may not actually be interested in while also spreading out the code of interest.
In the horrific example code posted, all that should be there is the URL, that it is a GET, and parsing of JSON (maybe into struct). Or in other words, "fetch(url).then(resp => resp.json())" with some validation code elsewhere, though in this case it's just one field so validation of the whole response is probably unnecessary.
I've actually written a lot of code and have used it for my own projects for some time, just need to get the release out now.
#1 Performance
#2 Performance (to make the list longer I guess?)
#5 Fast compile time... which is an advantage over Python how exactly?
#7 Strong ecosystem... which brings us to disadvantage #1 - lack of frameworks (?)
Also re #3 and #6 - I found learning Go to be relatively painless, coming from a Python background. It was helpful to have much more experienced teammates and to have some opinionated pre-existing infra (e.g. tests, lint, deploy, general app architecture). The lack of frameworks is good/bad -- more decisions to make (though the standard Go libraries seem to almost always be good enough) but if everyone is mostly using some combination of modular libraries, it means that problems are easier to debug and search for on StackOverflow, versus needing to add if the error is in Flask or Django or some other bespoke framework.
Compiled code can have fewer runtime errors, that's a big win. I read this as golang is better than python because it's compiled and golang is better than Java and C++ because the compiler is fast.
And as far as type systems go, Go is one of the worst in mainstream usage.
I'd also count Python with MyPy, but having a mostly untyped ecosystem means you can't take much advantage of it.
Golang's typing system is fairly half-done.
Heck, you can even use HKTs in Python if you are willing to install a plugin/library.
All things that Java, Go, C# and others just can't be the best at because of the overhead or GC.
Am I missing something? Is there some way in which numpy is superior to, say, Linpack (other than not having to use Fortran to call it)?
Is there anything unique about Python that enables this approach? Couldn't Go, say, do the same thing? Or C++?
Network effect. I don't think anything stops most languages from doing most of what NumPy does (although IMHO even post-generics Go is actually a bad choice), but you have to compete with the existing NumPy. Competitors exist, but, well, the fact you don't know about them and haven't heard of them kinda makes my point.
In fact in my personal opinion Python is an unfortunate choice for NumPy. NumPy is so big it is basically its own thing, and data scientists could have learned to half-program in almost any modern language. (Rust might have given them fits, and C++ would cause some issues, but most languages would have worked for them.) Unfortunately, they settled on a language with weak types, which makes the documentation really annoying to use because it's very hard to tell what will work with what. (I've been dipping into NumPy and pandas over the past few weeks, the docs are infuriating... it's like, they make it obvious there's something that will do what you want but it's quite difficult to backengineer what that "something" is if you don't already know, because nothing has types anywhere. I can already tell you just sort of "get used to it", but it would be easier and faster if there was some type indications somewhere.) And perhaps even worse, as you scale up the size of what you're doing in Python, anything that isn't accelerated becomes more and more mindbogglingly-slow, because when you're in Python you are in a terribly slow single-CPU language banged together with the highest-performance multi-core if not GPU-based code in the world, and the gap between those two things just keeps opening larger and larger. Knowing when you're in slow land and when you're in fast land takes expert-level knowledge and at times source code reading. I'm glad I'm just visiting, I think living there would drive me insane.
C++ could give you equal or better performance, but it's not anywhere near as friendly to beginners as Python. Plus, the value of REPL-based programming for prototypes and experimentation is hard to overstate.
https://www.youtube.com/watch?v=LoypcRqn-xA
Plenty of other frameworks which you have a specific framework is lacking in the Go ecosystem?
I think this would probably be more why they switched to Go, rather than why they switched away from Python.
Alternately, how fast are Python linters? It might be faster to compile Go than to run their linter (or whatever).
It is only a disadvantage if you want to have a framework.
I prefer frameworks the same way I prefer code linters and formatters. Which is to say they are annoying, but far less annoying than working with a team that spends obscene amounts of time discussing/rewriting code based on opinions. Frameworks come with their own opinions and can help remove a lot of those barriers.
It seems like only a couple of years ago when Java's speed was the butt of many a joke. The JVM has come a long way. As someone who doesn't work in Java and doesn't have a horse in this race, I find their turnaround impressive.
First of all, there are plenty of implementations to chose from, everyone is now talking about GraalVM, because the basic version is free beer, yet AOT has been a thing in commercial Java compilers for quite some time.
Nowadays OpenJDK and OpenJ9 offer JIT caches with PGO feedback across executions, well, OpenJDK one started in J/Rockit, and OpenJ9 traces back to Metrome/Websphere Real Time/J9.
My first large scale Java project was done in 2005, when our team at Nokia Networks started the transition of a C++ infrastruture used for base stations reporting into Java.
Did Java win against C++ in microbenchmarks back in 2006, not at all, yet Java 5 JIT compiler in IBM J9 and Sun's JVM already provided enough performance juice that we could perform such endevour, while keeping the system responsive.
Eventually some other issues plagued the project, however they were related with politics between the C++/CORBA folks and us bringing up the new platform.
But I think Java also still has the problem that it tends to use a ton of memory (unless you use it the fintech way, manually wrangling your data in arrays rather than using lots of objects).
My subjective impression is that garbage collected languages have gone through a bit of a rise and fall over time -- “GC is bad because it’s slow!” “huh, GC is fast now” “Java is the way forward!” “GC uses a lot of memory” “Rust [or Swift] is the way forward!”
It got the reputation in part because once upon a time Java programs were noticeably slow and heavy on resource use, no matter how many times Java fans posted micro-benchmarks of Java finding digits of Pi to "prove" Java was fast and we must all be imagining things. In actual fact, in the real world, it was slow.
I think improving hardware has more to do with its feeling faster now, than anything else. Hardware got fast enough that it no longer feels (as) slow. Plus its competition, as far as GUI programs go, now prominently includes Electron. On the server, we deploy tens or hundreds of megabytes of container or VM and associated overhead to run a 2MB program. So also, everything else has gotten far more bloated and heavy, making Java look better by comparison.
Is Eclipse fast yet?
1. trivial cross-compilation
2. native multi-threading (eg no GIL or multi-process hacks) that is easy to take advantage of
3. fast. An order of magnitude faster than Python in many cases.
4. easy to deploy. Most often just a single binary
To put it simply, with Go I ship a binary. With Python I have to ship my entire development environment.
That set of features isn't related to the language itself (though the language was designed to allow those features).
This is where Go is appealing, it's not the language itself, it's the properties it brings with it. Cross-compilation, multi-spec, reasonable performance and low memory footprint. Single binaries. And I'd add fast compilation times.
Honestly it needs more competition that offers those same properties, because I'd love to have more choice of language (syntax + semantics), but get the same properties. But for now, you have to use Golang if you want something that gives you all those properties and that's just where Golang differentiates itself.
The only languages I know that can compete with Go with similar properties are Nim, Zig, and Rust. I'd like to include Delphi / Free-Pascal, but cross-compilation can be a bit of headache at times.
Delphi incorporates a mobile solution that was bought for the purpose, called FireMonkey, that doesn't completely integrate with their desktop solution (VCL). Each new version of Delphi is making progress with integrating the different solutions more tightly, but we are still talking about payware that is priced above what most individuals would want or feel comfortable paying for.
Lazarus, for various odd reasons (or even suspicious reasons), their developers appear to be dragging their feet with making a mobile solution. They should be able to incorporate technologies such as OpenGL ES with LCL and their custom drawn interface, but don't. A 3rd party developer came up with their own solution, that works for Android only (LAMW), but doesn't integrate with the desktop solution (LCL) nor is a part of Lazarus. So for Lazarus, outside of desktop development, things get a bit confusing. However, if team Lazarus ever got their act together and took mobile development more seriously, they probably would have among the best solutions out there.
Rust I believe doesn't do true cross-compilation, isn't it just relying on you using virtualization like docker and other to do so?
There is a tool, "cross," that uses Docker as part of its workflow, but it's not inherent in Rust's abilities to cross compile.
1) Easy to use pointers. There's just pointers, not like C where you can have anywhere from 4 to 8 different types depending on your platform. And no, pass by reference vs. value are not the same as pointers.
2) Very simple to use concurrency.
Do you mean C++? Or am I confused?
As for wild and dangling pointers (I would argue the are the same - pointers that show somewhere they shouldn't), fair point. Of course one would expect this from a language so many decades younger.
But none of these are pointer "types" (far and near arguably are - iirc they determine the size of a pointer - but these four are not). I hope the question on the interviews are posed differently, I for one wouldn't know what you mean.
It’s not due to age, but due to having a GC and being a managed language. I really don’t get all the comparisons of Go to C, the former seems to want to sell itself as a modern C replacement but they are nowhere in the same niche (and that’s ok, there are very few places where a GC can’t work). They just operate by completely different semantics when it comes to pointers.
People make fun out of C++ all the time. Yet, Go has some extraordinary Gotchas of its own. You need to read "The Go Bestiary" to make sure you don't blow your head off.
I ask because I've started a side project in rust because all of its guarantees were appealing to me (I don't want to spend too much time on bugs, and my code uses some threads). I also need maximum performances. So I've chosen it. But now I have my compilation times that reach 45 seconds, so I'm slowly wondering if the go balance (towards fast compilation) would not be more advantageous; unless of course, performances are real bad...
Rust will have faster CPU and memory performance. It has no GC (well, other than RC). If you are IO bound, there won't be much of an appreciable difference, assuming you are using Rust's async/await for stuff. If you are using threads directly, then it will be a lot easier to make goroutines faster for IO stuff.
The long compile time of rust has a lot to do with it's LLVM backend, which is ultimately generating pretty fast code. You don't get that with go (well, unless gollvm lands). Go traded execution speed for compile speed by doing their own compiler/optimizer. Generally speaking, that doesn't really matter (hence the reason python is popular).
Make a decision, stick with it, ship it. If there's a real problem, change it after you have it up and running. There's no point in optimizing for millions of users if you have 0 users now.
that project was an excuse for learning rust. But now I understant rust a bit more, well, I'm a bit scare to wait 45 sec on each build for a side project (ie a project I do on my very limited spare time) :-)
but anyway, rust is really cool to work with (I have a python/C++/assembler/R background). The level of checking it does is really mindblowing (and frustrating at times :-))
[1]: https://fasterthanli.me/articles/why-is-my-rust-build-so-slo...
[2]: https://matklad.github.io/2021/09/04/fast-rust-builds.html
Thanks!
But Go is still reasonably performant and has a reasonably low memory footprint. We're talking C#/Java level performance with C level memory footprint.
Rust is the next level up in performance.
fwiw https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Logically, if you need maximum performance, you can't care about compile time.
It wasn't as hard to grok as C or Rust, it wasn't as slow as Ruby, Python or PHP. most of all, it it was designed to make use of multiple cores. Surprisingly, having testing and formatting built into the tooling is also a huge win after spending years debating and changing choices in scripting land.
Go is the new PHP/Node.js and Rust is the replacement for everything else. I know this is technically wrong, but that's how it feels.
For me, readability, simplicity and having compile time checks are wonderful. Go feels like a better Python. I still enjoy using Python but "runtime is funtime" even with type hinting and linting and everything else.
I am also a sucker for good tooling and a pleasant dev experience. Go has fantastic tooling. That being said, it has its warts and weak points. If I need to write something quick and dirty, I prefer Python. If I need to parse JSON and I don't have an easy way to generate client code, I prefer Python.
There are some quirky bits in Go I don't particularly like e.g. the reversal of parameter type and name in function signatures. . . that just confuses me because I typically am using 2-4 different languages in any given week and Go is the weirdo in this regard. :)
I would be thrilled to see Rust become the dominant systems programming language some day. I'm planning to learn it as soon as it looks like the right choice for me - for most of my work, Python + Go + bash (I love and hate bash) are more than sufficient.
I'd love to see a s̶t̶r̶o̶n̶g̶ statically-typed Python that could throw type errors at compile time. I've only taken advantage of the duck typing once, and the way I did it was a massive code smell and I ended up refactoring it out anyways.
Try running it through mypy with all the strict options. If you can get it to 100% pass and don't try to "cheat" the system too much with dynamic casts, I think you will find runtime type errors to be very rare.
Err...yes. I forget that "strong" and "static" are not the same and that "strong" and "duck" are not mutually exclusive.
I'm bringing someone up in Go and they forget to initialize some map or struct: `&Foo{data:map[int]int{}}` and crash the app.
However, that's completely missing the point: Golang brought CSP into the mainstream when most were still using CPS to scale beyond OS threads. The kind of people who rewrite services to get performance improvements and then write blog posts about it aren't gonna learn your complicated language (just like they're not gonna spend their time messing with gevent and profiling to scale their python): they're makers not PL researchers. Google rightly noticed that there's a huge gap between existing mainstream offerings on the ease of use-performance spectrum and filled it with a DSL for programming network services. It doesn't need to be fancy, it needs to allow someone with CS201 knowledge to set-up the codebase so that people with CS101 knowledge can write sequential code that talks to the network without callbacks, yield/async/await or any of that incidental cruft.
That said, personally I also think that React kind of went downhill for a bit due to the hooks (after seeing a few projects become really nightmarish to debug due to render loops without clear causes for them, after people sprinkled one too many hooks in there).
Oh, and the Vue 2 to 3 migration is also a bit problematic because still many UI component libraries haven't been migrated over - currently actually using PrimeVue on a project because BootstrapVue still doesn't have proper support https://github.com/bootstrap-vue/bootstrap-vue/issues/5196
Programming is about managing tradeoffs to solve real problems, not building a perfect ideological environment to write code in.
If you like the high-performance backend, then Java, Kotlin, Scala, or C# are great choices.
It's widely used in industry and isn't going anywhere.
All the people freaking out about performance didn’t realize that even in the Python days, half of our datacenters pretty much sat idle or severely underutilized. Computing is dirt cheap as long as you’re not renting a 15-year-old CPU from a Cloud Provider for 100x markup. The other thing is that engineers would frequently build things that were just wasteful or unnecessary, and sure, Go sped things up, but you could have changed the design instead of the language.
But Go isn't a silver bullet. It's another tool in the toolbox and sometimes it makes sense to reach for Python, or Kotlin, or Swift, etc. But choosing Python when you should've chosen Rust / Go / D is a tough design decision to quickly come back from.
Actually it came out about 10 years ago, in one way or another: https://go.dev/doc/devel/release So it largely depends on what timescales you're comfortable with - some people switch stacks or learn new tech every 3-5 years, for others this cycle is slower or faster, also depending on the stuff you're working with.
I'd say that web dev moves quickly, embedded moves more slowly, the rest is somewhere in the middle for the most part, maybe AI/ML and certain DevOps aspects rival web dev speeds of innovation/churn.
> I swear in 20 years everything important will still be written in C, PHP, Perl, C#, Java, Python (how could I forget JavaScript).
Perhaps, i think languages like Rust will be pretty widespread, both in the Linux kernel, as well as many tools and whatnot, albeit in a different and perhaps more conservative fashion than currently (the "rewrite everything in Rust" craze). In contrast, most of the modern web based SaaS solutions will be dead, most certainly the majority of Kubernetes tooling out there.
Go might just be boring and useful enough to stick around and join the dreadful bunch of languages used in maintaining legacy code. It's funny, though, because while the languages themselves are more usable than they were 10 years ago, people's sentiments (including mine) are largely influenced by how many bad codebases they have to maintain: https://earthly.dev/blog/brown-green-language/
On an unrelated note, it would be pretty cool to have extremely slowly moving and stable languages, projects and tooling out there. To see maybe 3 or 4 releases in my entire lifetime and to have the next generation take over the burden. Then again, seeing how COBOL and FORTRAN are now, perhaps that sense of grandeur isn't worth the effort and old projects should just die.
Isn't that Perl?
Some of the nicer Perl software that I use currently is BackupPPC, which has been pretty solid despite the slightly subpar UI: https://github.com/backuppc/backuppc
And another interesting piece that I can think of was RemoteBox, which was pretty niche but still worked nicely: https://remotebox.knobgoblin.org.uk/?page=about
From the more popular packages, one should also mention exiftool, which was written in Perl: https://github.com/exiftool/exiftool
Probably also a lot of other pieces of software, though it doesn't seem like there's much of a large/active community around Perl, for example, have a look at: https://github.com/trending/perl?since=monthly and then compare it to something like: https://github.com/trending/go?since=monthly
In regards to other contenders, I think FreePascal/Lazarus should also be mentioned, since they're pretty stable, capable and perhaps some of the better ways to write desktop software!
Actually, that repo has lists like this for most languages: https://github.com/EvanLi/Github-Ranking
Hmm all the languages you listed have certain niches that I think will likely remain alive in the next 20 years, except I don't think Perl deserves this spot. I think anything Perl is a great fit at, Python will also be a great fit and (subjectively) python code ends up being strictly more readable than perl.
I would guess PHP also has a similar relationship to Python but the effect, I believe, is a lot milder as PHP shops seem to be more decisive about sticking with PHP. All evidence anecdotal of course.
Use MetaClasses to self-register classes upon code initialization
Swap out True and False
Add functions to the list of built-in functions
Overload operators via magic methods
Use functions as properties via the @property decorator
---
Who actually does any of that, particularly at the office?
No one has the time for shenanigans.
They are going to find this out the hard way.
And fast? Please. If your database queries are a mess, or if you have no discipline elsewhere, it doesn't matter what language you use - you will blow right past the "superfast" baseline.
- Wrote a distributed app in Python (it aggregated a few GB / day of info from several thousand servers). Performance was terrible, though I was doing all the right things (async I/O, etc.).
- Rewrote it in Go. That worked really well, and took just a few days. But management freaked out, forbade Go in engineering, and so:
- Spent several months rewriting the whole thing in C++. What a fucking misery. Just getting HTTPS working was a depressing experience. Package manager? No, you get to trawl the internet for semi-busted projects that sorta kinda maybe work, and then bend them to fit.
Go is the sweet spot between "Python isn't fast enough and never will be" and having to reinvent the wheel in C++.
Is this just cluelessness and management by familiarity with buzzwords?
They mentioned it was a rewrite that happened in a couple of days in a shop that uses C++ and Python (at least). We can assume it was a small to medium utility program.
Someone who can write C++ and Python professionally will get up to speed with Go in a few days. This shouldn't be a hiring issue. The cost of rewriting it again in C++ can easily be higher and apparently was a huge PITA. Go is not some obscure language, it is literally designed to be accessible by anyone with basic programming experience.
It pains me to read this, especially since Go seems to be a language that is extremely easy to adopt.
We realized it took about as much specialization and wheel reinventing as Rust, which offered way better developer ergonomics in ways that mattered to us and wouldn’t panic over things that weren’t easy to predict or caught by the compiler.
There were still tons of trade offs, but it worked well for us. Go is cool but I still can’t seem to figure out where and when I’d prefer it most of all… Especially the poor error handling and lack of safety. These things really drive me crazy.
https://medium.com/safetycultureengineering/an-overview-of-m...
(and yet it will do so, as you try to find other ways to make your code safe to release)
It's true that if what you're writing is an application server consisting of business logic and glue code that delegates all the "heavy lifting", performance won't be the first concern. However, even in this case, I think you'll find its easier to get sub-second response times in Go than in a language like Python.
This has been the cause of basically all of the slowness in Rails projects I've worked on, at least. I've seen queries taking tens of thousands of times longer than they should.
I don't think I've ever seen a Rails codebase that didn't have speed problems, but every single time it's been because DB access is being done very inefficiently. I've yet to see one actually bound by the computational speed of Ruby (I'm sure it happens, though) and it's considered way on the slow end of popular languages.
The problem's always 1) Doing shit in Ruby that the DB can do a thousand times faster (joins[!], calculating things based on a huge number of rows, mashing together pieces of data, et c.) with a better query, or 2) firing tons of identical-but-for-some-values queries in a loop(!?)
[EDIT] Actually, just as I hit post, it occurred to me that 2 is really just a flavor of 1.
This is a "wut?" moment for me. Why add this external factor, when it's totally irrelevant and the article doesn't even mention it?
I'm however very much reliant on Pandas and Pyarrow for custom ETL works. Does anyone have suggestion what language and ecosystem I can switch to? Preferably something simple that does not require cluster or JVM to avoid deployment hassles.
To both you and the peanut gallery. You should basically* never be using PYTHONPATH or OS packages. And there are several solutions to improve ergonomics of virtualenvs out there.
* I've never had a legitimate use case to use OS packages and only one to use PYTHONPATH, which is tot temporarily set it during tests in a repo with multiple independent sub-packages, coverage reports get a bit squirrelly.
I'm by no means a Go expert, but it felt like Go forced consistent error handling.
> Another issue is that it’s easy to forget to handle an error by accident
Does anyone have specific examples of error scenarios that skip through the cracks?
> While this approach works, it’s easy to lose scope of what went wrong to ensure you can provide a meaningful error to your users. The errors package solves this problem by allowing you to add context and a stack trace to your errors.
Isn't this effectively just having propagating error Object that you stack context into ?
On the one hand, I agree something like this should "be supported by the language" or rather forced especially since Go supposed to be opinionated, but on the other hand I appreciate having flexibility.
The go language guys could fix both of these easily - force the caller to check error or explicitly drop it (_) optionally if non-null. That would almost completely eliminate the problem.
I really like Go - I can pick up pretty much any Go code, from any open source project or any fellow engineer and read it with ease. No context, secret macros, implicits, etc. The code basically is what it appears to be. This is an enormous benefit in most contexts.
I don't want that because I don't want to litter my code with `_ = fmt.Printf()`
`Printf()` is a very common function. It returns an error. I don't care about an error from `Printf` and I don't want to make my code more ugly to avoid bugs I'm not writing in the first place.
I've been writing Go code since before v1. I really can't recall a case where I wrote a bug because I forgot to handle an error.
I get how you can forget something you do once a month.
I don't get how you can forget to handle an error in Go when that's something you do every couple of lines of code written.
Most people, I admit, do not review their dependencies, but that in and of itself is a problem.
Your fmt.Printf issue is trivially addressed by implementing PrintfOrDrop() which wraps that - and if this was the way Go behaved, you can be certain the standard library would add these variants for conciseness.
These stories seem to have an undercurrent of mistrusting the judgment of your expensive and carefully chosen staff. It’s not doing anyone any favors to force an expert to laboriously write out the same junk a novice would have.
From Python??
An objective comparison would need:
* Authentication n Authorization
* Rate limiting
* Configuration loading
* OAuth integration
* Websockets
* Retry policies
* HTTPs (yes… all the way to uService)
* Testing
* Benchmarks
* Cloud tooling (integration)
* C interop
* Service Caching
* Middleware authoring
And many more…