Iris: Fast back-end web framework for Go
iris-go.com
iris-go.com
>Why creating yet another http package instead of optimizing net/http?
Because net/http API limits many optimization opportunities.
What I think could work is a pure in memory database with server combination. Something like redis for websites.
The selling point of Redis for me has been its simplicity. It has a fairly small set of features in comparison to other databases, yet extremely powerful.
[0] - http://oldblog.antirez.com/post/redis-persistence-demystifie...
(Disclaimer: Postgres guy, so I'm likely not impartial)
For read performance if your dataset fits in memory PG is just as fast as Redis.
The main attraction of Redis is in-memory writes with async persistence. Trading durability and availability for the increased performance. If you give up this then there is precisely 0 reasons to choose Redis over PostgreSQL.
FWIW, you can have that in postgres as well, cf. synchronous_commit = off
> Trading durability and availability for the increased performance. If you give up this then there is precisely 0 reasons to choose Redis over PostgreSQL.
As a PostgreSQL developer, I'd say that performance / latency / jitter can still be reasons to choose redis over postgres. The per-read overhead in redis is often a good bit lower than in postgres, and there's less variability in response times.
https://muut.com/blog/technology/redis-as-primary-datastore-...
Counter point of a real world implementation where they tried to make it the primary data store.
They resorted to weird hack-y shit for something I use MySQL or Postgres for on a regular basis. So have most people who built stuff on Redis like this:
> I use Redis as the primary and only database. > Additionally, it has data persistence reliability on par with Postgres and other battle-tested databases[0], so couldn't be happier about that.
Thanks
That being said, unless you're getting a massive number of requests, using a specialized HTTP library seems like overkill.
According to the README, there are plans for it in the future.
It might not be perfect, but it's common ground.
There's a bit more detailed bench here: https://github.com/smallnest/go-web-framework-benchmark
(seems to be missing a data/SQL/ORM layer)
Usage docs for rest api: https://kataras.gitbooks.io/iris/content/render_rest.html#us...
[2] https://github.com/gocraft/dbr
Given that interacting with a database is at least as common a webdev task as, say, i18n, I thought it was worth noting that this framework does not have a db solution of choice.
I have tried xorm and gorm (besides database/sql) and while I settled on gorm, they both have some fundamental design flaws:
* xorm onInsert/onUpdate hooks are designed the wrong way. Last time I tried, hooks received object instances by value and not by reference, meaning I could not actually update things before hitting the database.
* both xorm and gorm have schema-generation capabilities, but have you looked at the schema they generate? Last time I checked, gorm-generated schemas had basically no referential integrity whatsoever, and no foreign key was being generated.
But I guess it's only time before a well-done golang ORM shows up.
Go is somewhere between Python and C. Go has a garbage collector, scheduler, and runtime type information (aka reflection, aka introspection). Like C, Go has "value types" whereas everything in Python (or even Java/C#) is a reference (this gives more control over memory layout, generaly less indirection, and generally less work for the garbage collector). In this sense, Go performs similarly to Java for serial tasks.
For parallel and concurrent tasks (e.g., web servers), things get more interesting. Efficient concurrency in C is hard, and efficient parallelism in Python is hard (async IO makes efficient concurrency easier, but it's not widely used as far as I can tell). Go's goroutines solve both of these problems by providing a lightweight threading mechanism that abstracts over both OS threads and async IO (I/O is always async in Go, but there are no callbacks, promises, or async/await). These lightweight threads (goroutines) can be dispatched and moved across thread boundaries, and there is no Global Interpreter Lock (unlike Python) so shared memory parallelism is easy.
Basically, Go is as easy as Python (even easier for nontrivial applications in my opinion), about 20-30 times faster than Python (or about half as fast as C or on par with Java/C#), and much much nicer for concurrent and parallel tasks than all of the above.
Edit:
Ok, I definitely missed something ;-) I checked again C# documentation and kasey_junk is right. The tuple can be implemented as a struct, which is a value type (not a reference type), and arrays items are stored in contiguous memory. This is a big advantage of C# over Java (value types are planned for a future version of Java).
The closest language to go would probably be C++, and the language designers were quoted saying the main driving force behind go was to replace C++ due to it's complexity.
No. Both C# and Java use JITs, and their JITs have been carefully tuned to focus on hot spots. By not having a JIT, Go (in the 6g/8g/gccgo implementations) loses some important optimizations that are very helpful for making virtual-heavy languages like Go faster. JITs enable on-stack replacement, bailouts, and self-modifying code, which enable speculative devirtualization and (polymorphic) inline caching, just to name two techniques. These are very important optimizations that generally you can only get reliably in an ahead-of-time setting with profile-guided optimization, which Go 6g/8g don't have (and PGO is kind of a worse version of what good JITs do anyway).
The true benefits of AOT are reduced startup time and a greater tolerance for slower compilation, allowing for more sophisticated compiler optimizations—instruction scheduling, alias analysis, range analysis, etc. But Go 6g/8g are focused on fast compilation and omit most of those optimizations anyway.
How does Rust work in this regard? By avoiding virtualization, and compiling each function to many specialized variants corresponding to each possible permutation of input and output types?
However, you can opt into using trait objects instead, which do use virtual calls.
I believe pcwalton's points are that 1) running in a virtual machine does not imply that a language is slow (though I bet he'd agree with you that JIT introduces substantial memory overhead), and 2) Java, specifically, is not slow, thanks to a highly-tuned JIT.
Except there are things like ExcelsiorJET, CodenameONE, J9, .NET Native, CoreRT, Mono -AOT, IL2CPP.
As usual, language and implementation aren't the same thing.
Saying Go is much faster than Java is nonsense.
A JIT is not a performance feature per-se, either. It can be used for runtime code-optimization and -specialization which can improve performance. Some JVM implementations try and do this as well as they can automatically. The stuff they optimize, though, is exactly the kind of indirection that doesn't exist in Go in the first place.
The optimizations `javac` does are one of the few things that allows Java code to run at a competitive speed. And they're mostly trading space for performance, hence the unusually large memory footprint of Java applications.
"And Go allows for more control about the memory layout of structs/classes and arrays." Is an absolutely true statement, and it means that for some classes of problems (namely systems that prioritize GC latency for throughput) golang might be faster. The opposite is also true, especially with regard to large interdependent data sets, the golang runtime handles those worse than most JVMs.
I'd also completely disagree with your statement "Also, if the data set can be batched, it's trivial to parallelize processing in Go, while it's a big hurdle in Java." Especially with regard to "fast". There are more & better high performance concurrency libraries in Java than there are in go.
All that said, I was really reacting to this line "This is much faster than C# or Java, which compile to an intermediate interface" which is utter nonsense.
In the end, talking about performance in such large grained ways is usually not valuable but I'm willing to say that golang performs in the same category of performance as most jvms and that calling go faster than java is nonsense unless you speak about very specific cases.
Interpreters are typically slower (though simple to write and portable). The standard Python/Ruby/PHP implementations are interpreters. (The Python interpreter doesn't interpret from source every time, though; it usually uses the bytecode from previous runs.)
Implementations that generate machine code are more complicated and less portable, but have the potential to be faster. However, while compiling "down to machine code directly" can help startup time, it has little to do with what allows a language to be fast. What matters is whether it's possible to generate efficient machine code.
Static typing, for example, allows a compiler (ahead-of-time or JIT/VM) to generate more efficient machine code [1]. The ability to monkey-patch code -- which is common in many dynamic languages -- makes things harder for a compiler.
That said, there are compilation techniques to make dynamic languages pretty fast -- much faster than the standard Python/Ruby/PHP interpreters. For example, Javascript VMs have gotten 10x (or more?) faster over the last 5 years. (One of the big ideas: <https://en.wikipedia.org/wiki/Inline_caching>.)
A great example of all this is Facebook's PHP compiler. The first version would compile to machine code ahead-of-time, but it wasn't that much faster than the standard PHP interpreter (and definitely slower than the Firefox/Chrome Javascript VMs). They eventually switched to a JIT compiler, which is a better strategy for a dynamic language. Later, they transitioned to statically-typed sorta-PHP-compatible language called Hack, which allows for even more efficient machine code.
[1] There are different levels of static typing, too. For example, Go is statically typed but doesn't have generics. A C# generic data structure can be converted to more efficient code than a Go "interface{}"-based data structure.
"The liability to monkey-patch" in my opinion. :p
From what I remember - even leaving aside core language speed - the idioms of certain languages lead people to write slow data structures and algorithms. You can write (much) faster Python but it might start to look un-Pythonic.
But of course there is the counter-argument - if your web framework is your bottleneck then you're doing something weird. Personally I don't work on high-traffic sites and I'll choose expressivity over speed any day.
I don't think it's weird that the web server could be a bottleneck in a web application (even if it is a database-driven app, as most are).
Of course, if you're making an internal CRUD app that two people are going to use, it's unlikely to matter which stack you use.
> if you're making an internal CRUD app that two people are going to use
I know you're exaggerating for comic effect but still. You can run sites that have millions of visits a day on a $20/month VPS and still not have to worry about performance - unless you're doing something completely resistant to caching. I don't personally know anyone that has to handle more traffic than that but if you believed the general chatter on HN then that segment of the market doesn't even exist. Going from 30 servers to 2? I might be able to cut my overall hosting bill by a few hundred dollars a year but it's not top of my list of concerns.
FWIW, these cases don't come down to raw rq/s performance, but are more due to RAM usage.
A lot of the slowness in Python probably comes from the many memory allocations all over the place. Go gives you a lot of control over allocations even though it is a garbage collected language.
These slides posted by another commenter has some good points about why Python is so slow: https://speakerdeck.com/alex/why-python-ruby-and-javascript-...
Interestingly, there was actually a talk at PyCon US 2016 about using Go's http server in Python via C. https://www.youtube.com/watch?v=CkDwb5koRTc
So for '1 + 1', you can't just output some fast x86 instruction and inline it. You'll usually need to jump to where the python vm will do 'a + b' in C. You're having to do a bunch of redundant work at a higher level to emulate the CPU (sorta).
That's why you have JIT compilers that will read '1 + 1' and compile the appropriate machine code, store that in an executable memory region and jump to it. First time might be slow, but after that it's pretty fast. Because you need to jump to the dynamically generated code, you usually compile whole functions at a time.
This topic is super complicated but also super interested but I tried to simplify it a bit here.
http://magic.io/blog/uvloop-blazing-fast-python-networking/
This is almost exactly on par with golang's net/http for a simple echo server, yet it is written in python.
You're right though, why not use C? It's a good language, and it's hard to beat for it's low level powers, portability and speed.
What go gives you is high level productivity, testing, a solution for package management (abit a rubbish one), and a good ecosystem of 3rd party libraries for things like AWS.
The things that suck about C:
- It's hard to do right. There's a great book called 'Deep C Secrets' on this topic by Peter van der Linden. If you haven't read it, I recommend against writing a large project in C until you have.
- C has no package management solution at all. Go doesn't have a great one, but at least it has some kind of high level management for this. Working with C dependencies and the C various build tools for them is a nightmare.
- C has no memory safety, which means if you do screw up, the 'things that can do wrong' are much much worse than if you screw up in a relatively safe language like python or java.
- C (and even 'modern C++') suffer from major portability problems. Not that it's not portable; it is, but in order to be portable, you have to write weird, arcane and terrible code. It's entirely common to see code littered with `#ifdef WIN32 ...` or a typedef for every primitive type (eg. mInt32) to abstract across compiler differences etc. This means any code coverage you get is probably going to poorly represent the actual code in the library. Oh, did I mention C has no test runner? (although to be fair, CMake helps).
On the other hand, it is extremely embedable, and if you know what you're doing, it is the right choice. Have a look at this excellent highly portable IPC library: https://github.com/saprykin/plibsys <-- That's the right choice for the right job.
It's also the right choice, arguably, for a low level component that might import into some other slow-as-balls language like python. I'd argue Rust is a better choice, but hey, its much of a muchness.
...but for a web service or web framework?
nah.
Go was written specifically for those purposes, with high throughput performance as its goal, and a significant amount of effort devoted to optimizing that.
It's not suitable for something like plibsys either.
For short, there are some cases in which the C language specification does not tell you what to do and how to interprete code and in this cases, the choice is left to the compiler.
This is not trivial as it might look.
For example, what happens when you omit the return statement at the end of a function? GCC will to automatically insert a return statement for you, but Clang will not. Both are perfectly fine behaviours... As long as you are aware of it.
If you always compiled your code with GCC and suddenly switch to clang you might see weird shit, because the processor reached the end of your function, found no return instruction, and just went ahead and executed whatever code was put after your function (and it might not be so obvious what code is there).
(But in a way, this is part of the beauty of the C language: there is very little abstraction, and it is easy to understand what will happen when you run C code.)
>[...]
>...but for a web service or web framework?
>nah.
>Go was written specifically for those purposes, with high throughput performance as its goal, and a significant amount of effort devoted to optimizing that.
>It's not suitable for something like plibsys either.
I found this part of your comment difficult to understand. Is C not suitable for plibsys after all, or is it Go which wouldn't be suitable, or something else?
The point I was making is:
Don't pick go. Or C. Or Rust. ...unless it's the right tool for the job. Or at least the right sort of tool; there's plenty of cross over.
In this case (web framework), C isn't the right tool for the job. ...but, to be fair, C is the right tool for some jobs.
Curiously, I just found this today:
It seems to be a full featured package manager to use with C and C++.
if Sys.info()["sysname"] == "Windows"
even in R!https://www.reddit.com/r/golang/comments/4a8yit/is_this_the_...
https://github.com/gin-gonic/gin/issues/560
Some benchmark were made too.Don't they mean work well?
Just sayin'
Anyway, it's acceptable American English.
Comments empty and wrong.
I'd argue that at least the author tried to write a real documentation, which 95% of Go library authors don't do.
> the concept of a framework is fundamentally complex and at odds with the goals of Go
The famous "You don't need that with Go ™ ".
It's more like Gophers hate the word "framework","dependency injection" and "orm". It's hardly a framework, it's a router and a middleware stack.
Instead of being enthusiastic about others using their language, like in any other community, Gophers like to hate, shame, mock other people because they didn't do things "the Go way", whatever they think it is. It's one of the most toxic community I have ever seen.
https://github.com/iris-contrib/website
I just submitted a PR to fix a few minor issues. In general, the grammar issues here are really minimal and the page seems to communicate the project and its goals very clearly without being too distracting.
I didn't find any explicit typos, like misspelled words, but I didn't look that hard either, hopefully you can help out!
I mainly take issue with the proliferation of Go frameworks, especially HTTP frameworks. Frameworks are an antipattern and there are already a huge number of HTTP frameworks that are all incompatible and reimplement the same functionality. I don't think this is useful for the language and in fact think it hurts it.
What is fundamentally wrong with creating a framework with specific goals around performance and API?
If Go's stdlib were so comprehensive such that frameworks were not necessary, I would get your point, but it's my understanding Go exists as a simple language on which to build larger programs, not as a "one true way" language with everything included.
I agree. However, there is a big difference between libraries and frameworks. I believe frameworks are an antipattern in any language, not just Go, but Go's emphasis on simplicity makes frameworks for it especially grating to me.
Rather than poorly summarize, I'll link two of my favorite articles criticizing frameworks; one is humorous [1] and the other is more serious [2].
I took off my big boy trousers, put on my human being shorts, and decided to take my enthusiasm elsewhere.
Shame, because the language does have a lot going for it.