Postgresql 9.6: Support parallel aggregation
postgresql.org
postgresql.org
Or they could just release their own fork under an open license and focus on just adding features.
But that's not how they work. All of their contributions are pushed upstream which is a very considerable effort with how conservative Postgres is at accepting new functionality.
Aside of that: there are 2ndQuadrant employees in the #postgres IRC chat room, helping people with daily support issues. This is their core business and yet they still help people for free (within reason). This is bloody impressive.
If I'm ever at a point when I need help with a Postgres issue, then they will be very first of the list of companies I would consider.
Thank you very much for all that you are doing.
Kudos to them.
Over here, we only started to seriously thinking about what we're doing once we were handling in the order of 10K transactions per second.
Once you are at that level, you're probably going to need optimizations specific to your application and a generic database hoster might not be able to help you anyways.
I get that as a startup you don't have people for everything, but can you really afford to outsource the knowledge about the central piece of your application where all the value is stored at?
Hosting the dB has never been the knowledge about central piece of application. you may disagree (and I respect that), but for me it has been similar to building a RAID-10 dedicated server vs using AWS.
You can argue that maintaining data resiliently is a critical part of the organization - but at what stage? In the first 3 years of a startup, you are iterating the product. You are pretty much agonizing over drop rates, conversion rates for every single minute of your life. The fact that data will crash NEEDS to come a distant second.
Dropbox has only just moved out of AWS. Storage was probably the most critical part of Dropbox, but it chose (rightly) to focus on customers first.
What you are talking about will come - but it will come after some time. And till then I would love to pay some postgres devs to run a hosted dB for me...like RDS.
RDS is the most expensive thing we pay for. It's worth it.
1375s with 0 workers
131s with 10 workers
56s with 30 workers
There is also a point that Postgres will perform with enough features needed by most businesses that they'll choose it even though it doesn't match Oracle on a feature by feature basis.
We have hit peak Oracle. From this point forward it's going to be hard for Oracle to regain momentum. Expect a lot of FUD against Postgres - the more you see, the more worried you know Oracke execs are becoming.
The other thing is that some oracle features only work on exadata or other engineered systems. The hybrid columnar compression for one and query offload as well as some forms of the in memory stuff. The problem is that these are very high end features and most of us live happily without these. Postgres is going after the bread in the oracle db market while exadata is the exclusive cheese shop. Much smaller market even if the margins are better.
I wouldn't be so sure. Oracle has deep pockets, and I'm not sure there's a reason why anything PostgreSQL wants to roll out they can't pay to prioritize and have done sooner.
> There is also a point that Postgres will perform with enough features needed by most businesses that they'll choose it even though it doesn't match Oracle on a feature by feature basis.
No argument there.
The problem commercial businesses have when it comes to open source is that you might be able to reduce customer take up, by you can't compete with it like you would other businesses. In the closed source world you can purchase a company and shut down its product and thus kill off the competition.
As Microsoft have found, with open source software, that's not possible.
The other unfortunate thing for Oracle is that when they attack Postgres they have to publish lists of competitive advantages. All this does is give Postgres developers a todo list, and they then work towards implementing the features that matter.
I used to work consulting doing these types of migrations for customers for a huge IT services company.
I'm far and away not a database hacker, but have read parts of the code at various times to better understand what I was seeing. The codebase is incredibly well written and organized, and the documentation (both developer and user) is top-notch.
The only problem I can think of with using it in a course would be choosing what to focus on with only a semester. Add in the long revision history and I think there are also multiple theses in there on the sociology of open source.
[1] http://postgrespro.ru/education/courses/hacking [2] https://www.youtube.com/watch?list=PLaFqU3KCWw6Jfb8IBNk3hZ07...
Personally, I found it a delight to work with, and that has translated into a great respect for the product itself. It is no great surprise that the pace of development stays strong and that engineers gravitate toward it; it's set up for exactly that.
http://rhaas.blogspot.com/2015/11/parallel-sequential-scan-i...
Also, Robert is not the author of the patch, he did a review and committed it. The actual author are listed in the commit message: David Rowley and Haribabu Kommi.
The machine has 4x E5-4620, so 32 physical cores. And with 30 workers it gets ~80% of the theoretical speedup. Not bad, I guess.
That assumes I run PostgreSQL already, which I don't. I am interested in possibly switching at some point if it's worthwhile, but it's hard to muster the effort to do concerted testing of a representative sample of my data, including possibly changing how queries are done to take advantage of specific features, when I have little information to go on.
Not that I expect PostreSQL to do in-depth analysis of everything, but it would be great from both a promotional and technical standpoint if there was something like "we've seen something like X% speedup of queries utilizing Y, and up to Z% speedup in extreme cases." I mean, I assume they at least have rudimentary numbers for this, otherwise they would be making blind changes without knowing whether it improved or degraded performance. Providing just enough to get people interested in doing their own benchmarking (and possibly publishing them) would be great for everyone.
Edit: One of top comment is actually what I'm talking about (but apparently for a different feature). So it does get done, which is really nice. :)
Edit2: Now there's the link to the blog post for this feature. :)
It could be a 1% speed up from insanely fast to slightly more insanely fast, or a 100% speed up from unbearably slow to just extremely annoyingly slow.
I gather that people considering migrating who want some reassurance before investing time in testing should be more interested in such things things as benchmarks that show whether PostgreSQL can saturate your hardware for various query types, benchmarks that compare its query planner against that of competitors, and benchmarks that show how well it works under load.
> If you aren't using it now, and thus don't know how fast or slow it would be on your data and queries, why would you care about percentage speed ups?
I'm working under the assumption that similar operations between PostgreSQL and MySQL for operations that aren't using an advanced feature that the other plainly doesn't support will be roughly comparable in performance. I understand this isn't true, but it's useful in that it gives me a baseline to work with when reasoning without forcing me to migrate my application (or at least a significant portion of the data) to PostgreSQL just to test. If I'm then confronted with a feature that one supports and the other doesn't that has the capability to in some cases reduce the query time by an order of magnitude, then it's starts approaching the criteria needed for me to invest that time to see if it's worth migrating.
> I gather that people considering migrating who want some reassurance before investing time in testing should be more interested in such things things as benchmarks that show whether PostgreSQL can saturate your hardware for various query types
Isn't that exactly what this is? Throwing CPU cores at aggregating data to reduce time? I asked for benchmarks regarding this specific feature, to see what type of gain we were talking about with this. I didn't ask for comparisons to other databases because those benchmarks are generally harder to perform fairly, so I would have been (and am) happy with comparative gain benchmarks.
The GPG signing key used by the apt repo for Debian and derivatives is also served over HTTPS @ https://www.postgresql.org/media/keys/ACCC4CF8.asc, and the instructions for use direct you to install it as such.
It literally matters not at this point whether downloads are delivered over HTTPS or not outside of anonymity (which is almost moot, because you are obviously downloading PostgreSQL or the few related packages in these repositories) since package signatures are verified.
But in a production environment, you always want to use a package provided by your OS vendor. Postgres has excellent packagers for both Debian and RHEL-flavored distros (and I imagine more) and you really want to have the system-level considerations thought through by someone by someone who knows what needs to be thought about.
My OS's install mechanism downloads the source tarball. It does authenticate the download, of course. The point being you don't know what platform someone is using, and the source may be the only way they can install PostgreSQL.
> PostgreSQL wants
It doesn't matter what they want - if the source is available for download, it will be used. Unauthenticated downloads are an "attractive nuisance" that puts users at risk. The actual download links[1] at www.postgresql.org do use https, but the HTML that contains the download URLs irresponsibly redirects https requests back to http. While the download of the actual source tarball is authenticated, the URL to that download can easily be modified in transit.
I'm sure that someone who has capability to not only tap into but also modify traffic on the fly, won't have problem obtaining valid certificate from one of the hundreds CAs that everyone has in their web browsers.
If anything I feel https actually hurts, because it gives you false sense of security and you're more likely not to validate files with GPG, which you should do whether it is https or not.
I would be suspicious that it's the very first step that poses the most danger: there's little way to know whether minor-distro-X is "properly" hosted at minordistrox.com or distrox.org or even distrox.github.io, other than by blindly trusting the relative PageRanks of each.
---
Something that would be very nice, I think, would be a way to rely on the package-signing infrastructure of the OS you've already got (and trust), to guarantee for you the copies of any new OS images—even for different OSes!—you download. Sort of the same way you download new versions of OSX through the Mac App Store, but going a bit further.
Now that I think about it, the browser-preloaded HSTS list might do the trick... if it was coupled with a mapping of 'important well-known downloads' to a single (HSTS-preloaded) domain you should be allowed to get those downloads from. (Presumably with heuristic detection, so it could find "something looking mostly, from its metadata, like a Debian ISO image.") Then your browser would just tell you you're being phished if you're trying to get a Debian ISO from debianisamazing.info.
and I'm still here waiting for 9.5 to arrive in RDS of amazon :(
Oh, it does have upserts now. Awesome =]