I found nothing.
Do you have a URL of any kind, for more information about this, including contacts?
636 karma · joined September 24, 2022
Redshift investigative white papers here; https://www.redshift-observatory.ch/white_papers/index.html
Redshift replacement system tables here; https://github.com/MaxGanzII/redshift-observatory.ch
I found nothing.
Do you have a URL of any kind, for more information about this, including contacts?
I feel like I have a better understanding now of the situation and what happened with H1, and I feel better about it.
I will now see if I can figure out how to wipe everyone's RS clusters instantly with a single command, so I can report something on H1 after all :-)
I've not actually checked, so I don't know, but knowing how logging works on RS, I think the cluster crashing will mean your killer query is not logged.
Your session will have been logged by the time the cluster crashes. OTOH, maybe you were logged in for some time first, or there's connection pooling, or you slipped the query into an existing connection's query stream, and so on.
Actually, thinking about it, I think you could reduce the problem to a single query, rather than two, which would help cover tracks.
The problem I know of is a bit different, in that it is a direct and immediate server crash. It's not a denial of service by making the cluster slow. It's run-query, crash-server.
You are right of course that any normal user can issue crazy queries which hog resources, and hammer performance.
Probably mainly my misunderstanding, but H1 did not help in any way.
I opened an account, filed a report - I can easily crash Amazon Redshift as an unprivileged user. Provided the DDL/SQL to do so - dead simple, two statements, issue them and boom.
I received a reply, something like, "we have closed the report, if you can demonstrate a working issue we'll investigate further".
I was confused, replied and asked for explanation. No reply.
I tried going to their Support, 403 - doesn't work via Tor browser - no use for an anonymous report.
And that seems to be it - end of road.
I don't understand, no replies, no support, and I've disclosed valuable information and I have no idea what H1 have done or are doing with it (if it's been made public, for example).
(I asked on HN for advice. One line of reply was that this is not an exploit, but a bug, which I can see. OTOH, when I filled in the severity rating form, there was nothing in that where I was evidently going against the grain of what was expected, so I'm not wholly sure. Any further advice in replies now gratefully received.)
I can also imagine disgruntled or malicious employees or contractors.
So we're looking at malicious or disgruntled employees, and also normal queries where users do not realize what they're doing will kill the cluster.
Why, man, they did make love to this employment;
They are not near my conscience; their defeat
Does by their own insinuation grow:
'Tis dangerous when the baser nature comes
Between the pass and fell incensed points
Of mighty opposites.
I am sorry to see anyone shot, and here quite possibly die, but I think his party were elected on the back of Russian money; Putin did everything he could to influence the election. To ride corruption to power, against popular will, upon matters of great import, is to run grave risk.I have a bookmark for it, and whenever I want to kick back a bit, perfect.
However, I've noticed that it has a tendency to go blank (i.e. fail and stop working) when multiple colour spectrum triangles are in play and in particular when they are on top of each other.
I'd really like to see some additional objects to place into the machine.
One other problem I have is that the "perma-link" button doesn't seem to do anything. When I come back to the URL, my machine isn't there.
2. Temp tables do not participate in k-safety, so you avoid that performance cost.
3. I suspect I've seen something odd happening with compression with temp tables. I've not investigated.
4. Using `CREATE TABLE AS` is, for me, verboten. Absolutely forbidden. This is because Redshift selects column encodings, and does a very, very poor job of doing so. Never let Redshift select column encodings (or sort keys, or distribution keys, or rely on auto-vacuum, or auto-analyze, and above all, never use AutoWLM).
I specialize in Amazon Redshift.
I've written a lot of PDFs about Amazon Redshift - serious stuff, deep technical investigations and explanations, published along with the source code which produces the evidence which the PDF is based on - and when people asked questions where I'd written up the answer, I pointed them at the appropriate PDF.
After some months, I received a direct message, which looked to me to be a pro-forma, a standard message sent in this situation, from the staff that I was promoting my site and I should not do so. It was well written and polite.
That's fine - I have no problems with that, it's their web-site.
What I did not like, however, and what came over as slimey, was that the staff had also deleted every post I had made.
This was not mentioned, at all, in the well written and polite message, which then of course became disingenuous. If you're going to do something serious like that, you need to tell people, not let them discover it for themselves.
This was for all posts, where I'd explained something directly or pointed to a PDF - presumably it's a standard action SO take in this situation.
I deleted my account and left.
Digital nomad, relocate and remote both good.
Technologies : Amazon Redshift
Web-site : https://www.redshiftresearchproject.org
In particular, now offering cluster cost reduction. Fee is and only is one month of the saving made.
A month or two after my main account was banned (see my other post in this thread), I logged into a second account which I'd not logged into for months.
Upon logging in, I discovered the account was "permanently suspended", and the reason for this was, and I quote;
"Your account has been permanently suspended for ."
The sub appeared to be working normally, I posted about the Amazon Redshift Serverless PDF, and then Reddit began behaving oddly.
After some investigation, and some guesswork, I concluded my account had been silently shadow-banned, and the sub banned (and then shortly after, deleted).
(Shadow-banning means when you log in as yourself, you see all your posts, and you see them in the threads where they were made. If you view Reddit when logged out, you then see all your posts have been deleted.)
Two years of posts and the sub disappeared, instantly, abruptly, without warning, reason, appeal process or notification, and Reddit is trying to lead me into thinking my account is still active. Make of that what you will.
Having had that experience, I concluded Reddit is not a safe place to invest time in.
Speaking for and only for Amazon Redshift, as I have little knowledge of other AWS services, I hold AWS's blogs, messaging, Support communications, TAMs, the lot, as relentlessly positive and to my eye deliberately and knowingly obfuscating all weakness. I regard information from AWS regarding Redshift as safe to read when and only when you already know what's going on / the underlying truth. Otherwise you will be misled, and to your cost at AWS's benefit.
By the sounds of it, the messaging over this change in data policy is the same.
What happens as you pile mass into a planet is that the planet becomes dense, not large, and this is because of gravity.
Jupiter has more than twice the mass of Saturn, but is only moderately larger in diameter.
You can keep dumping mass into a planet, and it just won't get much bigger, until you have enough mass that fusion kicks off, and then suddenly the now-a-star inflates, because it becomes extremely hot and then you have something the size of the Sun.
I had a second Reddit account I used for non-Redshift stuff, which I've not used now for a couple of months. The two accounts to my knowledge are wholly unconnected.
I logged in just now to have a look. The account has been permanently banned, as of two weeks ago.
The reason, and this is exactly what is written in the automated message, is;
"Your account has been permanently suspended for ."
https://www.redshiftresearchproject.org/white_papers/downloa...
https://www.redshiftresearchproject.org/white_papers/downloa...
I misunderstood how Serverless billing works. Having read the documentation, I understood - incorrectly - that pricing was per-query. The docs talk about pricing being per RPU-hour, but billed on a per-second basis, where if a query runs for less than 60 seconds, it is billed for 60 seconds, and this is for a serverless product. Therefore pricing is per-query - as it is with Athena, and with Lambda.
In fact, it is not.
Pricing is per workgroup-second. I still have not found this stated in the docs; I was pointed to a re:Invent talk where an AWS developer presented a slide which made this clear.
When I was working on the investigation, I was always running a single query at a time, so billing looked right.
This change fundamentally changes the pricing proposition offered by Serverless; the original pricing conclusion was incorrect by an order of magnitude.
I have now rewritten the content regarding billing and have republished. The abstract below is the new abstract. The revision history explains what happened. Credits will credit the reader who pointed the issue out, once they let me know if they want a credit or not.
If you have been using that sub, please instead keep an eye on the RRP blog, or use the forums on the RRP site, which are here;
No other information is given, other than a ban has occurred, no links or information to routes to appeal, or find out what happened, or why.
https://www.redshiftresearchproject.org/slblog/2023-09.html#...
(I see now the sub has disappeared from my profile, too. Two years of posts, gone - instantly, no warning, no reason, no information, no notification and no appeal process of any kind, so far as I can see. Reddit appears to be a risky platform to invest time into.)
The dense paragraph of explanation at the of the the page, at its very end, says the function links are not yet working. I was thinking to convert the list to a few bullet points, to improve clarity.
What I will probably add next though is permissions - who has access to the object (view, table, function). Mainly they're rdsdb only, but actually knowing is useful (and it lets you spot any slip ups).