Injecting a Chess Engine into Amazon Redshift | Hacker News Reader

Injecting a Chess Engine into Amazon Redshift

michaelburge.us

91 pointschewxy8 comments

If I'm understanding how this works, the idea is to compile a program locally for x64, extract out the .text, convert it to hex, and have a Python UDF[1] shim in Redshift execute it.

Pretty nifty! The surprising thing for me isn't getting it to play chess though. It's that Amazon let's you run arbitrary binaries from within Redshift. I would have figured that the Python UDFs would be locked down to prevent this type of thing.

[1]: User defined function

It's not Redshift-specific. A similar function should work in Postgres, though it might be slightly more dangerous since it wouldn't be wrapped in a container like Redshift's is.

But this technique isn't useful for Postgres, because you'd just write an ordinary C extension at that point. That's why I focus on Redshift in the article.

> As of PostgreSQL 7.4, PL/Python is only available as an "untrusted" language, meaning it does not offer any way of restricting what users can do in it. It has therefore been renamed to plpythonu. The trusted variant plpython might become available again in future, if a new secure execution mechanism is developed in Python. The writer of a function in untrusted PL/Python must take care that the function cannot be used to do anything unwanted, since it will be able to do anything that could be done by a user logged in as the database administrator. Only superusers can create functions in untrusted languages such as plpythonu.

https://www.postgresql.org/docs/9.1/static/plpython.html

Thank you for writing this post. Learned a lot.

If it's like other databases, the Python UDF probably runs as a subprocess with limited abilities.

From http://docs.aws.amazon.com/redshift/latest/dg/udf-python-lan... :

> Amazon Redshift blocks all network access and write access to the file system through UDFs.

Wow this is somewhat surprising that you got this to work. I didn't really know Redshift was capable of this. It will take me some time to fully digest your post, but it is quite an interesting find none the less.

Quite neat.. but I kind of long back to the time where computing hacks didn't use relatively expensive pay-per-usage cloud services run by Jeff Bezos.

Depends on how you value your time.

The nickels I pay to spin up Redshift are nothing compared to the time it'd take for me to provision, install, configure, and host all the various components.

Obviously it is not about the actual few dollars, but the principle.

Reply on news.ycombinator.com