Hacker News Reader

Ive had issues in the other direction.

Importing data using the copy from s3 command has a nasty failure state if the client loses connection to the db AND the import command has a problem.

The query hangs with a row exclusive lock and the query _cannot_ be killed. Only option was to reboot the DB.

View on HN

You mean this function: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...? I don't get that Postgres RDS only allows you to import from csv, while the exports are strictly parquet. I know that Postgres implements the COPY command using csv or binary, but i would expect some end-to-end lifecycle implementation by AWS and parquet would at least enforce some schema and reduce some errors including those which caused yours.

As far I can tell, exporting query results from Postgres RDS to S3 uses Postgres's Copy and only supports CSV and Text [1]. Can you provide a link for parquet exports?

[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/postg...

You can use a RDS Snapshot to export to S3 in Parquet format: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...

Reply on news.ycombinator.com