tobilg

You can even use DuckLake with Datafusion: https://github.com/tobilg/datafusion-ducklake-provider

I can't confirm this, I have several instances which have statically linked extensions...

yeah, sure, those 5-10 different API calls would surely be a huge toll to refactor... I'd rather run an additional service to reimplement the S3 API mapping to my local drive /s

I'm 100% aware of how S3 works. I was questioning why the S3 API is needed when the service is using local storage.

I don't get it, if it's running on the same (mentioning "local") machine, why does it even need the S3 API? Could just be plain IO on the local drive(s)

Built with DuckDB-Wasm: SQL Terminal in the browser

https://terminal.sql-workbench.com

Impressive, congrats! I‘m building https://sql-workbench.com which is similar, but focussed on DuckDB WASM instead of SQLite… Love that you offer different databases.

Literally the first dozen search results if you google for "lakehouse" are about data lakehouses: https://www.google.com/search?q=lakehouse

This is a well-known term for everyone working with larger datasets...

Try https://sql-workbench.com if you‘d like to do this directly in the browser, for free. Including Parquet and Arrow support as well.

IMO that’s not really possible because of the size limits of Cloudflare Workers. Neither the WASM nor the Node version are small enough.

I‘m running it on AWS Lambda functions with some success.

Not sure why the comparisons were made with pretty outdated versions to be honest.

I‘m using a (older) v1.29.1 dev version with https://sql-workbench.com w/o any bigger issues.

But they got a custom CLI argument in the OSS product for it. And hosting/proxying it under the OSS domain ui.duckdb.org

Actually, the UI was built by MotherDuck…

Have a look at https://sql-workbench.com eventually. It runs DuckDB WASM in the browser, and with Perspective, which is used for data visualization, you can also visualize timeseries.

You can either drag & drop data, or use remote data sources via https

Hahaha, nice. It's a small world.

You can potentially use Ollama running a model locally, e.g. https://ollama.com/library/duckdb-nsql

Have a look at https://sql-workbench.com eventually, as it's using DuckDB WASM & Perspective to render the query results. Let me know what you think!

I created https://sql-workbench.com a while ago, mainly to let people analyze data that's available via http sources, or on their local machines, w/o having to install anything.

A recent project is https://shrink.video, which is using the WASM version of ffmpeg to shrink or convert video in the user's browser itself, for privacy and similar reasons mentioned before.

Sorry, but I don’t understand what you‘re trying to say…

I‘m building https://sql-workbench.com in my spare time.

It‘s a SQL Workbench in the browser, based on DuckDB WASM. You can query remote and local datasources, such as CSV, JSON or Parquet files.

You can also visualize the results, and share the queries via URL. Let me know what you think!

That's a complete other topic. DBOS also doesn't fit in this stack. And what has SAML to do with it?

I created a similar project, https://serverlessmaps.com which lets you host PMTiles on S3 and CloudFront, with a Lambda@Edge layer where you could also do auth...

With 1TB free traffic from the CDN, and pretty small costs for S3 and Lambda@Edge, it's probably even cheaper to self-host I guess. Even lower costs would be possible by entirely using CloudFlare services (CDN, R2)...

If you store the PMTiles in S3 or any other object store that supports HTTP Range Requests, that's a no-brainer... In a normal disk on you own server, this might become interesting, yes.

Thanks for the feedback, are you using dark mode eventually?

Exactly, it's "just" extracting the AWS IAM data from the publicly available documentation, stores it in multiple formats and publishes a NPM package containing the data with some wrappers/accessors, so that people can theoretically leverage it to build their own tools.

I publish a daily updated AWS IAM dataset at https://github.com/tobilg/aws-iam-data as well as historized managed IAM policies at https://github.com/tobilg/aws-iam-managed-policies

There's also https://www.awsiamdata.com/ which analyzes the AWS IAM data and also contains a changelog. Maybe this helps some people.

You can also query the data from your browser: https://tobilg.com/chat-with-a-duck#heading-explore-aws-iam-...

No, it's implemented via parsing the current schema and translating it to a Mermaid ERD, see https://mermaid.js.org/syntax/entityRelationshipDiagram.html

Currently it‘s not open sourced. Eventually considering it for the future though.

Yes, you can share a query, and also the visualization.

See https://tobilg.com/using-duckdb-wasm-for-in-browser-data-eng...

There‘s a query history tab, not sure if this what you want?

Otherwise, I‘m very open to feedback, just create an issue in the linked GH Repo. Thanks!