pqyzwbq

I checked rath some days ago, it is pretty cool. It's great to know graphic-walker.

by the way, for RATH's automated data exploration, it seems like https://github.com/rilldata/rill-developer project.

Maybe you can check:

  - https://github.com/turbot/steampipe (which will query the sources on demand): Use SQL to instantly query your cloud services (AWS, Azure, GCP and more). Open source CLI. No DB required.

  - https://github.com/cloudquery/cloudquery (which will retrieve data in batch, and save them into another database for query)

To write web gui in Python, there are some other open source alternatives.

If just want to port simple shell interactive interface to web gui, can check https://github.com/pywebio/PyWebIO

If want to get a production level dashboard by using Python, https://wave.h2o.ai/ <<Realtime Web Apps and Dashboards for Python and R>>

Already switched to fly.io. It is also great, and its VM will not "sleeping while inactive"

The document for pocketbase's realtime is: https://pocketbase.io/docs/api-realtime/

The Realtime API is implemented via Server-Sent Events (SSE). Generally, it consists of 2 operations:

  * establish SSE connection

  * set client subscriptions

SSE events are sent for create, update and delete record operations.

Pocketbase is mainly a single application with SQLite embedded, so, whenever you update records through PocketBase's API, the main application always knows that.

This is different with postgres, since it may be connected by several different applications/servers remotely.

It is very hard to develop on premise product.

First you need to works with different Linux OS (ubuntu, redhat, etc.), and nowadays, product may contain a lot of microservice, then you need a kubernetes cluster, which is very hard to maintain. And then, you need to find a good storage (object/blob storage) for kubernetes.

Then, you may need to often troubleshooting server issues (like, disk full, slow disk issue, out of memory, kubernetes network broken). And you need to often check whether the data backup system works.

It is much easier to operate as SaaS, and you can pick your OS and your machine spec, and your programing stacks

pocketbase is great, I used it in several personal projects, and I really like it

https://github.com/pocketbase/pocketbase

PocketBase is an open source Go backend, consisting of:

  * embedded database (SQLite) with realtime subscriptions

  * built-in files and users management

  *convenient Admin dashboard UI

  *and simple REST-ish API

Scala was fun when my company was small.

Later, we need to recruit more people, it became harder and harder. Each need to learn a lot before they can contribute bug fix to product. They need to learn:

  - Scala

  - Akka

  - very basic of sbt (if they need to change more than basic usage, they come to me for that change, as I'm the only one who learned complex sbt syntax in our team currently)

  - Play framework

  - Slick

But now, several years later, Akka's license change really make us to consider switch back to plain Java and Maven one part by another. For the actor model, I really like it, it can simplify Synchronization A lot, we researched Vert.X, which seems can do most of Akka (Although I haven't research whether Akka Clustering, persistent can work similar in Vert.X yet).

I planned to write a series of articles about how to build distributed applications on Akka Clustering, now, there's no need for that. I wish I didn't recommend Akka to my friends. Thanks.

I would also suggest prql, which is also great. Plus: - the license of prql is better (APACHE v2 vs GPL v2) - although having modeling layer is good, but something like dbt-metrics maybe better. In this point, prql is more focus, on querying part.

you may check: https://github.com/jitsucom/jitsu. "Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days"

You can create an API endpoint, and send those JSON to it. In the "destination" part, it can sync to clickhouse (one of many choices, like redshift, snowflake,besides clickhouse) very quickly, and flatten the JSON into columns. If there is new key found in JSON, it will create a new column in clickhouse.

Noticed this is answered in: https://github.com/apache/logging-log4j2/pull/608#issuecomme...

``` I believe that applications that use log4j-api with log4j-to-slf4j, without using log4j-core, are not impacted by this vulnerability. (Because the lookup and JNDI implementations are in log4j-core.)

```

OK, thanks, didn't notice this when I read it.

Do anyone know if I depends on the following 2: - org.apache.logging.log4j:log4j-api - org.apache.logging.log4j:log4j-to-slf4j

But without dependency on - org.apache.logging.log4j:log4j-core

in this situation, is this safe from this RCE? Thanks.

Edit, This may affect both log4j 2.x and log4j 1.x (see comments bellow, thanks.)

FoundationDB is awesome

Yes, Best programmers can win against any integration tools. But in real world, it rare to find enough "good programmers", so they can not abstract good components and good layers. So, at this case, the business integration tools will win. Those tools mainly sell best practices and well-tested components, so, although you are limited to what those tools provided, but will solve those tricky small things without your worry.

In the meantime, the business integration tools can learn more concepts from general programming languages, like: version control, unit test, external variables, integrate sub-components into another combined component, shared macros, etc.

Yes, you're right. I just changed my prev comment to add my usage case to make it clear. Thanks.

For us (we build business software distributed to customers), the License is the key factor. MySQL/MariaDB is GPL, and even worse, the JDBC driver is also GPL (which makes any business software built on MySQL jdbc driver violate the GPL license). While postgres's license is more permissive.

I use dbt to build small dremio data warehouse. dbt is using software engineering ideas to build data, it is quite awesome. I can use github to version control it, can did some unit test (data quality check mainly).

Another thing is its macro systems. Although Macros are not a good programming language. But it makes integrate new DBMS much easier, because most 3rd party plugins are mainly written in Macros. So, Macros is good too if it is provided by other people (not need to be written by myself :) )

PyWebIO seems similar: https://github.com/pywebio/PyWebIO, it can turns python command line script into web with only very small change.

And PySimpleGUI mainly for Desktop: https://github.com/PySimpleGUI/PySimpleGUI