sherifnada

Great tool. Secret sharing always felt needlessly complicated and clunky. I love that this lets you securely share secrets with zero barriers to entry in seconds. No need for "what's your email" or "hey make sure you accept the share on lastpass" or any of that. Just a quick one-two secure envelope exchange.

In some sense, Data engineering today is where software engineering was a decade ago:

- Infrastructure as code is not the norm. Most tools are UI-focused. It's the equivalent of setting up your infra via the AWS UI.

- Prod/Staging/Dev environments are not the norm

- Version Control is not a first class concept

- DRY and component re-use is exceedingly difficult (how many times did you walk into a meeting where 3 people had 3 different definitions of the same metric?)

- API Interfaces are rarely explicitly defined, and fickle when they are (the hot name for this nowadays is "data contracts")

- unit/integration/acceptance testing is not as nearly as ubiquitous as it is in software

On the bright side, I think this means DE doesn't need to re-invent the wheel on a lot of these issues. We can borrow a lot from software engineering.

The terminal has been such a core part of my workflow for so long that I sometimes forget how difficult it can be to use; I've just learned to hack my way around the friction.

It's so refreshing to see a product try to reinvent something so core to developers everywhere. Really excited to see where this goes!

Congrats to the Flycode team on their recent launch! This is a really exciting product

If you run your own datacenter, there is also the opportunity cost of slowing down R&D and new development work.

Let's say the year is 2012 and Redshift is introduced, completely changing how organizations can generate insights from their data. Running your own datacenter? good luck waiting for the ops team to install something similar! It might take you a couple of quarters assuming they are already competent at it. On the Cloud? Press a few buttons and you're off to the races.

velocity is a competitive advantage

I think this is saying that particular class expects to receive a refresh token as input. The "full oauth loop" means the UI needs to produce a refresh token via user consent in the browser.

Airbyte recently added support for custom GSQL & Facebook Marketing queries FYI ;)

Ultimately our goal is not to enforce that any derivations of Airbyte become public, so AGPL was much heavier than needed. By contrast, the Elastic License toes the line perfectly as it only places the restrictions mentioned above.

FWIW this has 0 impact on 99.9999% of our current or future users. The only practical limitation is that they cannot repackage Airbyte as a hosted service, which was only ever a threat from perhaps less than 10 players in total. Users can still build products on top of Airbyte, fork it (for their own use or even to build a product on top of it).

Extensive user surveys prior to this change didn't uncover a single user whose workflow would be impacted.

Just some context :)

Could you say more? Is your issue with the license's content or branding?

I work at Airbyte and am genuinely curious about your view.

It's a module which extracts data from a source (an API, database, etc..) or pushes data to a destination (warehouse, blob storage, API, etc..). They're ubiquitous in data engineering & analytics workflows where users often need to consolidate their data in one database or warehouse to build dashboards, run analyses, etc..

It would be really interesting to see an article about biggest companies YC passed on around that time in the crypto space. We hear about the YC success stories but I think the “could have been”s are also equally interesting

a to-do list app is a sure fire way to get on the front page of HN

Hi, Airbyte engineer here.

All connectors and the core functionality of Airbyte will _always_ be open source and free to use. You will always be able to download and operate an instance of Airbyte, containing all connectors and ELT features (whether for personal projects or business operations in your company) for free.

However, there are some things that we might offer as a monetized product in the future, such as:

* enterprise compliance requirements (E.g SOC2)

* SSO

* Hosting Management

Hope this clarifies the direction we’re taking.

Hi, Airybte engineer here! We do support incremental updates for MSSQL & Postgres -- I updated the docs to make this clear.

More broadly as an ELT tool, Airbyte is focused on supporting the long tail of integrations. The number of data silos is increasing rapidly, and it's intractable for a single engineering team or company to integrate with them all. So as an OSS tool, Airbyte provides a core platform for replicating data (e.g: a UI, scheduling, logging, state management, etc..) and the most popular connectors out of the box, then gets out of the way of its users. If you need a connector you need that Airbyte doesn't already provide, you can implement a Docker image which conforms to the Airbyte protocol, then start using it in your Airbyte instance without ever needing to go through us (Although many users do end up contributing their connectors back to the core codebase).