Fantastic! Congratulations on the launch.
Is there a way to version control the sync configurations? Any thoughts on putting that in the roadmap?
I'd love to be able to put my 'Castled config' in the same repo as my dbt project, for example.
Is there a way to version control the sync configurations? Any thoughts on putting that in the roadmap?
I'd love to be able to put my 'Castled config' in the same repo as my dbt project, for example.
I actually mean the 'definition' of the syncs themselves.
I am picturing JSON or YAML that describes the source fields, their mapping to the destination fields, and any other meta about the sync: frequency, number of retries, whatever else that you could configure in the UI
So when I go and update my dbt model to modify one of the tables that I am syncing from, I can make the corresponding changes to my Castled settings file, and release it all as one atomic update to my data infrastructure.
It might be a small number of people who would want something like that, but it's definitely something I would have been excited about when I was running a data team.
But I see value in exporting the config to a github repo after the pipeline is created and thereafter future edits can be done via the github repo. Does that make sense?
Looks awesome, I am rooting for you guys!
Yeah this one certainly depends on the target customer. For me, any tool that didn't have source control integration for configuration would be a non-starter. But it's quite possible that the target audience for this tool doesn't even understand the term "source control".
At Grouparoo, this is a primary use case. We have a UI that engineers use locally. This helps gets things right. It outputs a JSON configuration that is checked in. When that is deployed, it does all the syncing.