ianyanusko

Salesforce's scattered docs don't make best practices super clear, so designing the syncs took us some time.

We think the Salesforce syncing market by itself is 500 million to 1B. If you start including all of the tools we have our eyes on (Hubspot, Monday, ERPs), the market size comfortably gets into the billions.

Going after Heroku Connect makes sense as a starting point, but we've got our sights beyond that.

So we decided to double down in this smaller, underserved area for now as we try to build something people love, even if it's a bit niche.

That being said, there probably is room in the backup space for challengers against the incumbents - it's a massive market.

> Syncing data from Salesforce (to seed a database for example) is done via REST too. It works OK.

Have you thought about using the Bulk API for seeding? We started relying on that instead of REST, which helped us seed massive DBs much faster / more efficiently.

Sounds similar to the use cases we're seeing, where it's not only easier to process/build on Postgres, but also saves you on the Salesforce API.

Our polling approach is relatively database-agnostic. We just need to handle each DB's quirks with our transformers (e.g. dealing with MySQL's lack of BOOL field types).

Streaming is currently Postgres-specific. We're planning on rolling out support for MySQL next, after we've finished our Hubspot integration. Do you have a specific DB in mind?

> (And, how are conflicts resolved? In a huge system with millions of records coming from everywhere it can fast become nightmarish?)

The primary source wins any merge conflicts that happen within a sync period. With polling, it's pretty straightforward: at every poll, we see how each side has changed, and for any record pairings for which there were edits on both sides, we prefer the primary source.

With streaming, we employ a hybrid method, where we only poll when events occur in either Salesforce or Postgres. If at that poll, the same record has been edited on both sides since the previous poll, we still prioritize the primary source (Salesforce). You can read the step-by-step flow here: https://docs.usebracket.com/streaming#the-streaming-sync-met...

> Many clients I can think of this being most useful for would rather host it themselves, is that an option?

Right now you can self-host the associated datasets (like the Postgres event log table), but we're still working on allowing you to self-host the entire service. Stay tuned :)

> do you support subsetting a data source -- only sync records matching criteria?

In a one-way sync from Postgres to Salesforce, yes, you can apply filters using a SQL statement, but we’re working on adding this to the Salesforce side as well as two-way syncs. From your perspective, how would you want to set these filters? A SOQL query, or something else?

> Do you support to/from different instances of the same connector (e.g. Salesforce to Salesforce)?

Yup, our infra is data-source agnostic! But Salesforce <> Salesforce is not heavily tested, so there may be some funky behavior with oauth tokens if you’re trying to connect objects from two different instances during onboarding. Curious what use case you have in mind?

> Can you perform any transformations like map over the data, normalize or denormalize tables, etc?

Besides the one-way SQL filter I mention above, we try to make field mapping easy between the sources by automatically transforming when necessary (e.g., transforming a Salesforce picklist to a Postgres integer and vice versa). But we’re working on allowing users to create more detailed field-level transformations in the next few months.

Thanks for the feedback! Totally hear you on SSL, we’ll move that to every tier.

IMO, the reason many YC companies struggle with enterprise sales is because it's generally much harder than PLG, with long sales cycles and delayed feedback loops. It was definitely a learning curve for us!

If you're curious, you can read more about our approach to it and the actual SQL we use here: https://docs.usebracket.com/streaming#postgres

At first, it was a matter of doing discovery with people who we thought would find Bracket useful. These weren't sales convos, but pure feedback/discovery. From that process, we got a good sense of which roles face these pain points, and exactly what those pain points are.

Since then we've split our time between 1) writing content/distributing it in places people would find it helpful and 2) reaching out directly to people who would use Bracket. In the latter case, we really don't like generic sales campaigns, so we spend a good bit of time crafting each message.

That’s one reason why people trying to stand up two-way syncs have been stuck with dated tech (Heroku Connect) or have had to build in-house.

Right now, we're focusing on use cases that require changes to be synced within a minute. Fivetran and its ilk are generally more focused on large data batches at less frequent intervals (e.g., to unlock 1-minute syncs at Fivetran, you must be on the Enterprise plan, whereas we offer that out of the gate with the Starter plan).

We’ve worked with some folks who were cobbling together ELT and rETL to achieve a type of two-way sync, but they faced three problems with this approach: most importantly, handling for infinite loops, where an insert in one data source triggers an insert in the other, which then gets picked up anew, etc. Second, it wasn’t clear how to handle merge conflicts when the same record is edited from both sides (we give priority to the primary source). Finally, it was frustrating to debug problems when having to log into two separate tools.

These follow 1) the evolution of human conceptions of God, 2) the historical Jesus, and 3) the historical Muhammad, respectively. They're extremely accessible and have deeply influenced my thoughts on religion.

So why debate now, only to have to debate again with new info?

We still practice disagree & commit among the three founders, which is simply a majority vote game, but now the number of heated debates has decreased.

Exploratory, but the general themes are startups, religion, the Beatles, article reviews, and travel!