penciltwirler

I think it's useful for implementing Sagas https://temporal.io/blog/saga-pattern-made-easy

Basically, a multi-step event handling system, where the data could be spread across multiple databases/systems, so there's no way to rollback a transaction atomically across all the databases. Instead, you explicitly codify "compensations" to undo your previous commits so that you eventually end up in a consistent state.

Temporal is the orchestrator/framework/library to implement the above in an easier way.

Hugging Face's internal storage system is more sophisticated now after they acquired XetHub. They have a clever way to chunk and content dedupe the data that makes it a lot more efficient to version control large files.

Dumb question, but how do they know if a codebase is actually written by an LLM? I know that sometimes code agents write "Co-authored by" in the commit messages, but this is something you can over-write.

The premise is that Git-LFS sucks, so we need to build a new data versioning system (in Rust, from scratch). While I mostly agree with this premise, but there are already lots of existing (mature) data versioning systems with the same tricks under the hood:

- Pachyderm (Go): https://github.com/pachyderm/pachyderm

- XetHub (acquired by HuggingFace): https://huggingface.co/blog/xethub-joins-hf

- LakeFS (Go): https://github.com/treeverse/lakeFS

- Oxen (Rust): https://github.com/Oxen-AI/Oxen

I guess with AI, anyone can vibe code a content-addressed, chunk-level deduped, versioning system in Rust these days...

But jokes aside, Lore seems really cool! What's interesting is the realization that different domains/industries have similar problems, but they don't seem to be cross-polinating. In this case AI and Gaming both need a storage system that can version control large binary files at scale. I think there's lots of opportunities to share ideas here, but perhaps the lack of idea sharing (currently) creates opportunity!

blatant copyright infringement of https://rosalind.info/problems/locations/

"Chromebook" had a better ring to it.

I think in Picasa's case, they learned from that and turned it into Google Photos. I don't think there's any shared code, but at least they learned from it from a product perspective. I would say Google Photos turned out great.

There's lots of tools built on top of K8s to accomplish this tho. For example, Argo, Tekton, Flyte etc.

Does this affect Hacker News?

Have a look at ULIDs

Lots of comments about Avro Arrow here lool. As a Canadian, I'm both proud and saddened by the fact that although we have elite engineering talent, that talent tends to be drained to the US.

Side note, just a couple decades after the Arrow, Canadian researchers once again lead the world, but this time in the field of AI, specifically deep learning. However, what happened after that? Google hired Geoffrey Hinton, and the industrial might of the US took over.

Perhaps this is inevitable given that we are much weaker economically compared to the US.

However, one last thought is that, aerospace vs software is not the same. For aerospace, yea I think you need a giant super power of a nation to keep it alive. However, software can be built by one guy in a garage. I'm not sure why Canada just let its dominance in AI "slip away".

Also side side note about Avro Arrow, is it just a weird naming coincidence that the two Apache projects exists: Avro and Arrow? I think Avro is named after the original British Avro company. However, I don't think Arrow has anything to do with aviation.

just use dbdiagram.io

Haha a lot of funny comments here. I think overall it's neither here nor there. You should be proud that the "elites" at Google copied your code ;)

Just read the original paper "Attention Is All You Need". Cohere's founder was the guy who wrote that paper in the first place.

Hold on... aren't there a bunch of incredily successful "open source" / "source available" companies? Databricks, Cockroach Labs, Hashicorp, MongoDB, Elasticsearch etc?

I switched out of data engineering, and became a software engineer instead. I work on building the data infrastrue tools that I used to use. Maybe this only applies to me, but building the tools that I used to use is somehow incredibly satisfiying. I think it's because as someone who used to use the tools, I know what I want, and then I build it, completing the "cycle of needs".

So my advice to some of the data scientist/engineers out there is to go a little deeper into the tooling and try to understand how they were built.

Why is NASA pumping money into Boeing? Isn't NASA's money mostly from public funding? Boeing on the other hand is a dying company.

What's everyone's opinion on youtube music? You pay $11.99/month for YouTube Premium, you get both ad-free youtube (important on mobile), as well as music. Whereas Spotify is $9.99/month, but you only get music and podcasts. Granted Spotify has a much bigger library.

I think it’s a mix of “execs are the top dogs so they can decide to splurge a little on themselves”, the key is “a little”, I have no idea how much a private concert costs but anything less than $1M is probably considered “acceptable amount of splurge”

One can easily self host a bitwarden server on digitalocean. https://bitwarden.com/blog/digitalocean-marketplace/

However, I'm curious what y'all think about the cost. A digitalocean droplet for the recommended specs (4 GiB memory) is $24/month. This is hard to stomach when you compare with Bitwarden Premium which is <$1/month. I guess it depends on how much you value your own data.

Just do what Go did.

This is old news already.

Why does every game have to be zero-sum? Why can't chatgpt live alongside search engines as just another category for results? Like "Images" and "Maps"?

In fact, I look forward to Bing integrating with chatgpt.

Everyone already knows Google is tightening up, but no actual mass layoffs yet. There's no new information here.

Dex is also an interesting project in this space. It can act as an OIDC provider, but can be extended with other providers via "connectors".

https://dexidp.io/

Nicee, but I feel like really the only thing you need to know as an eng is DNA -> RNA -> Protein. Sometimes RNA -> DNA via reverse transcriptase. Everything else is just normal Python scripting.

Wait until our particle accelerators suddenly start saying "you are but bugs"...

Reminds me of Minority Report.

I feel ya, I was a data engineer at my previous job, and I hated it. In the end, it boils down to the fact that data scientists simply use the tools, while the SWEs build them. Trust your instincts, and follow what you are passionate about, instead of chasing hype/hotness in tech.

I think Zendesk has taken over the CRM part of Salesforce.