HNHacker News
TopNewBestAskShowJobs

bfirsh

4,246 karma · joined November 29, 2009

Founder of Replicate (W20) https://replicate.com/

https://firshman.com

[ my public key: https://keybase.io/bfirsh; my proof: https://keybase.io/bfirsh/sigs/I2UKihiqAntrGCCwICWwJXcyy9Bt95eBXaBlml8VkuQ ]

submissionscomments
bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Sorry yeah -- not saying nobody's using MLflow. We see teams and organizations using MLflow. What we don't see though is individual researchers/engineers/data scientists pick it up and use it.

From the people we have talked to who use MLflow, we hear it gets the job done, but individual contributors don't love it.

We really believe that widespread adoption comes from making something individuals love and use every day. That's the reason Docker was so successful, for example.

The lack of flexibility really resonates. That's the reason we're trying to be small and not too opinionated. We're something we can drop into your in-house system as a component, kinda like lots of deployment systems are built around Docker.

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Yeah, I agree this space is crowded. But we’ve found so few ML researchers/engineers are actually using these tools. This could either be that people aren’t aware of them yet, or that they’re not good enough.

I think it’s a mix of both, honestly, but we’re betting that there’s more of the latter in the mix. :)

I could do comparisons of each of these tools, and some of them are solving quite different problems, but the overarching difference is we’re just trying to do less. These systems might make sense if you’re setting up a company’s ML pipeline, but we found lots of individuals struggling to keep track of their work and store their models. They balked at the idea of setting up things like Kubeflow or MLflow.

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Funnily we applied with a different thing. We tried a number of different ideas before we settled on this lower-level thing, as I describe in the main comment.

Even still, I think most people apply to YC without a concrete plan of how to make a business. It's normally so early stage, that the plan is yet to be validated and will probably change. A "plausible" plan is perhaps a better way to put it. ;)

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Yeah, YC is funding lots of open source projects. PostHog[0] was in our batch. GitLab, Docker, Mattermost, and CoreOS come to mind as other open source YC companies.

There are a number of businesses we could build around the project. A cloud service or enterprise products/support are the obvious ones. Right now, we're focused on community building, because a potential open source business can't be successful with a healthy open source project.

[0] https://news.ycombinator.com/item?id=22376732

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Thanks! It's open source and you're in control of your own data. See https://news.ycombinator.com/item?id=25151741
bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
You can save data to a path on the filesystem, so one way to do this is with a network mount. Lots of academic departments have their own GPU clusters, and they tend to have a shared network filesystem.

We want to have more ways to do this though. We were close to adding SFTP support, but didn't get round to it. Another method could be to implement our own server, but we're trying to keep it simple for now. I'd be curious to hear your feedback here: https://github.com/replicate/replicate/issues/366

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
This came out of a practical problem: at Spotify, Andreas couldn't let any data leave their network. He wasn't going to go through procurement to buy an enterprise version of one of those products, so his only option left was open source software.

But it's also out of principle: we think such a foundational thing needs to be open source. There is a reason most people use Git and not Perforce.

Replicate can work alongside visualization tools -- your data is safe in your own S3 bucket, but you can use the hosted visualization tool to complement that.

You could also imagine visualization tools built on top of Replicate. One thing we've been thinking about is doing visualization inside notebooks. It's like a programmable Tensorboard: https://colab.research.google.com/drive/18sVRE4Zi484G2rBeOYj...

I'd be curious to hear your thoughts about that. It's pretty primitive so far, but we've got to start somewhere I suppose. :)

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Would love to chat -- I'll shoot you an email. :)
bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Yep, and we can do lots of other nice things. We can produce nice tables with the key/value data, filter it ("show me all experiments with an accuracy greater than 0.9"), produce well-formatted diffs across an arbitrary number of things, give you a nice Python API for analyzing the data in a notebook, and so on.

There are some examples of these things on the home page, all of which would be very fiddly to do with JSON files in Git: https://replicate.ai/#features

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
We talked to a bunch of MLflow users, and the general impression we got is that it is heavyweight and hard to set up. MLflow is an all-encompassing "ML platform". Which is fine if you need that, but we're trying to just do one thing well. (Imagine if Git called itself a "software platform".)

In terms of features, Replicate points directly at an S3 bucket (so you don't have to run a server and Postgres DB), it saves your training code (for reproducibility and to commit to Git after the fact), and it has a nice API for reading and analyzing your experiments in a notebook.

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Thanks!

DVC is closely tied to Git. We've heard people find that quite heavyweight when you're running experiments.

We think we can build a much better experience if we detach ourselves from Git. With Replicate, you just run your training script as usual, and it automatically tracks everything from within Python. You don't have to run any additional commands to track things.

DVC is really good for storing data sets though, and we see potential for integration there: https://github.com/replicate/replicate/issues/359

bfirsh··on Launch HN: Replicate (YC W20) – Version control for machine learning
Hello HN!

We're Ben & Andreas, and we made Replicate. It's a lightweight open-source tool for tracking and analyzing your machine learning experiments: https://replicate.ai/

Andreas used to do machine learning at Spotify. He built a lot of ML infrastructure there (versioning, training, deployment, etc). I used to be product manager for Docker's open source projects, and created Docker Compose.

We built https://www.arxiv-vanity.com/ together for fun, which led to us teaming up to build more tools for ML.

We spent a year talking to lots of people in the ML community and building all sorts of prototypes, but we kept on coming back to a foundational problem: not many people in machine learning use version control.

This causes all sorts of problems: people are manually keeping track of things in spreadsheets, model weights are scattered on S3, and results can’t be reproduced.

So why isn’t everyone using Git? Git doesn’t work well with machine learning. It can’t store trained machine learning models, it can’t handle key/value metadata, and it’s not designed to record information automatically from a training script. There are some solutions for these things, but they feel like band-aids.

We came to the conclusion that we need a native version control system for ML. It’s sufficiently different to normal software that we can’t just put band-aids on Git.

We believe the tool should be small, easy to use, and extensible. We found people struggling to migrate to “AI Platforms”. A tool should do one thing well and combine with other tools to produce the system you need.

Finally, we also believe it should be open source. There are a number of proprietary solutions, but something so foundational needs to be built by and for the ML community.

Replicate is a first cut at something we think is useful: It is a Python library that uploads your files and metadata (like hyperparameters) to Amazon S3 or Google Cloud Storage. You can get back to any point in time using the command-line interface, analyze your results inside a notebook using the Python API, and load your models in production systems.

We’d love to hear your feedback, and hear your stories about how you’ve done this before.

Also – building a version control system is rather complex, and to make this a reality we need your help. Join us in Discord if you want to be involved in the early design and help build it: https://discord.gg/QmzJApGjyE

bfirsh··on Pimp My Microwave
The best microwave I have ever owned was the cheapest one you could buy from Argos: https://www.argos.co.uk/product/9174030

Two dials: power and time. The time dial moved to indicate how much time was left. It went "ding" when it was finished. That was it.

I can understand the need for features that automatically calculate how long to cook the food, and so on, but if you want those features you seem to have to take a massive hit in usability. They all seem to have cumbersome button-based user interfaces like this.

bfirsh··on Ask HN: Jack-of-all-trades of HN, how do you approach job search?
If you have skills outside engineering too, the job title for this is "product manager".

I formally switched to product management in my last job. I have background doing startups, and this combined with jack-of-all-trades engineering, makes for a perfect technical product manager.

bfirsh··on The sad state of PDF-Accessibility of LaTex Documents (2016)
Converting LaTeX to HTML may be a route to making it accessible. I'm working on this project: https://github.com/arxiv-vanity/engrafo

It's 80% of the way there, but with 80% more work it could be a pretty complete implementation.

It powers this: https://www.arxiv-vanity.com/

bfirsh··on Show HN: Muse – Tool for Thought on iPad
If you're in the early stage building a product, their podcast is extremely helpful. It really deserves a Show HN all of its own.

https://museapp.com/podcast

I particularly enjoyed two recent ones, Authentic Marketing[0] and Principled Products[1] (which also has some background about 12 Factor Apps, if you remember that). Very useful for our stage where we have a new thing, but we're still trying to figure out how to explain it.

Thanks Adam, et al!

[0] https://overcast.fm/+Y-HVh9MxU [1] https://overcast.fm/+Y-HUBwJwA

bfirsh··on Launch HN: Nestybox (YC S20) – Containers beyond microservices
Ex-Docker person here. I got an early peek at Sysbox and I'm really excited by it -- it's really neat.

Docker is missing a bunch of features that make some software work, which is why you can't run Docker inside Docker by default. Instead of dropping from containers all the way down to hardware virtualization, Sysbox is "augmenting" containers with the missing features by simulating them in userland. That gives you all the power of a VM, without any of the downside of slow start-up speed, provisioning blocks of memory, not being able to run them on EC2, etcetc.

It reminds me a bit of user-mode Linux [0], weirdly. There's something kinda interesting about simulating a bunch of the kernel in userland.

[0] https://en.wikipedia.org/wiki/User-mode_Linux

bfirsh··on The Overfitted Brain: Dreams evolved to assist generalization
If you’re on a phone, here’s an HTML version: https://www.arxiv-vanity.com/papers/2007.09560/
bfirsh··on When Chevrolet Ruled Uzbekistan (2019)
I visited the country a couple of years ago, and a thing that puzzled me was why almost all the cars were white. Do you know why this is?

Some said it was because it is better at keeping the cars cool, but this doesn't explain why surrounding Kazakhstan/Tajikistan/etc had different color cars.

The best explanation I heard was the president liked white cars, so only allowed white cars to get made. This seems most plausible to me, and seems like similar things happened in Turkmenistan: https://www.motor1.com/news/226932/turkmenistan-president-ba...

bfirsh··on Tracking Pico Balloons Using Ham Radio [pdf]
The tracker is very cool: http://habhub.org/

It’s a crowdsourced set of antennas around the world that upload data to central server (think Flightradar24 for high altitude balloons). It’s been running for 15 years or so.

Various bits of more reading if you’re interested in this stuff: http://picospace.net/ https://ukhas.org.uk/

bfirsh··on Space Jam's 1996 website is still alive
It's got a status page, too: https://twitter.com/spacejamcheck?lang=en
bfirsh··on I tried to cancel Virgin Media broadband contract
I had the opposite experience with BE when I left the UK. I found out there the account wasn’t cancelled properly and there was an outstanding debt on the account.

I emailed, then they replied and admitted fault, apologised, and refunded me. No sitting in phone queues, no treating me like a criminal.

Unfortunately looks like they got bought and shut down. https://en.wikipedia.org/wiki/Be_Un_Limited

Seems like the good guys lose in this business.

bfirsh··on NSTM: Real-Time Query-Driven News Overview Composition at Bloomberg
If you're on a phone, here's a responsive HTML version: https://www.arxiv-vanity.com/papers/2006.01117/
bfirsh··on Seven years later, I bought a new MacBook. For the first time, I don't love it
It’s got a name! https://en.m.wikipedia.org/wiki/Electrovibration

As others have said, it’s due to a little bit of current passed through to the outside of the laptop which would otherwise have been passed through ground, not static.

bfirsh··on Have you ever asked yourself “how did research get done before LateX?”
The important/difficult problem of generating beautiful web pages is most of the way there already: https://github.com/arxiv-vanity/engrafo

There are some features in LaTeXML, which powers Engrafo, for adding some "LaTeX++" features, like embedding JS, etc.

Still could do with a lot of work, though.

bfirsh··on Volkswagen Currywurst
https://us.peugeot-saveurs.com/en_us/pepper-mills

Peugeot, the car company, invented the pepper grinder and still makes them.

They started making pepper grinders before they made cars though, so perhaps you could argue they're a kitchen tools company that also makes cars.

bfirsh··on Honda bucks industry trend by removing touchscreen controls
My 30 year old Land Rover has one of my favourite user interfaces.

For example, the cooling system. It's a big lever on the dash connected directly to a big flap on front of the car. You pull down the lever and it opens the flap, and air gets rammed in through the vent. The only moving part is the lever.

Everything is just so tactile. The cooling system, the gear stick, the transfer box, the switches. The user interface goes CLUNK and you can feel the thing on the other end doing the thing you made it do.

bfirsh··on Launch HN: Release (YC W20) – Staging environments made easy
As the creator of Compose (née Fig), I am very excited to see this. We initially created Compose with the intention of it being used to deploy to staging and production environments.

Docker Swarm was intended as the target of Compose deployments, but that never materialized because Swarm didn't catch on. I'm very glad to see someone carrying the torch in a Kubernetes world. :)

bfirsh··on Launch HN: Got-it (YC W19) – Bluetooth labels for tracking things at work
Newer tiles have replaceable batteries, which is great, but the find phone button is even bigger and easier to press accidentally. Arrgghh.

You can disable find my phone individually for each tile in the app, but there’s no way to disable for all tiles and it just stops the phone ringing so the tile still bloops.

bfirsh··on Twitter prepares for cull of inactive users
This is actually useful for me. The name of my new company is taken by a 10 year old account that has never tweeted, and I haven’t managed to find somebody at Twitter who can get the name for me.

Does anybody know what’s going to happen to the names? Are they going to become available to register on 11th Dec? I hope somebody isn’t waiting there with a dictionary to squat them all...

← PreviousPage 4 of 9Next →