Here's a script to install and configure, in case it's helpful for anyone's dotfiles: https://github.com/bfirsh/dotfiles/blob/48eff70daa754216eff9...
4,246 karma · joined November 29, 2009
https://firshman.com
[ my public key: https://keybase.io/bfirsh; my proof: https://keybase.io/bfirsh/sigs/I2UKihiqAntrGCCwICWwJXcyy9Bt95eBXaBlml8VkuQ ]
To me, it was obviously possible, and I was determined to prove them wrong.
Anyway, this now exists because of that: https://github.com/bfirsh/jsnes
Is this duct tape over historical architectural decisions that assumed trust? Could we design something with less complexity if we designed it from scratch? Are there any operating systems that are designed this way?
Compute in R&D will be only training and development. Compute for inference will go under COGS. COGS is not reported here but can probably be, um, inferred by filling in the gaps on the income statement.
(Source: I run an inference company.)
If something's not as fast let me know and we can fix it. ben@replicate.com
Replicate makes it easy to run AI in the cloud. You can run a big library of open source models with a few lines of code, or deploy your own models at scale.
We're an experienced team from Spotify, Docker, GitHub, Heroku, NVIDIA, and various other places. We're backed by a16z, Sequoia, NVIDIA, Andrej Karpathy, Dylan Field, Guillermo Rauch.
We're hiring:
- An infrastructure engineer
- A machine learning engineer who's an expert at image models
- An engineer who likes talking to people to look after our customers
... and more: https://replicate.com/about
I am from the UK and it makes me wonder why road design in the US is so bad. Just one minute of thinking about this as a lay person would reveal the problem with the design.
Is there some structural reason in the US that would cause it? Perhaps some lack of standards or approval process? Perhaps iteration speed is slower so they don’t get better? Some other incentives going on?
Replicate makes it easy to run AI in the cloud. You can run a big library of open source models with a few lines of code, or deploy your own models at scale.
We're an experienced team from Spotify, Docker, GitHub, Heroku, Apple, and various other places. We're backed by a16z, Sequoia, Andrej Karpathy, Dylan Field, Guillermo Rauch.
We're hiring:
- An infrastructure engineer
- An expert at deploying and optimizing language models
- An engineer who is good at humans to look after our customers
... and more: https://replicate.com/about#join-us
Email us: jobs@replicate.com
Official blog post: https://www.snowflake.com/blog/arctic-open-efficient-foundat...
Weights: https://huggingface.co/Snowflake/snowflake-arctic-instruct
It's missing the name though (Arctic). That might be worth adding.
You can try it out here: https://arctic.streamlit.app/
Weights are here: https://huggingface.co/Snowflake/snowflake-arctic-instruct
You can also chat with it here: https://llama3.replicate.dev/
Incentives are aligned for us to make it better. :)
Here's what we're doing:
- Fine-tuned models now boot fast: https://replicate.com/blog/fine-tune-cold-boots
- You can keep models switched on to avoid cold boots: https://replicate.com/docs/deployments
- We've optimized how weights are loaded into GPU memory for some of the models we maintain, and we're going to open this up to all custom models soon.
- We're going to be distributing images as individual files rather than as image layers, which makes pulling images much more efficient.
Although our cold boots do suck, the comparison in this blog post is comparing apples to oranges because Fly machines are much lower level than Replicate models. It is more like a warm boot.
It seems to be using a stopped Fly machine, which has already pulled the Docker image onto a node. When it starts, all it's doing is starting the Docker container. Creating the Fly machine or scaling it up would take much longer.
On Replicate, the models auto-scale on a cluster. The model could be running anywhere in our cluster so we have to pull the image to that node when it starts.
Something funny seems to be going on with the latency too. Our round-trip latency is about 200ms for a similar model. Would be curious to see the methodology, or maybe something was broken on our end.
But we do acknowledge the problem. It's going to get better soon.
Hypthetically, I found a remote code execution vulnerability in that version of phpBB, read the configuration file to get the MySQL details, and then used mysqldump to download the database.
You can find exploits by just Googling or looking in Metasploit. They’re usually pretty simple query string things.
We set up a clone of the forum with a similar name just in time, then emailed the user database to tell them about the new site when the old one disappeared.
Sadly this caught the previous owner’s attention and he sent a cease and desist, despite his version of the site not existing any longer. So, we wiped the database, and then all the users just signed up again. It still lives on as https://www.basschat.co.uk/
Hypothetically.
Somebody happened to stumble across our fork of the model and submitted it. We didn't submit it nor intend for it to be an ad. I hope the submission gets replaced with the upstream repo so the author gets full credit. :)
[0] https://github.com/Vaibhavs10/insanely-fast-whisper/pull/42
Replicate makes it easy to run machine learning models in the cloud. You can run a big library of open source machine learning models with a few lines of code, or deploy your own models at scale.
We're an experienced team from Spotify, Docker, GitHub, Heroku, Apple, and various other places. We're backed by a16z, Sequoia, Andrej Karpathy, Dylan Field, Guillermo Rauch.
We're hiring for:
- An infrastructure engineer who is an expert at deploying ML systems
- An engineer who is good at humans to look after our customers
- An engineering manager based in SF
... and more: https://replicate.com/about#join-us
Email us: jobs@replicate.com
OpenAI aren't doing anything magic. We're optimizing Llama inference at the moment and it looks like we'll be able to roughly match GPT 3.5's price for Llama 2 70B.
Running a fine-tuned GPT-3.5 is surprisingly expensive. That's where using Llama makes a ton of sense. Once we’ve optimized inference, it’ll be much cheaper to run a fine-tuned Llama.
Replicate makes it easy to run machine learning models in the cloud. You can run a big library of open source machine learning models with a few lines of code, or deploy your own models at scale.
We're an experienced team from Spotify, Docker, GitHub, Heroku, Apple, and various other places. We're backed by a16z, Sequoia, Andrej Karpathy, Dylan Field, Guillermo Rauch.
We're hiring:
- ML engineers
- An engineer who is good at humans to look after our customers
- Hackers to build cool things with machine learning to show you how to use Replicate
... and more: https://replicate.com/about#join-us
Email us: jobs@replicate.com
We're an experienced team from Spotify, Docker, GitHub, Heroku, Apple, and various other places. We're backed by a16z, Sequoia, Andrej Karpathy, Dylan Field, Guillermo Rauch.
We're hiring for:
- A generalist engineer who has a refined sense of what makes a good developer tool
- An engineer who is good at humans to look after our customers
- Hackers to build cool things with machine learning to show you how to use Replicate
... and more: https://replicate.com/about#join-us
Email us: jobs@replicate.com
We've got some big cold improvements rolling out across everything soon. We can also just keep models switched on to avoid cold boots entirely.
It's also worth noting that we bill by the second for how long your prediction is running, and we don't bill for any idle time, so in practice Replicate works out cheaper for many workloads. We can give discounts if you're putting through a decent amount of traffic. We should be able to match Runpod's pricing.
Drop me an email: ben@replicate.com
Replicate makes it easy to run machine learning models in the cloud. You can run a big library of open source machine learning models with a few lines of code, or deploy your own models at scale.
We're an experienced team from Spotify, Docker, GitHub, Heroku, Apple, and various other places. We're backed by a16z, Sequoia, Andrej Karpathy, Dylan Field, Guillermo Rauch.
We're hiring for:
- A generalist engineer who has a refined sense of what makes a good developer tool
- A designer who can code or understands developer tools
- A machine learning engineer who is an expert at running models in production
... and more: https://replicate.com/about#join-us
Email us: jobs@replicate.com
Somebody who does this for fun, normally with some kind of puzzle that needs solving to get the address.
Replicate makes it easy to run machine learning models in the cloud. You can run a big library of open source machine learning models with a few lines of code, or deploy your own models at scale.
We're an experienced team from Spotify, Docker, GitHub, Heroku, Apple, and various other places. We're backed by a16z, Sequoia, Andrej Karpathy, Guillermo Rauch.
Hiring for a technical designer, infrastructure engineer, machine learning engineer, and more. https://replicate.com/about#join-us
Email us: jobs@replicate.com