amasad

We do let you change it; under your account setting there is a email with an edit button next to it. And it shouldn’t need to match your GitHub email.

Perhaps there is some weirdness if you’ve signed up with GitHub. Feel free to email me and we can take a look: amjad@repl.it

We're on a budget :) trained on 128 H100-80GB GPUs for a week (200B tokens over 5 epochs, ie 1T tokens).

Tech talk here with timestamp: https://www.youtube.com/live/veShHxQYPzo?si=UlcU9j2kC-C4oWvj...

Crates is especially challenging -- the build process is very expensive on all resources (cpu, ram, and disk), and packages are very hard (impossible?) to cache. It works super well on the Pro plan.

It’s communication meant to existing users and it’s a nuanced picture worth elaborating. But if you are not an active user it’s best if you just look at our pricing page for a complete picture: https://replit.com/pricing

This is for hosting. Development remains free

Yes, in fact we recently made the free plan more powerful with non-preemptible VMs, premium networking, and package caching for everyone. And working on an AI free tier coming soon. Will likely invest more in the free plan once the abuse from unlimited hosting go away.

Also — the title is slightly misleading. Static hosting is still free, and autoscale (with scale to zero) will effectively cost next to nothing for most users. Especially new programmers get very little traffic that according to our historic data it will cost $0.2 / month.

Finally beginners who cannot afford to pay but learned enough coding can make money on the site. eg, a story earlier today: https://x.com/replit/status/1709577707435274414?s=46

the blog is slightly outdated — 3000 is out of beta and can be used now. Here is a fun game built with it: https://replit.com/@slmjkdbtl/Bean-Survivor

Is this the best you can find? not even top 10 bangers.

Interesting. This seems like a weakness of natural language understanding. If you rephrase your prompt slightly it would get it right. Try:

  // return even numbers that are also more than 10
  const arrayFilter = (array) =>

It would do the right thing. The fine-tuned version gets your prompt right so maybe it benefited from natural language data. Will look more into it.

Yes we have a robust extension system and some are already building alternatives.

Broadly finetuning is any post pretraining training. Most of the time it is used in the context of fitting a more narrow task. In our case, it was the same training objective as the pretraining but meant to be more representative of what Replit users like to code. However, we were surprised by how well it boosted overall performance. Best guess: it's a) novel data and b) the model could take even more training!!

The model is not RLHF'd or instructed. It's an inline autocomplete model so it will get confused if you talk it like you're talking to a person. Altho it is possible to finetune it this way. To get better full function completion try giving it the function definition and a descriptive docstring as a prompt.

Can't find it now but pretty sure BigCode said somewhere they explicitly looked for it and removed it. Also subjective measure does match up to the benchmark. Our finetuned model performed +50% on HumanEval and then when using it felt at least that much improved.

1- We trained on languages that are most popular on Replit. Markdown is important because you need some amount of natural language in the data, and it will act as a sort of "natural language label" for code.

2- I like how portable it is being a single small model doing a lot of languages. Single code models are an approach that models like Salesforce/Codegen did that, but I believe we beat (or get very close) to their mono models on benchmarks.

Hopefully on phones too

it is 2.7B

LLMs generally but more so small models will keep going and generate seemingly unrelated things. On the frontend tools like Copilot and Ghostwriter do a lot of things like use stopwords or simply not show completions outside a single block.

As for your prompt, it's following your prompt a little too closely and generating just the function. You can however condition it that this is the start of the program it will do the import, e.g.

   # python function that returns a random integer between min and max
   import

This is in fact a suggestion from OpenAI on best practices for prompting called "leading words" https://help.openai.com/en/articles/6654000-best-practices-f...

Some links:

- Repo: https://github.com/replit/ReplitLM/tree/main/replit-code-v1-...

- HuggingFace: https://huggingface.co/replit/replit-code-v1-3b

- Demo: https://huggingface.co/spaces/replit/replit-code-v1-3b-demo

- Early benchmark results: https://twitter.com/amasad/status/1651019556423598081

A lot about this project was surprising. We knew it was going to be good, but didn't expect to be this good -- especially surprising was the finetuned performance boost, and the fact that the model is decent at language tasks and reasoning (in some cases much better than much larger general-purpose models).

It feels like there is a lot more to do with this model, and I have a suspicion you can even make a half-decent chatbot (at least one focused on code) by finetuning it on conversation (and/or instruction) datasets.

Will follow up with a more comprehensive technical report and the UL2R version (fill-in-the-middle support).

Yes we have stories from all over the world where people are using mobile to build real things. There is a kid in Egypt, for example, that built discord bots for a living on his phone.

Additionally there are also lots of cases were fairly well off folks with access to desktop computers and wifi that find themselves with a phone and want to prototype an idea or make a quick change to a project. I’ve been using it and I find it relaxing to lay back on the couch and do some fun coding. Especially with Ghostwriter (our Copilot-like thing: https://blog.replit.com/ai) it’s super usable. I built large part of my toy Lisp in Python in the app while in the park, waiting at the doctor’s office, or simply relaxing after a long day: https://replit.com/@amasad/Lisp-in-Python

We've had Code Explanation (based on Codex)[1] in production for over a month and while it leaves a lot to be desired it was surprising that 75% of users thought the explanation were useful to them.

[1]: https://blog.replit.com/codex

If you fork it you can program it in the IDE including writing C and ASM programs.

But it’s a good idea — will make it a template!

Yes, most things on Replit are open-source. Just click "show files" on the top-left corner. Also it's powered by Nix, and we wrote a ton about it here: https://blog.replit.com/

Added nix.dev to the list of resources in the post. Thanks.

I think donating to the Nix foundation would be as good as any.

To see and play with what people are building with Nix on Replit check out the #nix hashtag here: https://replit.com/apps/nix

Users vary from teachers building environments for their classrooms, to hackers tinkering with emulators, to kids making old games like run online.

We think we’re just scratching the surface with what we can do with Nix. It’s an outrageously underrated technology.

We’re trying to make it more accessible. Making a GUI or a no code interface is also on the table. But nonetheless we are seeing kids build nix environments. See this hashtag: https://replit.com/apps/nix

Done.

Hey everyone, I want to apologize for the inappropriate use of power here. While I do believe there is an ethical line that was crossed here, I should have called him to understand his point of view and work it out. Which is what I'll try to do now, and see if we can get his project back up again. I'm sorry Radon.

The lesson for me here is to internalize how I'm no longer the struggling kid from Jordan fighting for more than a decade to build something, and that I now have a responsibility towards our community and supporters to be kind and model better behavior. I'm sorry I let you down and I promise to do better in the future.

Thank you -- this is really good feedback.

That was a clone of Replit that we made work at Codecademy. I started working on Replit (or repl.it) back when I was a student in Jordan. I didn't have a laptop so every time I wanted to get some programming done I had to setup a development environment at the university or at work. The idea for Replit was when you needed a repl to do some coding you should easily get one from anywhere including a mobile device. I thought it would benefit many people, especially those who don't have the means to buy expensive computers.

It took 2 years of work to get something working and in 2011 we launched on HN (2011 web archive snapshot here https://web.archive.org/web/20111007050930/http://repl.it/ and HN launch here https://news.ycombinator.com/item?id=3056490). It was the first of its kind and it inspired a lot of projects and still does today. It was totally open-source (https://github.com/replit-archive/repl.it) and after the launch it was used as infrastructure by Codecademy (which later employed me) and Udacity and many others to deliver interactive coding in the browser. I was thrilled about that.

Now, a lot of people implicitly assume that in a dispute between for-profit company and an open-source project, the for-profit company must be in the wrong. But there is some line that it's unethical to cross in copying a former employer's product (if you don't believe that, you can stop reading now, because no argument will convince you) and I think to someone who knew Replit's architecture well, this project would clearly be across it. It copied even unique, invisible aspects of Replit's architecture that I consider to be flaws. That's the hallmark of copying versus merely writing one's own program to solve the same problem.