Replit's in-browser coding AI
replit.com
replit.com
https://intuitiveexplanations.com/tech/replit/
Looks like this CEO isn't of good character after all. He looks almost like a jerk when looking at the end of the story. Even in his last email he tried to get his (obviously wrong) point. He never apologized for the things that mattered most, only tried to extinguish the social media fire all in all.
Also he doesn't look very smart, imho:
Big LOL here! The abstract things are the simplest, yeah! That's why progress in something like math or theoretical physics is made by the dumbest people, in contrast to something like sociology where you need genius level of intelligence to come up with some new ideas. Sure, sure.
But that's of course not everything this dude got completely backwards.
Would explain why replit is the most useless of all the online IDEs: It has no direction, no true value proposition. It's not a good cloud coding environment. It never was a good code snippet playground (actually one of the worsts). Now they even require accounts, so the quick code snippet aspect is also gone. Also they badly positioned in the education space…
Of course I wish them luck!
But I guess they have no chance against something like Gitpod, Github, or OpenShift codespaces, which are light-years ahead.
OK, maybe the exit-strategy is "just" to be visible enough that at some point they get bought by one of the above. (Which doesn't look like the most ethical thing to do ;-)).
I feel like, if there's ANYTHING we have learned in the past decade or two it's that people who defend a company tend to be doing so for the wrong reasons. See Sony or Microsoft, or Apple or Android, etc. Defending a company is just weird.
I look at replit as a tool, run by people. The tool might be cool, but the CEO made a bad decision and now I judge the product on that CEOs actions. There's no definitive time frame or action that just magically makes it better.
But in general, I'll stop thinking about the stupid actions of the CEO when my brain stops reminding me "Oh, no matter how cool this is, the actions of the CEO were incredibly poor." When will that be? No idea, but maybe sometime down the road he does enough good things that I will suddenly stop and think "cool, looking back, he's done enough good that I can probably forget about the poor decision he made and start looking at this again, because he's proven he isn't that one stupid action."
Goodwill is earned, it's not simply given. It's often hard won, but incredibly easy to lose.
Google => when will they cancel this?
Meta => metaverse what a joke
Etc...
We do it every day, whether we realize it or not. As a people we should support the companies that do good, and we should be aware of the companies doing bad. There's room for grey in there, it's not a one size fits all. But if you aren't aware of the bad then you aren't informed in your decision making.
If it bothers you, sorry. But I see it as a pro. I had a really bad experience with Remarkable, every time it comes up, I point out my experience so that others who might be making their own decision can use my experience in their decision making. When a company performs poorly. I guess by the metric you've provided above, this would be a low effort comment by your definition.
iirc the team managed to bring it to a lever an order of magnitude lower than off-the-shelf models
I think it's also a bit scary, because 8 weeks is very little time for testing, tuning, and validation of something as opaque as a machine learning model. If it worked right the first time, that's great. But there is still a lot of inherent uncertainty in ML projects. Decision makers need to take that uncertainty into account when planning.
That, or, the 8 weeks only covers the final training runs and the implementation/deployment, and doesn't include time spent developing and tuning proof-of-concept prototype models.
To me (programmer in Sweden) the largest single team I've been on was 14 people and that was _very_ large (indeed the largest in the tech department). We actually broke ourselves up into two more informal groups since we thought that was a more manageable team size.
can i get a clarification - when it says "in-browser" i hear "on-device" as in it doesnt call back to replit to get the predictions. i assume that's inaccurate?
for cost/compute purposes i'm wondering how small models have to get in order to run "truly in browser"
I've gotta let you know that people copy code snippets from all sorts of codebases with little regard for licenses anyway, because they're toothless in 99% of cases, AI or not. It's a nice illusion that anyone respects licenses, but it's just not true.
It's been proved in court that AI does not infringe on copyright or licenses since it generates things from an understanding of the whole, instead of directly stealing, just like the human brain does.
Do you have a source for that?
This SF Conservancy article[0] says that's not true:
>Consider GitHub’s claim that “training ML systems on public data is fair use”. We have not found any case of note — at least in the USA — that truly contemplates that question.
The first major court case I know about is the class-action case Matthew Butterick is trying to build.[1]
[0] https://sfconservancy.org/blog/2022/feb/03/github-copilot-co...
If you ever plan to steal someone's code and justify it with "my brain is able to learn, therefore copyright doesn't exist" I warn you right now this will not fly.
If the “new” code is close enough to be considered a derived work then you will need a license.
And how is that determined... in court at trial? By an unbiased 3rd party competent enough to understand both codebases?
Hypocrisy dressing up as progress in the machine learning field has been one of the most embarrassing scenes in software engineering recently. The genie may be out of the bottle but the fact is that a bunch of software engineers with a “move fast ethics later” attitude are the ones who let it out and they shouldn’t get to shrug it off for free.
But as long as these are tiny parts of codebase (which will most probably be the case), I doubt anything can be done with that. No one will go to court because of a few generic functions.
Fully respecting the licenses this code was published under, one would hope?
For a simpler example than code-generating ML, if I write a program to recognize a directory of source code vs non-source code, and my logic is if (unbalanced parenthesis count in all files > X) { return "not source code"; } else { return "source code"}.
And then I compute X by scanning over the linux kernel source and counting the amount of unbalanced parens, have I just committed a GPL violation if I don't GPL my source code recognizer?
"Where's the line between two separate programs, and one program with two parts? This is a legal question, which ultimately judges will decide. "[0]
[0]: https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation
Unlicensed source code - the default - is still protected by copyright law. If it's hosted and served from a different jurisdiction where no exception exists for training data models.
Then there are also licenses that explicitly prohibit commercial usage to consider.
What it comes down to, as it always does, is that a small group of (practically) untouchable people are making money by abusing and thereby irreparably damaging the trust and good will of the collective.
It's a complex topic eh
That's the main difference from a human learning from copyrighted code (which is totally legal). If they have a good memory they might be able to reproduce copyrighted snippets, but they would usually (probably not always!) know they are doing that.
It blew my mind half of the times. It was like it knew what I was going to do.
The other times it was dumber than the standard auto complete. It doesn't have any awareness of already defined variables and doesn't use them to complete halfwritten variables. Hope this gets better soon.
Overall, I like my Jetbrains IDE but Replit is coming with appealing features for the side projects specially. Easy to use auth, db, analytics, deployment...
I wish either Replit levels up its IDE game or Jetbrains(or community) build plugins to match the state of art/joyful experience of programming.
ref: https://github.blog/2022-09-14-8-things-you-didnt-know-you-c...
- What areas/languages/tasks is it good at and what is it bad at?
- How often is it generating code with bugs?
- How often is the code that gets generated used as is vs immediately edited?
- What are the new frustrations that this causes that existing IDE code completion doesn't?
I work with trained models daily and I know that their failure cases are unintuitive, unexpected, and exasperating. I'd like to know as much about the failure cases of this model as possible before diving in.
- Ghostwriter is especially good at Python and JS, but supports up to 16 languages (to varying degrees of effectiveness) you can read more here: https://docs.replit.com/ghostwriter/faq#which-programming-la...
- As for tasks, it's great for reducing the amount of boilerplate code you need to write (Complete Code), writing React components (Complete + Generate Code), explaining code in plain English (Explain Code), translating code between languages (Transform Code), writing exhaustive tests (Complete Code)
- No stats on how often its generating code with bugs or how often the code gets generated used as is vs immediately edited, we're interested in getting both to help improve Ghostwriter
- Like any LLM, it can get stuck in a long-tail of repetitive loops; we're working pretty hard to improve and mitigate these issues, but, especially for new users, the repetition and hallucination type problems can be distracting.
- Developers who work at a company (e.g., as employee) and need to spit out features every sprint? Velocity is important, so I imagine these kind of developers need to squeeze every minute they are in front of the screen in order to produce working code?
- Developers who think of written code as one way to solve (tech) problems, so they don't really care much about the process of creating code, but mainly about the output (i.e., does the running program solves the issue at hand?)
- Senior developers who don't like to write boilerplate code?
I don't see myself as the target audience of Copilot or Ghostwriter. I do work as an employee, but I'm not a "feature machine". Usually the hardest part about my job is solving problems while communicating with other people. I don't need to write code "fast", and by the time I hit the keyboard to start coding, I don't really need that much help (granted, I'm not working on code that goes into space rockets... just normal e-commerce stuff)
I like to work on side projects and learn new technologies. When I was starting with programming, as part of the learning I liked to write boilerplate code (actually, that's how I learnt programming. I remember writing C boilerplate code by reading "The C Programming Language". Skipping the "boring" parts wouldn't have helped me in my learning).
If any, Copilot and similar tools take away all the joy of actually writing code (because, when I work on side projects, 50% of the satisfaction comes from actually writing code for the sake of writing code. The other 50% comes from the ability to solve a problem). So, yeah, maybe for the people like me who does find the act of writing code for the sake of writing code (you know like painting or taking photographs), Copilot seems like an unneeded tool?
So we have been considering using Codex or something for generating the code in a more streamlined version, the key reason of it being a benefit is we are a small team with each person owning more than one large repositories. It's gotten very annoying and our pace is far slower than what we would like, here something like this makes quite a lot of sense.
Though the problem with such specific tools is they can't generate any customized code for our codebase, we can finetune other codegen models and that's what we plan to do down the line, but this specific tool just not really useful if it can't specialized for our codebase.
you can't get around it. We have pretty low boilerplate in all the codebases I happen to manage but the sad part is there is no getting around porting of specific rules, setting up better metric analysis and reporting systems and such.
If you have been involved in programming professionally for a while, you would know you just can't get around the chore like works sometimes. Ofc it's not a long term goal to keep going this way but we needed a solution to simplify our challenges as we move on.
See it as a better autocomplete for people who don't want to or can't learn by diligently doing the boring parts.
My primary UX issue with copilot is that it is trying too hard to be helpful, often suggesting code that I don't need. You also can't trust it with more complex cases but that's actually pretty reassuring : - )
I like to challenge my own beliefs, and I reluctantly tried it out. At least I would have a basis for my criticism, so I thought. I'm about 10 hours into using it, maybe.
If anything it has increased the joy of writing code for me. It eliminates the mundane busy work and lets me focus on solving problems. For me, the "real" coding happens in my head, putting it into an editor is just process. I still also have to check it's work whenever I use it, so I'm still deeply embedded in the coding process.
I believe it's akin to an easel vs Krita/Photoshop. Some people enjoy interacting with the physical medium, others enjoy the creative process.
I would strongly recommend trying it out in anger (i.e. a reasonably real codebase), at least for 30 minutes. Form a better opinion after that (which may well be the same as yours right now).
For reference: I've been coding since I was 8 (almost 30 years).
For me, same as writing actually, the thinking of what I want to do is everything, and the doing it is nearly trivial. I don't picture having copilot write nontrivial code for me and then reviewing it would be different than writing it, even if I didn't know the exact syntax and had to look it up. So I agree, it feels like a solution to a problem I don't have, like is solves something that I don't spend time on.
Cynically, like GPT-3 probably help write content farm stuff, copilot probably helps write some junk code for something, but there are probably domain specific low code tools that do that better as well
Plus, Ghostwriter is integrated with the Replit platform, meaning you get all the benefits Replit offers as a portable development environment you can take with you anywhere you go and host instantly.
I can't count the number of times I've copy pasted a chunk, commented it out, then put a comment describing what I'm after. Copilot will get it exactly right ~40% of the time, ~40% of the time it gives me a good starting place, and the other 20% I just scrap.
If you are doing something new, you need to let people see what it will do, before they will give you money.
At the bottom of the original blog post. He put his project back online and it is still up: https://riju.codes/
> When someone shows you who they are, believe them the first time.
- Maya Angelou