Half the tasks they were not allowed to use AI.
47 karma · joined January 9, 2015
Metr may overall have an ok mission, but their motivation is questionable. They published something like this to get attention. Mission accomplished on that but they had to have known how this would be twisted.
They used 16 developers. The confidence intervals are wide and a few atypical issues per dev could swing the headline figure.
Veteran maintainers on projects they know inside-out. This is a bias.
Devs supplied the issue list (then randomized) which still leads to subtle self-selection bias. Maintainers may pick tasks they enjoy or that showcase deep repo knowledge—exactly where AI probably has least marginal value.
Time was not independently logged and was self-reported.
No possible direct quality metric is possible. Could the AI code be better?
The Hawthorne effect. Knowing they are observed paid may make devs over-document, over-prompt, or simply take their time.
Many of the devs were new to Cursor
Bias in forecasting.
16 devs. And they weren't allowed to pick which tasks they used the AI on. Ridiculous. Also using it on "old and >1 million line" codebases and then extrapolating that to software engineering in general.
Writers like this then theorize why AI isn't helpful, then those "theories" get repeated until it feels less like a theory and more like a fact and it all proliferates into an echo chamber of AI isn't a useful tool. There have been too many anecdotes and my own personal experience to ignore that it isn't useful.
It is a tool and you have to learn it to be successful with it.
So some "experts" could be staying quiet because they don't have one. But some may stay quiet because they are working on or benefiting from it?
Looking at the OI docs wasn't too helpful.
"I did spend a non-trivial amount of time fiddling with the prompts" was it writing prompts?
I am really interested and this seems like a cool use case that I want to explore. Could you share the prompts on a github gist?
This would allow crowd sourcing and statistics to be applied by the best minds in the world.
I know regulations can make this difficult but many difficult boundaries are being pressed now.
What industry? How long you worked on it, etc?
https://gitlab.com/gitlab-org/gitlab-ce/issues
also some other rails apps to look at:
discourse https://github.com/discourse/discourse
dev.to https://github.com/thepracticaldev/dev.to
just recently mentioned in another thread consul https://github.com/consul/consul
Or search for large open source rails apps.
Having fully remote employees within the US is a large burden in and of itself. It is very very hard unless a company has resources that can address that. Each employee in a different state or location is basically treated like a separate office. You are basically asking a company to open an office in Mexico.
'1. A method of treating a symptom of diabetes, the method comprising: identifying a subject exhibiting symptoms caused by pancreatic beta cell destruction, the subject having a normal caloric intake; and administering multiple cycles of a diet protocol to the subject wherein a fasting mimicking diet is provided for a first time period and a re-feeding diet is provided for a second time period, the fasting mimicking diet providing less than about 50% of the normal caloric intake of the subject with both protein restriction and sugar restriction and the re-feeding diet providing 60-100 percent of the normal caloric intake of the subject, depending on a need to lose weight wherein the first time period is from 2 days to 6 days and the second time period is from 7 days to 85 days.'
This seems like un-patentable subject matter based on USPTO guidelines (https://www.uspto.gov/sites/default/files/web/offices/pac/da...). It seems to me that it fails the machine or transformation test for a process patent (https://en.wikipedia.org/wiki/Machine-or-transformation_test). The only way I could reason that it was patentable is if the 'diet protocol' has to be followed strictly based on the definition in the patent. The diet protocol has a long section with very specific micro-nutrient content. So that is the 'product', a calorie restricted diet with a narrowly defined micronutrient supplementation.
If there is anyone well versed, what is your opinion on the patentability of the first claim?
'Can you work as a sw dev? I'm probably not as skilled as a professional dev. I would never be time-efficient as a professional.'
I have seen many fellow developers say things like this (impostor syndrome). Productivity as a code monkey is rarely relevant to jobs and probably not relevant to the jobs you want. You have worked in a variety of roles and have lots of life experience. This may make you many times more efficient in other essential parts of software engineering. Clear communication, problem solving, planning, understanding actual customer and business needs. The most efficient software engineers I have seen don't type [insert some huge number] lines of code a day, they ask good questions and fully understand the domain before the tackling a problem.
As for your problem finding a job, I would be working on creating a portfolio of code samples/projects in the field I wanted to work. Also I would make sure to tailor my resume depending on who I sent it to and what the job description lists.
One successful approach I have seen is contacting companies you want to work for (works best on smaller companies) and trying to talk to someone before giving them a resume. This can be accomplished at meetups or cold calling/emailing :shudder:. When you talk to them you identify what real problems they are facing. Then tailor the resume to those specific problems (or even throwing in a case-study of how you would approach and solve them).
When CS majors or programmers estimate how fast/efficient code is they generally are intuitively applying big 'O' notation or some similar concept. This can be gained from an intro level algorithms class. Or you could look up and read about the topic.
The classical coding interview questions mostly stem from an intro algorithms and data structures class. Or you could get a book on interview prep and work through a couple a day.
Finally, the coding style issue usually is ancillary knowledge and not found (or at least not done well) in typical CS curricular. I would suggest the book "Clean Code" by Robert Martin. It goes through code smells, commenting, general style, and is well written.
So overall, you could look at Algorithms and Data Structures MOOC's. Also read a book on CS interview prep and writing maintainable code. Hope this helps a little. Good Luck!
I would argue that the current pre-req system is responsible. It lumps a full class as a pre-req. And someone can pass a full class while not understanding pieces of it. Those pieces could be the essential ones needed for the specialty knowledge a person pursues. If instead of saying generally, this requires a 200 level linear algebra course and instead enumerated the topics from that course that were necessary than people would know the foundation that is needed. You saying that 'fluency' is required in some broad course is intellectual elitism.
"I see the pursuit of quick and shallow ways of learning advanced subjects"
Also in what way is clearly defined prerequisite knowledge 'shallow'. And where is this coming from:
"People who only ever search for the quick secret trick to learning X"
You are injecting biases into this debate that weren't there in the first place to make it seem like I am advocating some kind of get rich scheme for learning things. I am actually advocating clearer and more descriptive knowledge ontologies. This would allow for better learning efficiency and overall may benefit the field. People would be able to spend more time on depth in the field they want to pursue.
Also using 'never' in an argument is a habit you need to break. It is hyperbolic and usually trivial to find counter examples.
I strongly disagree with this statement. I see many college curricula as hugely time inefficient. Especially when you want to learn deeply on a specific topic. I personally disregarded the prerequisites for many elective engineering classes because of scheduling constraints. And anecdotally I found you only needed small pieces of the prereq's.
You can argue for a 'foundation' all you want. I was simply stating that I would like a detailed list of knowledge required to take a course instead of a generic 'you need a 100 level linear algebra course.' A detailed list would then allow people to decide if they know/remember enough info or need to learn/brush up on a topic. Speaking of linear algebra, seems like a struck a nerve with my made up example.
I also think it is a disservice to say that you need 'understanding [of] those subjects deeply if you want a high ROI.' These kind of statements discourage learning. I have seen many people struggle with so called necessary prereq's only to flourish in more advanced classes.