siscia

You don't need AI for it.

You can just install the tool and use it. It is a CLI with very verbose output. Verbose output is good for both humans and AI.

The perspective from an educator is quite concerning indeed.

Students are very simply NOT doing the work that is require to learn.

Before LLMs, homeworks were a great way to force students to approach the material. Students did not have any other way to get an answer, so they were forced to study and come up with an answer to the homeworks. They could always copy from classmates, but that was considered quite negatively.

LLMs change this completely. Any kind of homework you could assign undergraduates classes are now completed in less than 1 second, for free, by LLMs.

We start to see PERFECT homeworks submitted by students who could not get a 50% grade in classes. Overall grades went down.

This is a common pattern with all the educators I have been talking with. Not a single one has a different experience.

And, I do understand students. They are busy, they may not feel engaged by all the classes, and LLMs are a way too fast solution for getting homeworks done and free up some time.

But it is not helping them.

Solutions like this are to force students to put the correct amount of work in their education.

And I would love if all of this would not be necessary. But it is.

I come from an engineering school in Europe - we simply did not have homework. We had frontal classes and one big final exams. Courses in which only 10% of the class would pass were not uncommon.

But today education, especially in the US, is different.

This is not forcing student to use LLMs. We are trying to force student to think and do the right thing for them.

And I know it sounds very paternalistic - but if you have better ideas, I am open.

I created something similar, but instead of final oral examination, we do homework.

The student is supposed to submit a whole conversation with an LLMs.

The LLM is prompted to answer a question or resolve a problem, and the LLM is there to assist. The LLM is instructed to never reveal the answer.

More interesting is the concept that the whole conversation is available to the instructor for grading. So if the LLMs makes mistake, or give away the solution, or if the student prompt engineer around it. It is all there and the instructor can take the necessary corrective measures.

87% of the students quite liked it, and we are looking forward to doubling the students that will be using it next quarter.

Overall, we are looking for more instructor to use it. So if you are interested in it please get in touch.

More info on: https://llteacher.blogspot.com/

I will be crucified by this, but I think you are doing it wrong.

I would split it in 2 steps.

First, just move it to svelte, maintain the same functionality and ideally wrap it into some tests. As mentioned you want something that can be used as pass/no-pass filter. As in yes, the code did not change the functionality.

Then, apply another pass from Svelte bad quality to Svelte good quality. Here the trick is that "good quality" is quite different and subjective. I found the models not quite able to grasp what "good quality" means in a codebase.

For the second pass, ideally you would feed an example of good modules in your codebase to follow and a description of what you think it is important.

I am toying with something similar.

However my approach would be to use duckdb and S3 over lambda.

Leaving many of the concerns to the infrastructure. Like basically no OOM. No need to manage servers.

With my partner we have been working to invert the overall model.

She started grading conversation than the students have with LLMs.

From the question that the students ask, it is obvious who knows the material and who is struggling.

We do have a custom setup, so that she creates an homework. There is a custom prompt to avoid the LLM answering the homework question. But thats pretty much it.

The results seems promising, with students spending 30m or so going back and forth with the LLMs.

If any educator wants to Ty or is interested in more information, let me know and we can see how we collaborate.

[flagged]

Just to echo the point of MCP, they seem cool, but in my experience just using a CLI is orders of magnitude faster to write and to debug (I just run the CLI myself, put test in the code, etc...)

This is absolutely neat.

I find myself on both sides actually.

I did have some great luck producing quite useful and impactful code. But also lost time chasing tiny changes.

Do you really?

Frontier models seems remarkably similar in performance.

Yeah some nuances for sure, but the whole article could apply to every model.

Also, do you have a better way to spend that money?

In fairness, their design does not seem to be regional. With problems in one region bringing down another, apparently not unrelated, region.

With this kind of architecture, this sort of problems is just bound to happen.

During my time in AWS, region independence was a must. And some services were able to operate at least for a while without degrading also when some core dependencies were not available. Think like loosing S3.

And after that, the service would keep operating, but with a degraded experience.

I am stunned that this level of isolation is not common in GCP.

As engineers we are (rightly) very scared of AI taking our job.

If I were a lawyer I would be even more scared.

What I see working extremely well, arguably in a setting where cost was not really an issue was a much simpler approach.

Keep compacting files at some regular cadence `t` and keep a bucket policy to delete files older than `t+delta+buffer`.

Then have an alarm for files older than `t+buffer`

Anyone who has tried to write formal verification will tell you that there is a WIDE gap between thinking and writing the specs.

Any tool that makes formal verification more accessible, should be welcome.

I believe the valuable part is how accessible we make thinking together with machines.

Us human are great at create innovative solutions, not so great at check and verify every single thing that can go wrong. Machines help with that.

This problem have been solved by LSP (language server protocol), all we need is a small server behind MCP that can communicate LSP information back to the LLM and get the LLM to use by adding to the prompt something like: "check your API usage with the LSP"

The unfortunate state of open source funding makes buildings such simple tool a loosing adventure unfortunately.

I wonder if it would make financial sense to offer the "skeleton" of the car for sale and let design studios integrate all the pieces.

Found funny that for something that is pretty much a commodity at this point, adoption seems to be the most important metrics.

Yes, there are differences between the models, and yes some may work better.

But picking the model at this point is just picking the cheapest option. For most use cases any model will do.

To be fair, beside technical documents, most of the link from Google are not worth reading.

It is mostly SEO spam, written by LLMs anyway. At that point I prefer the summary from Google.

I am not defending the model, but it is just the reality right now.

What are the stats of the new system?

Cost per giga?

Latency curve on read? Latency curve on write?

Support for concurrency?

Max throughput?

I think the article is somehow shallow, but S3 is a very deep and flexible system. Of course, someone will just need it's basic features, but someone else may be interested in advanced features as well.

Thanks!

I am looking explicitly for people who want to "run ffmpeg"

I don't have the skills, nor the capital, to build a solution for the whole market of people that want to mess with videos.

Just for reference Mix raised 177M since 2016.

The goal is not to run a startup, but a small business with recurrent business.

Of course there are a lot of way to approach this.

One would be an API sold at cost plus (which is the closest to my skillset.)

Building on top of that would be trivial for more pay-for-value product.

Thanks!

Why you think that an UI would make sense?

An UI like what? Something to drive the user toward what it wants to do?

The plan would it be to charge it at cost plus. Whatever the user is consuming plus a markup.

Are ffmpeg user interested in a cloud base solution?

You push the input files, the command, and fetch the output when done.

Would you, or anyone else, be interested in ffmpeg in the cloud?

Connect credit card, open a web UI, send the command, the files, and eventually get the output?

I see few blind spots from the write up.

1. Traffic for a new version was loaded up too quickly. I usually lobby for releasing updates slowly. This alone would have prevented the issue.

1. Tasks cannot fail under load. Load Shedding should be in place exactly for this reason. You don't take more than you can chew. If more arrives you slowly and politely refuse the request. You need to be both, slow and polite, so that the client will slowly retry and you won't incur in the herding issue.

1. The monitoring issue should have triggered (most likely) an increase of latency. That should have been enough to not complete the deployment and rollback carefully.

I am sure engineers in canva had their reason, and that the write up does not account for everything. Just some food for thought for other engineers.

In my experience, coding with AI is much more mentally taxing than coding without.

But it is much faster.

When I use AI I need to continuously review, direct, and manage the AI.

I go through every change and I agree with them, updates nits and regenerate code that is not up to par with a better or more specific prompt.

Not doing this exercise is disastrous for the codebase.

It really explodes in complexity in no time.

Moreover it always try to fix error with more code. Not with better code.

I wrote down the same concept as a more structured substack.

https://slowtechred.substack.com/publish/posts/detail/154138...

I am very biased on this topic.

I have started using AI coding assistant and I am not looking back.

This comes from an engineer that KEEP telling the junior on his team to NOT use GenAI.

The reality is that those tools are POWER TOOLS best used by engineers very well versed in the domain and in coding itself.

For them, it is really a huge time saving. The work is more like approving PR for a quite competent engineer than writing the PR myself.

My tool of choice is Cline, that is great, but not perfect.

And the quality is 100% correlated to:

1. The model

2. The context window

3. How well I prompt it.

In reverse order of importance.

Even an ok model, well prompted gives you a satisfactory code.