1,943 karma · joined April 3, 2012
It's why at Tessl we treat evals as a first-class part of the development process rather than an afterthought. Without some mechanism to verify quality beyond adoption, you end up with a very efficient way to spread confident nonsense at scale.
It's agents all the way down!
Submit a GitHub repo containing skills to Tessl, and it will generate the evals, run them, and present the results. https://tessl.io/registry/skills/submit
The evals and results are all shown, no login necessary, so you can assess them yourself. e.g. https://tessl.io/registry/skills/github/coreyhaines31/market... (click details to see the eval texts).
I submitted a package-lock.json file to the playground and got a vulnerability report after processing. The sort order next to the pie chart is weird. Medium / High / Critical / Low. I'd expect Critical / High / Medium / Low?
The vuln report ended up in my email spam folder.
I had to hit 'resend' multiple times to receive the verification email. Once I did, I had to either create a new account or login. I don't yet have a password. When I tried to create an account, it said my email was already taken. This onboarding flow seems quite janky.
Is Vulert Open Source software? I couldn't find any links or repos. What does "Join the Open-Source Security Movement" mean in this context?
Having your developer workstation break while you have a backlog full of stuff to do, would absolutely make you less motivated to run the developer release. Especially if you're not on the desktop team.
So, yeah. Okay.
(Speaking as ex-Canonical, and still Ubuntu user. I upgraded my ThinkPad 2 days before release, and it was a catastrophe I had to manually un-fudge with the help of the apt maintainer. It was a packaging problem).
My feeling on this particular release is that it was rushed out, and should probably have been kept back for a month or two. The xz and t64 (2038) issues occupied some unexpected time this cycle.
Also, there used to be a dedicated QA lab which did a whole slew of automated tests. I don't believe that still exists.
Also, also. The Ubuntu community has shrunk, which means fewer people doing QA.
Also, also, also. The guy running the desktop team left the day after the release. Read into that what you will.
I blogged about something related and included this anecdote part way through. https://popey.com/blog/2023/09/a-virus-for-the-bbc-micro/ - under "A short aside"
I still use it for everything. I don't have time or inclination to switch. However I have been somewhat convinced to take a look at Nix (packaging) for some of the tools I use. But all my existing systems are fine. So likely when I next get a work machine (next week) I'll probably (if allowed) use Nix to install anything developer related over and above the stock image and supplied packages.
Like the desktop app store, the Ubuntu installer is now written in Flutter.
https://web.archive.org/web/20060315081659/http://www.ex-par...
I appreciate your comments, as they made me think more about that topic.
They might consider it further if the store got to a decent scale (like the contemporaries like iOS, Play and Microsoft). But with "only" 6K applications published, and the money canon being pointed in other directions, I can't see it happening any time soon.
Or perhaps not, and they have a ton of other wallets full to the brim with crypto-nonsense.
I still have the snap, and could test further, but I suspect the endpoint linode boxes will disappear and popup somewhere else sometime.