scriptsmith

Lots of bloviating AI-prose. Here's the original report:

https://www.telstra.com.au/exchange/what-we-ve-learned-from-...

https://www.telstra.com.au/content/dam/tcom/dynamic-media-pr...

It's a reference to this story where an OpenAI model broke out of its sandbox during cyber benchmarking and hacked into HuggingFace, in order to obtain test solutions: https://news.ycombinator.com/item?id=48997548

I made these to create styled text art for the MOTD of my university's HPC login node:

https://github.com/ScriptSmith/tart

https://tart.sh/

Is GitHub Copilot the best positioned platform for enterprise? They support Claude, GPT, Gemini, and now even open weight models. Larger orgs are paying at API rates anyway so it costs just as much as anywhere else. They have a pretty good agent CLI and SDK, and now a desktop app. They have hosted agents, and you can run their 'Agentic Workflows' in CI.

Has their reputation tanked so much that the alternatives get all the buzz? Or is it that non-enterprise users are priced out by the usage costs, so no free marketing?

Definitely took longer than I was expecting, then after two weeks I thought we'd never get it.

And now is this going to be a one-off, or routine with every new generation of models?

It's based on a Gemma 3n model, and yeah it's not the best. But if you have a use case that needs constrained JSON output for example, it's pretty neat.

Maybe it would do better with the new Gemma 4 models, which the Chrome devs have been hinting at moving to. And why the API doesn't let you introspect / pick the model, I'm still not sure.

I've got some demos of what the new Prompt API in Chrome that uses a local model can do: https://adsm.dev/posts/prompt-api/#what-could-you-build-with...

As OP says, it shines in constrained environments where the model is transforming user-owned data. Definitely less useful for anything more open-ended.

Chromium mostly does not support this, because it doesn't have the binary blob required to run the inference. However, it does still download the model weights and expose the LanguageModel API, because that part is hooked up.

https://adsm.dev/posts/prompt-api/#which-browsers-support-th...

Packagers might eventually disable that but I tested this behaviour in chromium 148 a few hours ago, and it would download the weights but has trouble running them.

> My understanding is that you’ll have to explicitly agree to download alternate models in the future, per the specification

I don't get that from the spec. Closest thing I could find in the Writing and Assistance APIs spec [1] (which the Prompt API refers to) says:

> [It] allows the user agent to prompt the user for permission

But that's talking more about initiating the 'download' process more than choosing the model.

I think Chrome wants the browser to be 'opinionated' about the model used however, rather than letting the webpage pick, at least for now.

[1] https://webmachinelearning.github.io/writing-assistance-apis...

Depends on where you get it. By default the flags will be enabled, but some packagers may choose to disable them. I haven't seen a major distro release chromium 148 yet.

Weirdly though, chromium won't be able to actually use the model even though it can download it, because the inference engine is a closed-source blob.

https://adsm.dev/posts/prompt-api/#which-browsers-support-th...

I wrote a more detailed blog post here:

https://news.ycombinator.com/item?id=48028662

Chromium doesn't support this API because it needs a binary blob to run the inference, although in theory it may still be configured to download the weights:

https://adsm.dev/posts/prompt-api/#which-browsers-support-th...

It can only be called after the user has interacted with the page, but there's no dialogue from the browser

https://developer.chrome.com/docs/ai/get-started#user-activa...

Author here. After trying out the Prompt API over the last week, I wrote up some details on the chromium internals, how to use the API, and made some toy demos.

It's a 4 GB model that can be used to run on-device inference.

In my understanding, yes. I wrote a blog post about some of the internals here: https://news.ycombinator.com/item?id=48028662

Google has been trialling the Prompt API in chrome for the over a year, so before Gemma 4 existed. But they are indicating they'll move to Gemma 4: https://groups.google.com/a/chromium.org/g/blink-dev/c/iR6R7...

It's based on Gemma 3n, and it's not the best.

I find it works fine for simple classification, translation, interpretation of images & audio. It can write longer prose, but it's pretty bad.

It can also write text in the format of a JSON schema or regexp for anything you might want to do with structured data.

Those flags will exist already, but will default to enabled in 148.

That other flag is for using a different open-source inference engine to the (from what I can tell) closed-source one that's used by default.

If Chrome has the #optimization-guide-on-device-model and #prompt-api-for-gemini-nano flags enabled, either because it's part of some Origin Trial / Early Stable Release or something, then web pages will have access to the new Prompt API which allows any webpage to initiate the (one-time) download of the ~2.7 GiB CPU or ~4.0 GiB GPU model using LanguageModel.create()

https://developer.chrome.com/docs/ai/prompt-api

When Chrome 148 releases tomorrow, this will be the default behaviour on desktop.

To download, it should check for 22 GiB free disk space on the volume where your Chrome data dir is, and at least double the model size of free space in your tmp dir.

Yes, you could turn it around to say that using Anthropic models in Cursor, Copilot, Junie, etc. is 'subsidising' Claude Code users.

The "First-class syntactic selection" reminds me of my most used shortcut(s) in Jetbrains IDEs: the Expand / Shrink Selection.

  Ctrl + W
  Ctrl + Shift + W

https://www.jetbrains.com/help/idea/working-with-source-code...

It really changed my perspective on interacting with the 'text' of a file.

VS Code, Zed, etc. have similar operations, but in my experience they expand and shrink too coarsely.

Yes, I've used the v3.2 3B-Instruct model in a Slack app. Specifically using vLLM, with a template: https://github.com/vllm-project/vllm/blob/main/examples/tool...

Works as expected if you provide a few system prompts with context.

To keep on-top of tabs in Firefox, I use 'Auto Tab Discard' [1] to discard tabs after a certain amount of inactivity. Then when I need to clean up my list of tabs, I click on any discarded tabs I want to keep, and then use my extension 'Close Discarded Tabs' [2] to clear the rest.

[1] https://addons.mozilla.org/en-US/firefox/addon/auto-tab-disc...

[2] https://addons.mozilla.org/en-US/firefox/addon/close-discard...

Someone posted this in the thread, but I also can't log in to verify

> Summary

> CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon Sensor.

> Details

> Symptoms include hosts experiencing a bugcheck\blue screen error related to the Falcon Sensor.

> Current Action

> Our Engineering teams are actively working to resolve this issue and there is no need to open a support ticket.

> Status updates will be posted below as we have more information to share, including when the issue is resolved.

> Latest Updates

> 2024-07-19 05:30 AM UTC | Tech Alert Published.

> Support

> Find answers and contact Support with our Support Portal

It's crowdstrike: https://www.reddit.com/r/crowdstrike/comments/1e6vmkf/bsod_e...

> 7/18/24 10:20PT - Hello everyone - We have widespread reports of BSODs on windows hosts, occurring on multiple sensor versions. Investigating cause. TA will be published shortly. Pinned thread.

> SCOPE: EU-1, US-1, US-2 and US-GOV-1

> Edit 10:36PT - TA posted: https://supportportal.crowdstrike.com/s/article/Tech-Alert-W...

> Edit 11:27 PM PT:

> Workaround Steps:

> Boot Windows into Safe Mode or the Windows Recovery Environment

> Navigate to the C:\Windows\System32\drivers\CrowdStrike directory

> Locate the file matching “C-00000291*.sys”, and delete it.

> Boot the host normally.

Seems like he actually disagrees here:

If you train a bigger model on more text, we have a lot of confidence that the next-word prediction task will improve. So algorithmic progress is not necessary, it's a very nice bonus, but we can sort of get more powerful models for free, because we can just get a bigger computer, which we can say with some confidence we're going to get, and just train a bigger model for longer, and we are very confident we are going to get a better result.

https://youtu.be/zjkBMFhNj_g?t=1543 (23:43)

How are people using these local code models? I would much prefer using these in-context in an editor, but most of them seem to be deployed just in an instruction context. There's a lot of value to not having to context switch, or have a conversation.

I see the GitHub copilot extensions gets a new release one every few days, so is it just that the way they're integrated is more complicated so not worth the effort?

Is there some trick to doing validation of request data using this process? That's a valuable part of using something like tRPC, JSON Schema + type generation, zod, etc.

Don't forget Right-To-Left languages, that also affects how UI elements are arranged (position within the page) and rendered (input widgets like sliders get reversed).