Hacker News Reader

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

The reMarkable comes with root-ssh out of the box, so installation here is scp'ing a rust-built binary over, and then ssh'ing and running it. I haven't wrapped it in a startup-on-boot service yet.

It is triggered right now by finger-tapping in the upper-right corner, so you can ask it to respond to the current contents of the screen on-demand. I think it would be cool to have another out-of-band communication, like voice, but this device has no microphone.

Also right now it is one-shot, but on my long long TODO list is a second trigger that would _continue_ a back and forth multi-screenshot (like multi-page even) conversation.

awwaiidon Vi Hart Videos on the Internet Archive

Having watched the remaining video, maybe this is related/protest to the recent statements and policies of the US federal government declaring gender+sex as strict booleans and actively harassing and destroying any institutional existence that acknowledges or embraces any alternative to the strict model?

Type checker gone extreme-strict and having issues compiling in the face of real world and real humans.

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

The trick here is to inject events as if they came from the user. The virtual-keyboard works really reliably, you can see it over at https://github.com/awwaiid/ghostwriter/blob/main/src/keyboar... . It is the equivalent of plugging in the reMarkable type-folio.

Main limitation is that the reMarkable drawing app is very very minimal, it doesn't let you place text in arbitrary screen locations and is instead sort of a weird overlay text area spanning the entire screen.

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

This uses LLM Tools to pick between outputting an SVG or plugging in a virtual keyboard to type. The keyboard is much more reliable, and that's what you see in the screenshot.

If nothing else it could use an SVG font that has handwriting; you'd need to bundle that for rendering via reSVG or use some other technique.

But if I ever make a pen-backend to reSVG then it would be even cooler, you would be able to see it trace out the letters.

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

Buy me one and I'll find out! hahahaha

But also -- the main thing that might be different is the screenshot algorithm. I'm over on the reMarkable discord; if you want to take up a bit of Rust and give it a go then I'd be happy to (slowly/async) help!

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

The limitations for the reMarkable made it so that I took a screenshot and then injected input events to interact with the proprietary drawing app. Cross-app screenshots with the right permission are probably possible on Android, I'm not sure about injecting the drawing events.

The other way to go would be to make a specific app. I just picked up an Apple Pencil and am thinking of porting the concepts to a web app which so far works surprisingly well ... but for a real solution it'd be better for this Agent to interact with existing apps.

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

Generally if I can read my handwriting then it can! It has no issues with that. Really the problem is more in spacial awareness -- it can't reliably draw an X in a box, let alone play tic-tac-toe or dots-and-boxes.

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

Thank you! Still a WIP, but a very fun learning / inspiration project. Got a bit of Rust jammed in there, bit of device constraint dancing, bit of multiple LLM api normalization, bit of spacial vision LLM education, etc.

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

Let me know if you need any help, I think only one other person has tried to get this working. I'm over on the reMarkable discord server, https://discord.gg/u3P9sDW (linked from https://github.com/reHackable/awesome-reMarkable)

Rust binary so should be easy to install. In theory :)

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

Project author here -- happy to elaborate on anything; a continuous WIP project. The biggest insight has been limitations of vision models in spacial awareness -- see https://github.com/awwaiid/ghostwriter/blob/main/evaluation_... for some sketchy examples of my rudimentary eval.

Next top things:

* Continue to build/extract into a yaml+shellscript agentic framework/tool

* Continue exploring pre-segmenting or other methods of spacial awareness

* Write a reSvg backend that sends actual pen-strokes instead of lots of dots

awwaiidon Ghostwriter – use the reMarkable2 as an interface to vision-LLMs

Two reasons. One, because I haven't gotten to it yet. Two... er no just the one reason! Do you have a particular one, ideally with a hosted API, that you recommend?

awwaiidon Llama.vim – Local LLM-assisted text completion

The blinking cursor in demo videos is giving me heart palpitations! But this is super cool. It makes me wonder how Linux is doing on M* hardware.

awwaiidon Backdooring Your Backdoors – Another $20 Domain, More Governments

All property, physical and digital, is rented if you squint just right.

awwaiidon What Is miniKanren?

Not sure if you heard, but recently more net neutrality rules were struck down and that is expected to continue -- so ISPs (who have already done this) are allowed to inject / replace ads with their own ads, as an example. (https://thenextweb.com/news/comcast-continues-to-inject-its-... was the first article I hit in the googles)

awwaiidon I still don't think companies serve you ads based on your microphone

The people I know who believe this myth go into their settings and turn off microphone access on an app-by-app basis.

awwaiidon Fogus: Things and Stuff of 2024

They update on that reddit makes it sound resolved?

I heard boox was very not gpl comiant too

awwaiidon Nvidia Jetson Orin Nano Super [video]

Any guesses on using this for LLM inference how it compares performance-wise (like tokens/sec ultimately) to like the new mac mini m4pro? I'm not sure how to figure that out.

awwaiidon Λ-2D: An Exploration of Drawing as Programming Language

Talk about burying the lede!

awwaiidon Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

The llm CLI is much more unixy, letting you pipe data in and out easily. It can use hosted and local models, including ollama.

awwaiidon Vizdom: Diagrams as Code

Cool, thanks - it makes sense to build on or be (directly) inspired by these other projects.

awwaiidon Vizdom: Diagrams as Code

Doing a layout algorithm is very difficult, and the DOT code made me curious, so I decompiled the wasm and looked around a bit. I see this appears to use or build on Dagre (https://github.com/dagrejs/dagre/wiki), though I've seen no citation of that work.

What's up with that? How much of this is original work vs pulling together tools in a new way or improving their API?

awwaiidon Elasticsearch is open source, again

... Yes, they submitted labor backed security fixes as their form of rev share.

awwaiidon Show HN: Remove-bg – open-source remove background using WebGPU

Sure!

awwaiidon Hermes 3: The First Fine-Tuned Llama 3.1 405B Model

Last summer was like a million years ago! You should try again :)

awwaiidon Mermaid Gantt diagrams for displaying distributed traces in Markdown (2023)

At that point you could use PlantUML (which has nicer syntax, but worse theme defaults)

awwaiidon I am starting an AI+Education company

I was going to say beware because there isn't an Anthropic Claude official app.... but I checked and I guess as of today there is one hah. https://www.anthropic.com/news/android-app

awwaiidon I am starting an AI+Education company

"Our first product will be the world's obviously best AI course, LLM101n."

Well obviously the target audience is actually AI-agents as part of the Super Intelligence ramp-up!