samsquire

I've been interested in this data structure for the ability to scan forward efficiently to any depth.

I am thinking of a binary protocol that can be navigated extremely efficiently and also be inserted into at any depth.

I enjoyed Namecheap before the redesign. It was a dense GUI and had small icons.

I like old style Windows 95 GUIs and "portlets"

I prefer dense GUIs. I know Japan has a different design aesthetic than the rest of the world. It's similar.

Thank you so much for your reply and thoughts.

Wow. Keep doing it. Well done.

I don't know how drivers are implemented but I assume they are async and state machines.

Do you have a shell?

From just thinking about the architecture of Rama I feel it would scale very well.

Data parallelism and partitioning and sharding is a very effective scaling technique.

Nathan, I would appreciate writings about the mental model of mapping software to your mental model of implementing behaviours in the streaming data approach because it is a different paradigm.

I can't read the DSL yet and know what is going on!

I think our programming languages haven't caught up to an ideal notation of semantics of computer system behaviour at a macro level.

The design patterns (Gang of Four etc) and software architectures that are so complicated they require extreme discipline to keep understandable. (And cognitive load)

Hi I would greatly enjoy talking to you.

I play with io_uring and multithreading. I am looking at event loops.

Thank for you for sharing.

What do you think are the next steps for a next generation event loop?

I've been experimenting with barriers/phasers, LMAX Disruptors and my own lock free algorithms.

I think some form of multithreaded structured concurrency with coroutines and io_uring.

I've been experimenting with decoupling the making sending and recv independently parallel with multiple io_urings "split parallel io" - so you can process incoming traffic separately from the stream that generates data to send. Generating sends is unblocked by receive parsing and vice versa.

Interested in seastar and reactors.

Is the industry (/academic) answer to distributed systems: you're building it by yourself?

There's no Postgresql or SQLite for distributed system or a template off the shelf standard design? (See LAMP or Linux/Docker/Postgres/Jenkins or rails or django)

I'm curious what makes it so fast?

The README.md says RegExpRouter uses no loops and just regular expressions, presumably it maps a route to a hashmap matched entry?

Related but slightly different:

I am curious because NFAs, deterministic automata are interesting to me. I remember one person on HN talked to me about using regular expressions to match event sequences. (That is: don't match against strings but match against tokens)

I am not a Rust developer, but the pattern of using Rust sum types and pattern matching or OCaml dispatch based on types are similar to me intuitively.

What are your thoughts on this style of programming?

I feel so much of modern business programming is dispatch logic!

If you think of Redux and state machines and event sequences in GUIs for behaviour, I feel there is a lot of untapped potential ideaspace here.

A mature production grade optimising JIT compiler codebase feels very it is weighty to understand for a newcomer in order to add value to the space because there's so much content to learn and be familiar with.

Godbolt for compiler internals and each compiler internal pass would be useful for me.

It is not an LLM comment.

I am interested in immediate mode GUIs but I think we need a different paradigm for GUIs that is less synchronous from a user perspective.

Encoding behaviour in GUI code is complex due to state management and complexity. See jQuery.

In most GUIs as a user you do an operation and wait for it to complete (direct manipulation). You can't queue operations unless you code. I haven't seen macro recording done well.

I follow the GUI development space out of curiosity. I have made a few GUIs in Java Swing toolkit and I enjoyed it.

When I did GUIs in Android native, I didn't enjoy it at all.

I have loosely played with Qt Quick but I didn't know C++ so I was clueless how to encode user behaviour into a program that implemented the behaviour. My immediate mode Javascript canvas work is far simpler.

LLMs are already generating web GUIs (such as vercel's v0 and others) and we can encode mapping of current state tokens to next behaviour tokens. Hopefully we can specify all the interactions and what should happen due to them.

React vdom refreshes the whole tree changed nodes for recomputation, LLMs could generate the new tree and use an animating library to transform between states - the difference in the trees.

Immediate mode GUIs lend themselves well to this; you just generate/rerender everything.

What I'm interested in is a novel desktop paradigm alternative to windows, mousing and drag and drop.

Intellisense and LSP lets us see the types of what an operation has. But computer GUIs are inherently synchronous unless you're writing a script or program. I would like to click through operations and transform and map from one kind of things to another type of things. This would queue operations up and the computer can schedule them efficiently.

Haskells knows the type of a pipeline at each stage of processing.

I would like to do "algebra with behaviour" and have built in transformations such as joins, swapping, queries and scheduling. If you can map a problem into something you understand based on position, you can do algebraic transformations between states such as binpacking or path finding.

"Break these files into batches, compress+encrypt them, keep them synchronized between these machines and back them up"

Thanks for sharing. Good work. It seems you've done a good amount of work on this, with the multiple backends.

From your other comments here it seems your emphasising "understandability by one person", which Oberon as you mentioned was designed to be understandable.

It reminds me of Taylor Troesh's wigwams

https://taylor.town/pardon-2023#wigwams

I need to document my JIT compiler's design which is really straightforward.

I've been loosely reading qbe's sourcecode but I need to go through the bibliography to understand the code more. At the moment it's all unfamiliar and not understandable.

Thank you for this post, very interesting.

Parallelism and async and multithreading is my hobby.

I think nodejs and browsers made typeless but coloured async easily digested and understood by everyone. Promises for example.

The libuv event loop used by nodejs is powerful for single threaded IO scalability, but I am looking for something that is multicore and scalable.

I like thread-per-core architectures.

The code written to an architecture has a high overhead in terms of mental understandability to new colleagues and barrier to transform it. Refactoring Postgres multiprocess implementation or nginx's worker model, it would be a major effort and undertaking. Firefox did a lot of work with Electrolysis Quantum to make it multiprocess.

I was yesterday and recently trying to implement a lockfree work queue that any thread can pick up work and "steal" work from other threads.

I have a multithreaded runtime I am building in C which is a "phaser", it does synchronization in bulk. So rather than mutexes, you have mailboxes which are queues.

Thank you for sharing your thoughts and ideas Sean, appreciated.

I am unfamiliar with the source material and I have recorded the paper you linked to my reading list.

From a description of "stream fusion", it reminds me of Clojure's "transducers".

It also reminds me of Kafka's Table/Stream duality.

Term rewriting is something is really interesting to me.

Communication, protocols and Communicating sequential processes, session types are all ideas I am thinking about and trying to understand.

Thanks for this interesting post.

Intuitively, relational algebra compresses enumeration over data in time that a CPU executing billions of cycles a second can feasibly and efficiently traverse and execute against many collections of millions or billions of records in human perceivable time thanks to indexes.

I've been trying to think of systems communicating with eachother as parts of a relational model in the sense we can model system behaviour as a series of events and a join is a communication between components.

I would love to talk about this with people.

This is such a major effort: getting up to speed on existing code bases to the point you can add to them.

I feel for the author of this comment, you want to help out and work on stuff you have energy at the beginning, I find it easier to write my own code than to get up to speed with someone else's code. Because you lose steam or the activation energy to get the project built and ran and then played with and customised/changed/"hacked on" is a major effort.

I have been thinking a lot on an idea inspired by compiler design: intermediate representations and term rewriting.

If software features were an arbitrary stack of crisscrossing intermediate representations that are rewritten and mutually recursive/referential or parsed or transpiled into actual code, then we could inspect the intermediate representations to work out how things work.

It would be nice to narrow down on a piece of behaviour and see how it works from end-to-end. But in practice, you have an opaque wholeness of a codebase to understand.

A modern system or codebase at a company or mature open source project: it's like those games of wooden sticks or wooden bricks jenga where they're arranged in a pile and you're piling things on-top of things and if you unsettle it slightly, it falls over or doesn't work.

I used a piece of software called OpenGrok which renders a large code base as clickable surfable wiki in the browser. So you can explore codebases.

I have a primitive python SQL database on my github that can execute simple graph cypher queries, simple joins of multiple tables, "dynamodb" style queries and document database queries.

Unfortunately it's unmaintainable and cannot be transformed without full time DevOps or SREs.

Large language models could be used to configure abstract syntax trees for desired behaviour, since much of devops and software architecture is slotting configuration together.

Kubernetes, linkerd, envoy, istio, cloudformation, terraform, cdk.

We're just configuring schedules and binpacking computation and data movement into a calendar.

The semantic space of understanding behaviour is different to understanding syntax.

What I find promising about LISP is the ability to do term rewriting and macros.

But people write lisps in imperative style rather than definitions of desired behaviour declaratively. I don't think we've sufficiently solved how to define desired behaviour to a computer.

Term rewriting behaviours. What are your thoughts?

I started trying to implement term rewriting into my LISP parser, which is the idea that we can match on trees and transform them, subsume branches or move branches around arbitrarily. You kind of want the matching function and transformation function to be imperative or sometimes like a query.

So use ASTs for behaviour and relationships and imperative LISP macros for transformations and rewriting.

The reason I say this is because I dream of a compositional language where I can create a cell in a spreadsheet and say that it should have these behaviours

  import load-balancing
  import batching
  import backoff
  import backpressure
  import retries
  import durable-execution
  import memory-contiguity
  import memory-alignment

Truly futuristic programming where we we program behaviours.

I am asked to define the terms that these behaviours require.

This is something I think a lot about.

I spend a lot of time trying to think of something that composes. Monads are one answer.

I think we need advanced term rewriting systems that also optimize and equivalise.

I really enjoy Joel on Software blog posts from this era.

Thanks for your lovely reply.

I don't have the code anymore might have been called "logtree", it was a HTML file and very small, it would parse dot "." separated lines such as (user.connected) and arrange them in a tree with indentation, if you clicked the dot syntax it would highlight in yellow other instances of that bucket of entries later in the log file.

I think you want elegant navigation between linear sequences (next, previous) of log lines that are apart in time, but related.

You might want to see all events of the internals of two different components in the tree structure.

Need to process log lines into a graph and then provide quick navigation for them.

It is indeed horizontal and vertical.

This would be very useful to anti-abuse measures if you could trace to the cause of something and cancel all thing caused by it.

Good idea. Thank you for sharing.

One thing I've always wanted was simple filters on log lines in a GUI that let me collapse tags in the log line. I created a proof of concept that showed logs as a tree.

Or I can say I want logs from the point where key=value and until method x.

A bit like a debugger but for logs. Show logs that are between these facts.

I like this thinking.

If someone could combine Supabase + Postgrest or Hasura with an easy visual admin dashboard for them inside this, this could be a complete easy to access, easy to deploy platform.

Thank you for graphviz.

I use graphviz dot syntax, parsing with networkx to plan expensive tool execution and the graph structure lets me automatically paralellize.

I actually really enjoy text UIs. I would like something similar to Vimperator or Pentadactyl to label keywords with numbers, perhaps in parentheses or square brackets so I can refer to things without typing them in. This is useful for identifiers which might be uuids, rather than type them in or use the mouse or tmux text selection to select and copy and paste, you can just refer to the numeric identifier.

The output for kubectl is text based output table, not a TUI. So it's a really important text format for user interaction.

I have had the idea of a text stream format that configures a terminal UI, so it could be used a bit like fzf and selections can be made with arrow keys.

In this design, your app would be decoupled from the UI framework (ncurses or Textual, your app is just printf), and you just output a special text format that configures collections which are mapped to portlets in the screen. You can use code similar to what tmux and screen does to arrange the portlets to different parts of the screen.

I really like the design of that website kulor.

It would be interesting if you could have a REPL in that style design, and you can reorder the panels of running processes, it reminds me of Yahoo, Lycos, Excite portals in the early web. I think they're also called portlets.

I enjoy a website called halfbakery

It's not wise to wonder why things aren't good anymore. The zeitgeist of what people value is different to what it was in the 90s.

I want to make and do good things so that's what I focus on writing about or programming.

Make good things and share them. If you don't value good then you get the fruit of it.

Thank you for you and for sharing your knowledge gpderetta, appreciated, TIL.