jasim

If tokens are going to get cheaper and faster - then we would stop caring about token efficiency, like how we're all using bloated and slow software, and which keep growing to take up all available hardware improvements.

ERPNext (Frappe) is absolutely brilliant. My only complaint was that it is in Python and is a bit heavy weight (setup etc) for quick prototyping.

They call a table definition - that defines a table at both the SQL level, and at the domain level - a DocType (see: https://docs.frappe.io/framework/user/en/basics/doctypes). When you declare a DocType you immediately get a working table, foreign keys transformed into combo boxes for lookups, forms, and a whole set of conveniences that are common to ERP-like applications.

Before seeing ERPNext I would've thought that such a declarative approach would fail to satisfy the critical remaining 20% of most requirements, but they've built around it in a way that has enabled them to ship a complex ERP system that thousands of businesses of different shapes use. I think it is one of those gems that are not in popular conscience among software engineers, compared to regular frameworks (but very much so in business-ERP adjacent circles), and is worth learning from.

It is so hard to predict the future, of all things. But I think there is another approach which is already tried and tested - frameworks.

I think frameworks would mutate to be agentic - they'll come with skills and documentation targeted to LLM agents, and they'll have far more breadth than anything we've seen, a certain lack of restraint, and will sacrifice composability and orthogonality because they are no longer useful for agents.

The framework I'm building - plug: https://sapporta.com - brings in a set of UI choices (datagrids for every table) and a set of API choices (everything exposed to agents) which is not typical to classical frameworks. If you're building database applications, it contains most of the things that a typical NoCode tool would have, but with complete code and the ability for agents to build upon it.

I think Sapporta is one of the first frameworks with this point of view, but it has only been a few months since Opus 4.8, and I think in the coming months we'll see a lot more vertically integrated horizontal frameworks (the solid base) that can get us malleable software with custom code.

I think this is somewhere in the middle east - Dubai likely - and it is hot there. It is quite predictable that there would be mass opposition with institutional support against turning off air conditioning in a common space, despite having a couple of hours of exclusive rental. That said, it is a good reminder that we have a choice in what we are willing to accept, and can fall back on a BATNA, even if it is not the most ideal outcome.

I've been paying JetBrains in a personal capacity for about 15 years. Almost from the first year of my professional career. I didn't renew this year. I feel so sad and disheartened that a company that produced such brilliant tools have completely lost the plot on AI. They had started work on Fleet, a lightweight VSCode like IDE, before LLMs became big, but abandoned the effort without seeing it through. It could've been a great agentic IDE (all I want is whole file+project view for every conversation, instead of just diff so I can have a better perspective of the codebase). Instead they built multiple confusing tools (might be internal org structure and competition leaking out), and took far too long to adopt SOTA CLI coding agents, and kept charging for it as a separate subscription. I wish they do a whole company rethink on agentic development, but it is going to be tough because the existing enterprise licenses will continue to be the main revenue driver for a few more years, until it all dries up all at once.

We might be moving towards putting more intent - in text - into the code itself, for agents. We should've been doing that for other humans in the past, but it wasn't always encouraged - "code should be self-explanatory, and comments are a sign of poor naming and abstraction", and all that.

These days I run a prompt of this shape to ensure that my code contains as much tacit knowledge as I have -- "I want to make sure all the implicit invariants and structures and concepts described in these plans, are present in the code comments, namings, comments around major types and modules. So even if the original plan file is not present, we should still be able to understand it (Peter Naur - Programming is Theory Building)"

However - I still need the type system as another preserver of invariants and intents. Coding agents still make subtle mistakes and miss very obvious things as the code gets more intricate and conceptually large.

A type error where an expected case is not handled - that is a feedback that can ground it back and force it to think through whether windspeed matters to is_dangerous or not. And the choice of is_dangerous in the article is a bit misleading, because anyone can at first read make the connection from windspeed to is_dangerous. It is a pretty bold variable name and attention grabbing situation.

However codebases are made of far more mundane but widespread and braided sets of tiny invariants that wouldn't catch either a human's or a coding agent's attention without needing a more firmer hand, and so I still do love my type systems despite my best laid comments.

But couple that with a deterministic system - the type checker with exhaustive destructing (and pattern matching, while we are it it) - that would become a fool-proof combination.

I hadn't considered that, you are right. Though I would still prefer TS because the language is all about types; the entire ecosystem is well-typed and the type system is quite powerful (I do enjoy an occasional Omit and Pick). But it is a personal choice; as long as LLMs can generate well-typed Python, then for people who like the language it makes sense. However from the article, I got the sense that they went completely dynamic.

I'm curious about the choice of Python, rather than TypeScript.

I find Ruby a very beautiful language, and Rails is an excellent web framework, but I need typed functions, record types and sum types.

They help not just with correctness, but also as living documentation that lets me understand AI generated code. TypeScript provides discriminated union, but not exhaustive pattern-matching, and its syntax is a bit verbose, but since I'm no longer writing most of the code myself, I can live with it.

However I can't imagine using Python or any other dynamic language going forward. There is likely good reason for you to choose it, and I'm curious to know what that is.

There is no other stack in the last 15 years or so that I look back with the same fond affection as building business applications in Clipper. One important aspect was that software was delivered physically, and I could watch my users using them. If I made a report that helped the operator, or made a workflow easier, their happiness was immediate and nurturing. And there was an immediacy to programming in xBase - if we're building a database backed business application, which was their strong suit, there was very little standing in between us and the problem space.

"There was no exploit. No vulnerability disclosure. No CVE for me to write. The attacker filled out my signup form 942 times, made 942 workspaces, sent 942 batches of about a hundred invitations each, and stopped. They used my tool exactly as designed. The design was just bad enough that the tool was good for phishing."

In my early programming days, working with Clipper, I used to look at Delphi from a distance with awe and a bit of jealousy. There also used to be PowerBuilder and Paradox, as competition to the xBase platforms.

I'd love to hear more about how you're using Delphi and what it excels at, compared to current web and native software stacks.

Everytime someone mentions Clipper (or dBase or FoxPro, or even FoxBase, but Clipper the most), I feel a sene of productive nostalgia, and a constructive anger at the state of technology today. xBase was a beautiful thing - I haven't had as much fun at building software that I've had from the first plink86 till CA-Clipper 5.3's blinker and exospace. Even prolific use of Opus 4.6 doesn't bring the sense of quality and satisfaction that those systems produced.

I'm building a new database tool for the web, a frankenstein of Lotus 1-2-3, dBase, MS-Access, and Claude Code. It is where that anger goes these days.

I'm not sure if this is what you're looking for, but there is a proposal in DataFusion to allow user defined types. https://github.com/apache/datafusion/issues/12644

Parquet files include a field called key_value_metadata in the FileMetadata structure; it sits in the footer of the file. See: https://github.com/apache/parquet-format/blob/master/src/mai...

The technique described in the article, seems to use this key-value pair to store pointers to the additional metadata (in this case a distinct index) embedded in the file. Note that we can embed arbitrary binary data in the Parquet file between each data page. This is perfectly valid since all Parquet readers rely on the exact offsets to the data pages specified in the footer.

This means that DataFusion does not need to specify how the metadata is interpreted. It is already well specified as part of the Parquet file format itself. DataFusion is an independent project -- it is a query execution engine for OLAP / columnar data, which can take in SQL statements, build query plan, optimize them, and execute. It is an embeddable runtime with numerous ways to extend it by the host program. Parquet is a file format supported by DataFusion because it is one of the most popular ways of storing data in a columnar way in object storages like S3.

Note that the readers of Parquet need to be aware of any metadata to exploit it. But if not, nothing changes - as long as we're embedding only supplementary information like indices or bloom filters, a reader can still continue working with the columnar data in Parquet as it used to; it is just that it won't be able to take advantage of the additional metadata.

I think this post is a response to some new file format initiatives, based on the criticism that the Parquet file format is showing its age.

One of the arguments is that there is no standardized way to extend Parquet with new kinds of metadata (like statistical summaries, HyperLogLog etc.)

This post was written by the DataFusion folks, who have shown a clever way to do this without breaking backward compatibility with existing readers.

They have inserted arbitrary data between footer and data pages, which other readers will ignore. But query engines like DataFusion can exploit it. They embed a new index to the .parquet file, and use that to improve query performance.

In this specific instance, they add an index with all the distinct values of a column. Then they extend the DataFusion query engine to exploit that so that queries like `WHERE nation = 'Singapore'` can use that index to figure out whether the value exists in that .parquet file without having to scan the data pages (which is already optimized because there is a min-max filter to avoid scanning the entire dataset).

Also in general this is a really good deep dive into columnar data storage.

Well put. It should always be "Created by <person>" rather than "Processed by LLM". We can already see it with Claude Code - its commit messages contain a "Generated by Claude Code" line, and it guarantees a pandemic of diffused responsibility in software engineering. But I think there is no point in railing against it - market forces, corporate incentives, and tragedy of the commons all together make it an inevitability.

I'm sorry I don't follow. The fact that you use an LLM to classify a transaction does not mean there is no audit trail for the fact. There should also be a manual verifier who's ultimately responsible for the entries, so that we do not abdicate responsibility to black boxes.

Accounting, specifically book-keeping, really plays to the strengths of LLMs - pattern matching within a bounded context.

The primary task in book-keeping is to classify transactions (from expense vouchers, bank transactions, sales and purchase invoices and so on) and slot them into the Chart of Accounts of the business.

LLMs can already do this well without any domain/business specific context. For example - a fuel entry is so obvious that they can match it into a similar sounding account in the CoA.

And for others where human discretion is required, we can add a line of instruction in the prompt, and that classification is permanently encoded. A large chunk of these kind of entries are repetitive in nature, and so each such custom instruction is a long-term automation.

You might have not been speaking about simple book-keeping. If so, I'm curious to learn.

I can think of two instances, where the LLM embracing best practices for human thought leads to better results.

Claude Code breaks down large implementations to simpler TODOs, and produces far better code than single-shot prompts. There is something about problem decomposition that works well no matter whether it is in mathematics, LLMs, or software engineers.

The decomposition also shows a split between planning and execution. Doing them separately somehow provides the LLM more cognitive space to think.

Another example is CHASE-SQL. This is one of the top approaches in Text-to-SQL benchmark in bird-bench. They take a human textual data requirement, and instead of directly asking the LLM to generate a SQL query, they run it through multiple passes: generating portions of the requirement as pseudo-SQL fragments using independent LLM calls, combining them, then using a separate ranking agent to find the best one. Additional agents like a fixer to fix invalid SQL are also used.

What could've been done with a single direct LLM query is instead broken down into multiple stages. What was implicit (find the best query) is made explicit. And from how well it performs, it is clear that articulating fuzzy thoughts and requirements into explicit smaller clearer steps works as well for LLMs as it does for humans.

With negative/positive, the invariant would be sum(amount) = 0; with your approach, it would be sum(debit-credit)=0. Both are valid, it is just two ways of expressing the same thing.

I think it is useful to think about double-entry book-keeping in two layers. One is the base primitive of the journal - where each transaction has a set of debits and credits to different accounts, which all total to 0.

Then above that there is the chart of accounts, and how real-world transactions are modelled. For an engineer, to build the base primitive, we only need a simple schema for accounts and transactions. You can use either amount (+/-ve), or debit/credit for each line item.

Then if you're building the application layer which creates entries, like your top-up example, then you also need to know how to _structure_ those entries. If you have a transfer between two customer accounts, then you debit the one who's receiving the money (because assets are marked on the debit side) and credit the other (because liabilities are on the credit side). If you receive payment, then cash is debited (due to assets), and the income account is credited (because income balances are on the credit side).

However, all of this has nothing to do with how we structure the fundamental primitive of the journalling system. It is just a list of accounts, and then a list of transactions, where each transaction has a set of accounts that get either debited/credit, with the sum of the entire transaction coming to 0. That's it -- that constraint is all there is to double-entry book-keeping from a schema point.

> Instead of using negative numbers, Accounts have normal balance: normal credit balance literally means that they are normal when its associated entries with type credit have a total amount that outweighs its associated entries with type debit. The reverse is true for normal debit balance.

But that is an interpretation made by the viewer. A customer typically is an asset account, whose balances are in the debit column. But if we somehow owe them money because let's say they paid us an advance, then their balance should be in the credit column. The accounting system need not bother with what the "right" place for each account is.

It is quite practical to have only a simple amount column rather than separate debit/credit columns in a database for journal entries. As long as we follow a consistent pattern in mapping user input (debit = positive, credit = negative) into the underlying tables, and the same when rendering accounting statements back, it would remain consistent and correct.

That part of the article felt quite wrong to me as well. I've built accounting systems that worked well for a decade, where internally the values were a single amount column in the journal table. If it was a debit, it'd be positive, if a credit, it'd be negative.

In fact, we could call these values yin and yang, for all it mattered.

Also, I'm not able to really follow what he means by "money = assets in the future".

Money is money, but if you wanted to track the intermediate state until the customer gets receipt, you would use an In Transit account (Good In Transit / Service In Transit etc.)

Yet, it doesn't change the fundamental definition of the value in the accounting system. I think the author confuses an engineering concept (sagas, or thunks, or delayed but introspectable/cancellable actions in general) with accounting.

Thanks, that makes sense. I avoid using LLMs for math because it is only a text token prediction system (but a magical one at that), and can't do true numeric computation. But making it write code to compute works well.

I'm curious to hear more about this. I've seen very little hallucination with mainstream LLMs where the conversation revolves around concepts that were well-represented in the training data. Most educational topics thus have been quite solid. Even asking for novel analogies between distant and unrelated topics seem to work well.

Good question. LLMs are surprisingly less fragile than hand-coded parsers for unstructured data like the ones in a bank statement.

And to be clear - I'm not sending the entire statement to Claude; instead, only the account name/narration of those transactions for which I already don't have a mapping. Claude then returns a well-formatted JSON that maps "amzn0026765260@apl" to "expenses:amazon", and "Veena Fuels" to "expenses:vehicle" and so on.

I can also pass in general instructions saying that "Restaurants and food-related accounts are categorized under 'expenses:food'", and it does a good job of mapping most of my dining out expenses to the correct account head.

The actual generation of journal entries are done by a simple Python script. The mapping used to be the hardest part, and what used to need custom classification models is just a simple prompt with LLM.

Plain Text Accounting has become significantly easier to do for me on a regular basis, thanks to LLMs. Specifically: importing bank statements into hledger and avoiding manual entry.

I use a JSON file to map bank entries to my hledger accounts. For new transactions without mappings, I run a Python script that generates a prompt for Claude. It lists my hledger accounts and asks for mappings for the new entries.

Claude returns hledger journal entries based on these mappings, which I can quickly review.

Then another script prints out hledger journal entries for that month's bank transactions, all cleanly mapped. It takes me just a few minutes to tweak and finalize.

I can also specify these mapping instructions in plain-language which would've otherwise been a fragile hodgepodge of regexps and conditionals.

It is hard to believe that a company with the expertise of Adobe in graphics programming (Photoshop, Premier, After Effects, Illustrator) cannot build a vector design program.

Adobe XD was almost there, and they made some great decisions in nooks and crannies usually forgotten - like bringing in Chrome rendering engineers to build a subset of HTML and CSS for the plugin ecosystem.

While there are hundreds of little decisions to be made - from the runtime, text rendering, gpu vs cpu rendering, to how shadows are rendered and line heights are determined - none of these are engineering problems beyond the ken of a good team of systems and graphics engineers and designers, the likes of which Adobe has in spades.

The attempt to acquire Figma for such an enormous sum itself felt like a serious decision making mistake, and the final nail in the coffin is the complete abandonment of vector design tools.

I would pay good money to read the insider account of the corporate politics inside Adobe that led to all this.

`key` informs React of the identity of an element. That helps it during the reconciliation phase -- if it knows only one `key` in a list of DOM elements has changed, then it will run the DOM updates only on that one. Similarly if the order has changed, it only needs to move its index in the parent DOM element.

But it doesn't help in the rendering phase - aka when the virtual DOM is constructed when `render` is called on the root component, and all our JSX code is executed across the component hierarchy. Virtual DOM reconciliation cost is only a part of the performance penalty, re-running the "view as a function of state" computation is another major chunk.

Thank you!