HNHacker News
TopNewBestAskShowJobs

Chris_Newton

2,790 karma · joined February 17, 2010

I make software and web things, I run small businesses, and I live near Cambridge, UK.

If you’d like to discuss anything I’ve talked about on HN with me, you’re welcome to get in touch. I’m hn at firstname dash surname dot co dot uk.

If you’re interested in engaging me professionally or have a question about one of my businesses, genuine enquiries are also welcome at the same address.

submissionscomments
Chris_Newton··on Ask HN: Freelancer? Seeking freelancer? (July 2024)
SEEKING WORK | Cambridge, UK | Remote preferred (occasional travel OK)

Offering hands-on software development and technical leadership services, I specialise in:

• Mathematical, scientific and engineering applications

• Custom user interfaces and data visualisation

• Software for hardware (IoT, lab equipment, embedded)

I have over 20 years’ experience developing software, mostly working with smaller organisations. If your situation looks anything like these, I could probably help you:

• A scale-up struggling to overcome early tech debt and transition to more sustainable development

• A hardware start-up looking to outsource software development to a multi-skilled freelancer

• A software start-up that can’t hire permanent senior+ level developers fast enough to hit targets

• A research group needing a professional to implement models and visualisations in good code

Every project is different and will be treated as such, but recently I’ve been favouring TypeScript or Python for web development (with React or Vue for UIs) and Python, C++ or Haskell for more mathematical or hardware-related work. I have experience with a wide range of target platforms from AWS to embedded ARM.

If you have a project that I may be able to help you with, you’re welcome to get in touch. My contact details are in my profile.

Chris_Newton··on A bunch of programming advice I'd give to myself 15 years ago
This is a disadvantage of the more dynamic languages, I find. IDEs for languages like C++ and Java are relatively fast and accurate when jumping to definitions, showing references, performing automated refactorings and so on. I rarely see that same precision for any editor or IDE working with TypeScript, Python, and other very dynamic languages, even with the improvements recently thanks to LSP and generally better tools. When almost anything could theoretically have been patched at runtime — even if doing so would be a terrible idea and every style guide says don’t do it — it’s not safe for tools to make the same kind of assumptions with the more dynamic languages that they can with the more static ones.
Chris_Newton··on A bunch of programming advice I'd give to myself 15 years ago
I’m a big fan of writing things down for future reference. Keeping a journal with things like exactly which commands I used, or transferring that knowledge to something like a README or a wiki page for others to see, is a low effort but sometimes high reward habit. Even for something done only rarely, saving a lot of time next time can justify the documentation effort.

Similarly, if the automation effort is literally only copying those commands verbatim into a shell script or similar, that’s often time well spent. The trap, as the XKCD helpfully demonstrates, is when you then start spending significantly longer on making that script more “flexible”. Turning hard coded values into parameters. Looking context up from the environment. On multiple development platforms. In staging, UAT and production deployments too. With detailed help text to explain it all and say what the defaults for everything are if you don’t specify them explicitly. And then it turns out that you only actually run that script once every six months anyway and never use 95% of that extra flexibility anyway…

Chris_Newton··on A bunch of programming advice I'd give to myself 15 years ago
In Sublime, there is a command to paste while matching the indentation as well. I think it’s Ctrl+Shift+V on most platforms by default. Maybe that’s what you’re looking for?
Chris_Newton··on A bunch of programming advice I'd give to myself 15 years ago
Possibly my all-time favourite XKCD, Is it worth the time?¹, demonstrates two important points. If you only do something very rarely anyway, spending time to automate it won’t have a great ROI. But for things you do moderately often that take a minute or even just a few seconds, you can afford to spend a surprisingly large amount of time optimising them and still get a big pay-off over a time frame measured in years.

I view time spent learning to use my tools efficiently and automating common tasks as a sound investment. Editors are a great example. Sure, I could fire up any of the usual suspects and write code somewhat productively. But in my fully customised editor of choice, I can insert and adapt common code patterns that would take me 10–30 seconds to type out fully in mere moments using templates and macros. I can jump to any position visible on-screen with around three keystrokes. I can see syntax highlighting for various file types that I use all the time, including some unusual ones that don’t have definitions readily available, and warnings for several different programming languages in real time as I work.

These save me a few seconds every time I use them, but how many times is that every day? The effort to set them up has probably repaid itself 100x over by now! And the lack of latency is also a qualitative improvement since it means once I’m ready to start writing, I can do so roughly as fast as I can think, instead of constantly being held back and interrupted for a moment.

¹ https://xkcd.com/1205/

Chris_Newton··on The Functional Programming Hiring Problem
This hasn’t been my experience. One attraction of functional programming style is that you tend to separate pure/declarative code from effects, one way or another, and then express each using familiar patterns of computation or interaction that are the essence of what you’re implementing. This often results in simpler and more explicit code, with exactly the kind of high cohesion and low coupling you’re advocating.

It is no coincidence that an expressive functional language like Haskell has concise, descriptive names for numerous common folding patterns over common data structures such as mapAccumWithKey¹ that let you express what you want to achieve in just a line or two of code with minimal boilerplate or ambiguity, where using a generic loop in a mainstream imperative language would take 3x that and need a little thought to recognise the pattern being used and make sure there wasn’t anything else happening that was tangled up in the same code.

¹ https://hackage.haskell.org/package/containers-0.7/docs/Data...

Chris_Newton··on The Functional Programming Hiring Problem
Indeed. Category 3 seems to be about developers who insist on using Gooby regardless of whether or not it’s a reasonable choice for solving the current problem. What about developers who simply have Gooby as one tool in their toolbox, know how to use it, and enjoy doing so when an appropriate opportunity arises?

I once wrote some Haskell professionally. It was a successor to a previous generation of the application that I had written in Python and maintained for several years. Using Haskell was my choice and it was made because some of Haskell’s strengths would address specific pain points learned from that experience. Meanwhile, everything else I wrote for that client was still being done in mainstream languages for mainstream reasons and there was never any suggestion that we would rewrite anything else in Haskell or adopt it by default for any new work, so I think this situation is clearly distinct from category 3 in the article.

Chris_Newton··on AI Appears to Rapidly Be Approaching Brick Wall Where It Can't Get Smarter
There can be all kinds of strange and possibly perverse incentives at work for the “original content producers”, too.

Forums like LinkedIn have had regular listings in recent times looking for programmers to help train new models for answering programming questions and generating code. However, the rates offered are nowhere near enough to compete with real programming jobs for skilled developers, at least not in the major Western economies.

One wonders what quality level these new models will actually be trained at. Given that these roles are kinda-sorta asking professional programmers to train themselves out of a job, one also wonders whether 100% of those who do contribute will be doing so in good faith. And as you point out yourself, there is also a risk that the “original” content is anything but.

Given the scale of the data involved here, and the sheer amount of skilled human input that would be required to refine that data set manually, it seems unlikely that a Mechanical Turk strategy alone will solve many fundamental problems here.

Chris_Newton··on AI Appears to Rapidly Be Approaching Brick Wall Where It Can't Get Smarter
Ideally a LLM would have every text in existence in its training set, right?

Only if every text in existence provides good source material to factor into generating the response. Given that half of all texts are even worse than average, this seems unlikely, and Karpathy’s argument seems very reasonable to me.

The final paragraph from the original tweet, after the quote in the GP comment, mentions another interesting aspect. Even if we do have sufficient expert-level source material in a particular field to train a useful generative AI model and that in itself produces more desirable responses than training on a larger but more average-quality data set, is there still potentially useful information that could be extracted from the larger training set as well? How can we classify which aspects of a larger training set are desirable to keep while filtering out noise that competes with higher-quality source material? It feels like the progress of generative AI over the next few years might be defined more by these kinds of questions than just trying to build ever larger models using ever larger sets of training data.

Chris_Newton··on Ask HN: Freelancer? Seeking freelancer? (June 2024)
SEEKING WORK | Cambridge, UK | Remote preferred (occasional travel OK)

Offering hands-on software development and technical leadership services, I specialise in:

• Mathematical, scientific and engineering applications

• Software for hardware (IoT, embedded, lab equipment)

• Challenging presentation (custom UIs, data viz, multimedia/video)

I have over 20 years’ experience developing software, mostly working with smaller organisations. If your situation looks anything like these, I could probably help you:

• A scale-up struggling to overcome early tech debt and transition to more sustainable development

• A hardware start-up looking to outsource software development to a multi-skilled freelancer

• A software start-up that can’t hire permanent senior+ level developers fast enough to hit targets

• A research group needing a professional to implement models and visualisations in good code

Every project is different and will be treated as such, but recently I’ve been favouring TypeScript or Python for web development (with React or Vue for UIs) and Python, C++ or Haskell for more mathematical or hardware-related work. I have experience with a wide range of target platforms from AWS to embedded ARM.

If you have a project that I may be able to help you with, you’re welcome to get in touch. My contact details are in my profile.

Chris_Newton··on Old dogs, new CSS tricks
As a developer of a similar vintage, I can relate to this mindset, but I think we should be careful about it. We learned in an era when differences in browser behaviour were a big deal and new versions of browsers sometimes came along years apart. But those days are long gone — even Safari now gets several updates each year — and we should let the habits they created go with them.

Should we be cautious about relying on non-standard features or trusting the perma-beta culture that Google never seems to have grown out of? Of course. But there have been many recent developments that are useful and already widely supported, and not using those when they provide a good solution to an immediate problem seems counterproductive.

Chris_Newton··on Essays on programming I think about a lot
I used literate programming for a multi-year project in a specialist field with a lot of technical details once. I was working solo but on behalf of a client, and I was working in Haskell to replace an earlier system I’d written in Python, so I wanted to leave documentation with clear explanations of how everything fit together for anyone who might have to maintain that system later.

The pros and cons of the literate style turned out to be refreshingly obvious. It felt more like writing a book than a program. That provided a clear narrative structure and led to thorough documentation, which really was very useful when revisiting areas that had been designed and implemented months or years earlier. On the other hand, it also required a clear vision, a coherent view of how the whole story could eventually fit together, to know where to put things.

For a project like that, IMHO it was a trade-off worth making, in the same sense that maintaining any good documentation (or a test suite, or build automation, or…) is an investment of time up-front that pays dividends later. I would be more cautious about adopting literate style for a larger project with a team of developers and a process where the system is developed in small increments with relatively little planning first. Editing literate code is also like editing a book: you aren’t merely changing the code, you’re also rewriting the story. You probably need a team who are committed to doing that well for the idea to work, though again this isn’t really any different to any of those other up-front investments.

Chris_Newton··on Ask HN: Freelancer? Seeking freelancer? (May 2024)
SEEKING WORK | Cambridge, UK | Remote preferred (occasional travel OK)

Offering hands-on software development and technical leadership services, I specialise in:

• Mathematical, scientific and engineering applications

• Software for hardware (IoT, embedded, lab equipment)

• Challenging presentation (custom UIs, data viz, multimedia/video)

I have over 20 years’ experience developing software, mostly working with smaller organisations. If your situation looks anything like these, I could probably help you:

• A scale-up struggling to overcome early tech debt and transition to more sustainable development

• A hardware start-up looking to outsource software development to a multi-skilled freelancer

• A software start-up that can’t hire permanent senior+ level developers fast enough to hit targets

• A research group needing a professional to implement models and visualisations in good code

Every project is different and will be treated as such, but recently I’ve been favouring TypeScript or Python for web development (with React or Vue for UIs) and Python, C++ or Haskell for more mathematical or hardware-related work. I have experience with a wide range of target platforms from AWS to embedded ARM.

If you have a project that I may be able to help you with, you’re welcome to get in touch. My contact details are in my profile.

Chris_Newton··on Should you use upper bound version constraints?
Major Python libraries pretty much never just remove things over the course of a single major release. Things get officially announced first, then deprecated for at least one release cycle but often longer (which is communicated via DeprecationWarning etc), then finally retired.

I’m not sure that’s a realistic generalisation. To pick a few concrete examples, there were some breaking changes in SQLAlchemy 2, Pydantic 2, and as an interesting example of the “rename the package instead of bumping the major version” idea mentioned elsewhere, from Psycopg2 to Psycopg (3). I think it’s fair to say all of those are significant packages within the Python ecosystem.

Not all dependencies are internal. If library A exposes type X, and library B exposes type Y that by design extends X […] Now imagine that library C exposes type Z that also by design extends X

Yes, you can create some awkward situations with shared bases in Python, and you could split all of the relevant types into different libraries, and this isn’t a situation that Python’s object model (or those of many other OO languages) handles very gracefully.

Could you please clarify the main point you’d like to make here? The shared base/polymorphism complications seem to apply generally with Python’s object model, unless you have a set of external dependencies that are designed to share a common base type from a common transitive dependency and support code that is polymorphic as if each name refers to a single, consistent type and yet the packages in question are not maintained and released in sync.

That seems like quite an unusual scenario. Even if it happens, it seems like the most that can safely be assumed by code importing from B and C — unless B and C explicitly depend on exactly the same version of A — is that Y extends (X from A v1.2.3) while Z extends (X from A v1.2.4). If B and C aren’t explicitly managed together, I’m not convinced it’s reasonable for code using them both to assume the base types that happen to share the same name X that they extend and expose through their respective interfaces are really the same type.

Chris_Newton··on Should you use upper bound version constraints?
social expectations around Python package versioning are very different from JS package version

If anything, I’d say in my experience the Python community tends to be more willing to make big changes. After all, Python itself famously did so with the 2 to 3 transition, and to some extent we’re seeing a second round of big changes even now as optional typing spreads through the ecosystem.

Admittedly, the difference could also be because so few packages in JS world seem to last long enough for multiple major versions to become an issue. The Python ecosystem seems more willing to settle on a small number of de facto standard libraries for common tasks.

Since types from dependencies can often surface in one's public API, having a tree of dependencies means that many libraries will end up referring to different (and thus ipso facto incompatible) versions of the same type.

Leaving aside the questionable practice of exposing details of internal dependencies directly through one’s own public interface, I don’t see how this is any different to any other potential naming conflict. Whatever dependency model you pick, you’re always going to have the possibility that two dependencies use the same name as part of their interface, and in Python you’re always going to have to disambiguate explicitly if you want to import both in the same place. However, once you’ve done so, there is no longer any naming clash to leak through your own interface either.

Chris_Newton··on Should you use upper bound version constraints?
It's a problem because you can only have one version of any given package in your dependency tree. You can't have `foo 2` and `foo 3` in your dependency tree.

That does seem to be the fundamental problem with the Python model of dependency management.

If your dependencies have transitive dependencies of their own but your dependency model is a tree and everything is clearly namespaced/versioned, you might end up with multiple versions of the same package installed, but at least they won’t conflict.

If your dependency model is flat but each dependency bakes in its own transitive dependencies so they’re hidden from the rest of the system, for example via static linking, again you might end up with multiple versions of the same package (or some part of it) installed, but again they won’t conflict.

But if your dependency model is flat and each dependency can require specific versions of its transitive dependencies to be installed as peers, you fundamentally can’t avoid the potential for unresolvable conflicts.

A pragmatic improvement in the third case is, as others have suggested, to replace the SemVer-following mypackage 1.x.y and mypackage 2.x.y with separate top-level packages mypackage1 x.y and mypackage2 x.y. Now you have reintroduced namespaces and you can install mypackage1 and mypackage2 together as peers without conflict. Moreover, if increasing x and y faithfully represent minor and point releases, always using the latest versions of mypackage1 and mypackage2 should normally satisfy any other packages that depend on them, however many there are.

Of course it doesn’t always work like that in practice. However, at least the problem is now reduced to manually adjusting versions to resolve conflicts where a package didn’t match its versions to its behaviour properly and/or Hyrum’s Law is relevant, which is probably much less work than before.

Chris_Newton··on Ask HN: Freelancer? Seeking freelancer? (April 2024)
SEEKING WORK | Cambridge, UK | Remote preferred (occasional travel OK)

I have over 20 years’ experience building software, specialising in:

• Technical applications (mathematical, scientific, engineering)

• Software for hardware (IoT, embedded, lab equipment)

• Interesting presentation (custom UIs, data viz, multimedia/video)

My target market is start-ups/scale-ups/SMEs needing immediate help with either hands-on development work or technical leadership.

For small to medium applications — such as a UI for a hardware product or implementing mathematical models and visualisations for researchers — I can offer a fully outsourced development service if required.

I can also join an existing team temporarily to give a short-term boost, perhaps helping with a challenging project or co-ordinating a larger initiative across teams. This can include a training/mentoring element if you need to upskill quickly.

Every project is different and will be treated as such, but recently I’ve been favouring TypeScript or Python for web development (with React or Vue for UI work) and Python, C++ or Haskell for more technical or hardware-related work. I have experience with a wide range of target platforms from AWS to embedded ARM.

If you have a project that I may be able to help you with, you’re welcome to get in touch. My contact details are in my profile.

Chris_Newton··on Why are so many people being hit with £5 fines for 'counterfeit' stamps?
There actually is a return address on the back of the envelope that was open. However, given the story so far, it seems remarkably generous to trust that anything sent back there would firstly arrive at all, secondly result in a valid replacement for the affected stamps being sent, and thirdly result in that valid replacement reaching us properly.

(For avoidance of doubt, in my previous comment, I meant that we had no opportunity to refuse the delivery.)

Chris_Newton··on Why are so many people being hit with £5 fines for 'counterfeit' stamps?
There are certainly other plausible explanations for what I received. If there were some way I could check those stamps were legitimate, the question would be academic anyway.

The whole process is pretty daft if you think about it, though. Many of us had perfectly good stamps before. Royal Mail decided not to honour them — itself an ethically dubious policy, given how those stamps had been marketed and sold — and effectively forced us to swap them out.

To enable that, they set up a system where you had to send them old stamps collectively worth up to (IIRC) £200 without any verifiable proof of what you’d actually sent, in an envelope clearly recognisable as containing that type of stamps. Then they’d send you back replacement stamps, also in a clearly recognisable envelope, which ironically should now be voidable in the event of a problem, but evidently without any willingness on their part to actually do so. For as long as I can remember, the public have been advised never to send cash by post, so the postal service forcing everyone to send a cash substitute in clearly identifiable packaging both ways was an… interesting… strategy.

In a surprise to seemingly no-one except Royal Mail, there are now widespread reports of the clearly identifiable, untraceable, fungible value tokens going missing on the way in or their replacements themselves being replaced by counterfeits on the way back. There also seems to be a not-at-all-dodgy secondary market now, where you can buy the aforementioned replacement tokens for less than their “face value”.

However, based on my experience, Royal Mail don’t seem to think there is any problem here. Based on reports such as the one we’re discussing today, a significant number of innocent people might already be out some extra money as a result, and that money seems to be ending up with Royal Mail. A cynic might think someone had set up a scheme openly inviting theft and counterfeiting, forced a country full of people to use that scheme or lose money, ignored the obvious problems even when they were explicitly reported, and then directly profited from the criminal activity.

Chris_Newton··on Why are so many people being hit with £5 fines for 'counterfeit' stamps?
Unfortunately, in this case the item was delivered by Royal Mail themselves and left in our mail box, so there was no opportunity to do anything like that.

I might have bought the idea of a mechanical issue causing the opening if it had been a clean cut and the contents had been otherwise undisturbed, but with a very rough tear and the contents having apparently been shoved back into the envelope, it seems rather less likely.

Chris_Newton··on Why are so many people being hit with £5 fines for 'counterfeit' stamps?
I don’t know whether I should post this as it may be an ongoing case, but not so long ago I sent off my old stamps to the official “swap out” service to have them replaced by the new barcoded ones. A few days later, I did indeed receive equivalent new stamps back — in an envelope that had clearly been tampered with, being roughly cut open all down one side and having the paperwork accompanying the stamps obviously bent and having been pushed back in awkwardly.

I immediately contacted Royal Mail via their online system and noted the problem and my concern that this could mean the stamps had been replaced with fakes before reaching me. I asked if there was any way I could verify that they were legitimate before I used them.

A few days later, I received a bizarre email back from someone at Royal Mail, saying they were sorry to hear that I wasn’t happy with their response (what response?!) and can certainly understand my frustration but we are not able to progress any further (than what?!), along with various other words that seemed designed to try to appease me while not actually saying anything of substance, and a recommendation to contact some other part of Royal Mail, which I may yet do.

It was a surreal experience, but having been prompted to do it by seeing several reports of people who’d sent their stamps in and either never received the replacements or had problems with them, and having received such an obviously tampered delivery, with the entire process from collecting the stamps from my local post box to delivering the replacements to my home being under the control of Royal Mail, I can’t say I was particularly reassured by the response so far.

I haven’t yet tried to send anything using the new stamps…

Chris_Newton··on Difftastic, a structural diff tool that understands syntax
if you do this your difftool becomes a compiler

Some linters and formatters are effectively compilers already, so that doesn’t seem completely implausible in itself. Finding canonical representations of common coding patterns so you can quickly and reliably determine that they are equivalent is a different question, though.

Chris_Newton··on The hunt for the missing data type
graph is a data structure, not a data type.

It can refer to either.

Any concrete data structure that uses indirection — which means pretty much anything more complicated than dense arrays and records — is indeed a form of graph.

But graphs, and more constrained forms like DAGs and trees, can also be abstract data types, implemented by a variety of concrete representations.

One of life’s little ironies is that implementing a general abstract graph using a general concrete graph whose records and pointers correspond (roughly) 1:1 with the nodes and edges in the abstract graph is often a poor choice.

Moreover, it’s not unusual to have an abstract graph implemented using a non-graph data structure (for example, a dense adjacency matrix) or to use a graph-like data structure to implement an abstract data type whose interface doesn’t look particularly graph-like (for example, a piece table).

Chris_Newton··on The hunt for the missing data type
But it's orders of magnitude too slow and memory-inefficient for performance-sensitive graph computations—if you have even moderately sized graphs and graph algorithms are a bottleneck, you'll need to use something else, and probably something domain-specific.

This seems a little pessimistic to me. There are plenty of application domains that can be conveniently represented using graphs where you might have thousands of nodes and edges — which is what I’d characterise as “moderately sized” — and your needs might only extend to relatively simple and efficient graph algorithms. FGL is excellent in this kind of scenario.

If you do need the kind of algorithms that explode in complexity then even a representation a couple of orders of magnitude more efficient won’t help you much either. Big-O is the thing that is going to spoil your day in this story, not the constant factor. Some problems simply don’t have convenient fast solutions and ideally with those you find a way to change the representation so the original problem doesn’t arise in the first place.

It’s true that there’s also a zone where you have significantly larger graphs but still only need computationally tractable algorithms, and in that case the overheads of a library like FGL become a factor in what is viable. I also don’t disagree with you (and Hillel in the original piece) that it would be difficult to define comprehensive graph functionality to include in a standard library when there are so many different trade-offs involved.

A good — and not entirely unconnected — analogy might be calculating with matrices. It’s convenient to have support for simple but widely useful cases like 3x3 and 4x4 built into your language or standard library. However, once you’re solving systems with hundreds or thousands of rows, you probably want more specialised tools like BLAS/LAPACK, and the structure of your matrix and how you can decompose it start to matter a lot more.

Chris_Newton··on Ask HN: Freelancer? Seeking freelancer? (March 2024)
SEEKING WORK | Cambridge, UK | Remote preferred (occasional travel OK)

I have over 20 years’ experience building software, specialising in:

• Technical applications (mathematical, scientific, engineering)

• Software for hardware (IoT, embedded, lab equipment)

• Interesting presentation (custom UIs, data viz, multimedia/video)

My target market is start-ups/scale-ups/SMEs needing immediate help with either hands-on development work or technical leadership.

For small to medium applications — such as an MVP for a startup, a UI for a hardware product, or implementing mathematical models and visualisations for researchers — I can offer a fully outsourced development service if required.

I can also join an existing team temporarily to give a short-term boost, perhaps helping with a challenging project or co-ordinating a larger initiative across teams. This can include a training/mentoring element if you need to upskill quickly.

Every project is different and will be treated as such, but recently I’ve been favouring TypeScript or Python for web development (with React or Vue for UI work) and Python, C++ or Haskell for more technical or hardware-related work. I have experience with a wide range of target platforms from AWS to embedded ARM.

If you have a project that I may be able to help you with, you’re welcome to get in touch. My contact details are in my profile.

Chris_Newton··on Things You Should Never Do, Part I (2000)
The thing is, there are a lot of small systems maintained by a lot of small teams. Unfortunately, when Joel wrote

“They did it by making the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.”

some people took the “any software company” to heart and now apply the same principle even in those smaller contexts when sometimes the code really is that bad and rewriting the whole thing really is by far the best plan. I’ve seen very senior people citing the Netscape case study to support a no-rewrite position while discussing a 10kLOC system that could be rewritten from scratch in a couple of weeks by a single developer.

Chris_Newton··on Clean your codebase with basic information theory
Again, I like a lot of the thinking behind this. I’m personally a big fan of having clear data models and data flows, and I think there is a lot of potential in ideas like reactivity that clearly define sources of truth and relationships in single places so derived state gets updated automatically when its underlying source(s) of truth change. I also believe realising the full potential of some of these ideas will, as you’ve mentioned yourself, need better tools than we currently have. However, we can already get pretty far with these ideas, and when they work well, it certainly does make for a nice, clear, easy-to-follow design compared to having state and relationships scattered all over the code base.

Where I suspect we might disagree is the idea that everything should fit into this model. Sometimes the data models and relationships get more complicated and the one-way flows implied by the basic idea of reactivity are no longer sufficient. Sometimes the shape of the data flow graph changes on the fly, because we need multiple instances of certain parts of the state. Then we need a way to distinguish between those instances and often some way to connect the repeated parts of the graph back to the wider graph so that we can make changes to the duplicated parts but still enforce relationships with other parts of the data model.

I have never so far seen a reactivity-based approach to modelling state that handles more demanding scenarios like these gracefully, nor a convincing theory for doing so without introducing more than just a general global data store and simple reactivity. This is where I believe there is still a lot of potential for finding better ways to model the data and then better tools to implement those models. The trick, as again you said yourself, is to find the essential aspects and make it simple to describe them without the accidental complexity from the tools becoming excessive.

Chris_Newton··on Clean your codebase with basic information theory
I have actually started to think that, against conventional wisdom, everything should be a global variable. All the issues with global variables can be solved with better tracing tooling.

This is an interesting premise, and actually I think we have quite a lot of examples both of successfully applying the idea up to a point and of where it starts to break down in practice.

Modern distributed applications mostly have a back end with some kind of database and front end UIs that depend on that back end via an API. Those databases are often global data stores, accessible from anywhere in the back end implementation. A lot of work is done to design and manage them, probably modelling some real world system that our application is concerned with, and there are varying degrees of abstraction/isolation used to preserve that design intent.

If the data model is simple then this works OK, particularly if you have a SQL database that can enforce some basic constraints and relationships to make illegal states unrepresentable.

What we usually see as the data models and the actions that update them become more complicated is the introduction of some business logic layer. The rest of the system isn’t allowed free access to update the state any more; it’s required to go through some defined interface that provides specific actions that guarantee the state remains valid.

That’s the writing side. On the reading side, aside from security/privacy issues, we generally don’t have the same concerns with allowing free access to the whole database from anywhere. However, often we need some form of derived data that isn’t directly stored in the database itself but instead can be constructed from other state that is. So again we end up with some kind of abstraction/isolation layer between the rest of our system and the database.

In each of these cases, there is probably data that we’re working with that is not the state that ultimately persists within the database. So the question immediately arises, if we only have global data in our programs, where does all of this transient, intermediary data go? If we put it into our database as well then all the usual problems with concurrency and integrity immediately appear, so we are back to needing something that is local to our immediate logic and can’t conflict with any other logic or indeed any other instance of the same logic that happens to be running in some other context at the same time.

We see analogous issues in the front end UI code for those distributed systems. If there is a relatively simple model then maybe the front end can effectively just fetch/cache the state from the back end API. As things get more complicated, maybe you end up with a front end data store analogous to the back end database that becomes the central, authoritative store of your front end state. And again maybe this provides some defined interface for accepting valid updates to the state and/or for accessing derived data. And again the questions arise about where the intermediate data generated by all of that logic should go if we have only our global store to hold state, and the answer is likely to be some form of more local data.

On top of the persistent state and anything acting upon or derived from it, we also have other kinds of information we work with in front end code. Many UIs will have state that is used purely to control the user’s view into the inner world: the sorting criteria and current page of a table, the current position and zoom level over a map, the last item we’re currently showing in an infinitely scrolling list, look and feel settings like whether we’re using a dark mode theme. Some of this data might apply across the whole UI while other aspects might only apply to, say, a specific table, with each table needing its own instance of that “UI state” data. So again, if everything were global, that would mean we’d need to include every possible piece of UI state for the whole application in the global store.

This comment is already far too long so I’ll just quickly note that there are other recurring themes. One is how to synchronise “global” stores in a distributed system where you might have multiple front ends running with their own copies of the state, or perhaps multiple microservices on a back end that have duplication in their databases because everything is supposed to be independent and denormalised. A related issue is how to represent temporary clones of significant parts of the data during user interactions, like building up a transaction with several changes before atomically committing it or rolling it back (think dialog box on the UI side or an internally consistent batch of changes sent to the back end), or supporting an undo facility that needs to reconstruct a previous version of the persistent state one way or another.

I do believe there’s a lot more we have to learn about different types of state and transient data and how we can model those cleanly in our systems. There are certainly common patterns we touch on in a lot of different contexts. And I think both extremes of having too much data trapped too locally and having too much lifted to global storage have their own difficulties and probably there is some sort of structure in between that would be better than what we typically write today. But it’s not an easy problem or we’d all be solving it by now…

Chris_Newton··on How to commit part of file in Git
It seems a stretch to call having a + character at the start of an added line “its own language”. Staging one line of the two in your example probably takes less than five seconds using this method; in the editor, you just delete the line you don’t want.

If you personally prefer a GUI then that’s obviously your choice. Some people do and I doubt you’ll find many people claiming Git has the world’s best UI. It does work everywhere, though, and I doubt it’s any slower than using a GUI for a task like your example. Either way, you can have lines staged or not pretty much as fast as you can read them and decide whether they should be.

Chris_Newton··on How to commit part of file in Git
Is there something unusual about your example that means (e)dit doesn’t work if you want to treat those lines as separate hunks? Usually if you have multiple logical changes that are adjacent and get recognised as a single change by default, you can just open your editor and keep the lines you want to stage, much the same as you’d do by marking the lines you want to stage in most of the interactive GUIs built on top of Git.
← PreviousPage 3 of 33Next →