Visual programming should start in the debugger
interjectedfuture.com
interjectedfuture.com
I'm quite a bit into developing more tooling around that, and it's really hard - developers particularly want tools that completely integrate with existing IDE's. Debugger infrastructure is usually not built to be very exstensible or accessible. For my extension for Visual Studio https://d-0.dev/ I had to put serious effort into making sure it all still works correctly with VS breakpoints which is extremely difficult when the debugger expects all of the code to be in the same place all the time.
1. Incentives.
I’ve made my contributions to debuggers (Firebug mostly), and devs are not willing to pay for it (we had donations, but I’ve seen people create projects in the space) and so tool makers (browser makers in this case) eat the cost and see it as a cost center. Because of this, the debuggers are at the abstraction layer of the tool itself. The tools change and the cost of keeping the debugger in sync (usually well after the feature is out there) is already high enough.
I put hooks into Firebug to make it better for higher level debugging of frameworks. Later I made a paid extension [*]. It paid for a new computer, and spawned many framework specific extensions over time. I pushed Chrome's DevTools team to add some of the same hooks. Sadly, a decade later, many of those framework level extensions don't use what is available. Again, it is a cost center for the framework development, so just enough gets done, no more.
2. Tools are Siloed.
You may have OpenTelementry, etc., but that data is siloed off somewhere. The IDE doesn't have it. But the IDE has extensions, so maybe someone like Microsoft can bridge the gap and put up a sidebar that is a heatmap of the time that that code takes on the CPU.
3. Levels of Abstraction.
When you talk about state and data modeling, not only is it siloed, but the levels of abstraction are plentiful. It is somewhat possible, but I refer you back to 1--Incentives.
It is kind of wild, otel has these collection protocols but there's afaik not any enduring access protocols. It's all over the wire protocols, nothing is defined for reading an archive of traces. Otel avoids specifying any common APIs for the storage layers, seemingly.
There is no incentive to provide a common access layer because that is where all the players in the telemetry market see their profits.
Maybe it's time to flip the game on its head. Bring reactive, lispmachines, responsive UIs.. I don't know.
There are simply far too many hurdles to do basic computing things. I think Microsoft could do some minor additions to powershell and maybe get something like this.
I wish some billionaire would put up some money and fund a modern lisp machine or something like that. I know it'll probably never work, but I can dream.
MacOS has something like this: https://en.wikipedia.org/wiki/AppleScript#Basic_concepts You can send around events and macro/script GUI stuff.
I want to open my laptop to an operating system with no advertisements or 1000 programs running in the background. It should be pretty simple with basic application support like a text editor, office suite,: database...etc. It would have a super powerful scripting language built-in with DSLs for graphics, UI, mathematics/science, OS, sound, hardware...etc. It would also have some good hardware peripherals you could control. Basically an ideal work and hobby computer that has some centralized/integrated way of accessing all the tools you need with minimal fuss. It just makes me wonder why in some respects there are still many things that the Xerox Alto or Commodore 64 can do that I can't easily do in Windows.
Need to write a simple videogame? That should just involve opening the shell and typing in some high level primitives instead of downloading some crazy big software and libraries
If(Foo){ Draw-Circle -Vertex [100,25] -Radius [2] }
Want to do some linear algebra work?
Invert-Matrix -FileInput 'example.csv' -Sparse 'True'
Maybe you want to read some data from some kind of hardware device that has a sensor (e.g. measuring air quality) and send it to a chart?
ForeverLoop(Get-Data -Port [0]) | LineChart
The commands above are obviously made up, but that is kind of what I'm thinking about. With modern OS, I have to download, install, and configure many tools to typically do these things and it is so messy and cumbersome. I often can't share with anyone else either without them installing all the same apps (e.g .maybe they have to download anaconda Python and postgres).
The Web is also just way too complex now. Ideally there would be a much simpler way to build beautiful and simple sites similarly to the hypothetical apps above.
It would be cool if he did something like that though. Just a clean and simple OS and not the corporate hellscape that is Windows or the alternatives of Mac and Linux. I'd buy a license or the hardware itself if he sold it and develop for something like that.
The problem is that game development companies are willing to invest money in buying quality tools to make their developers productive, where as traditional trillion dollar software companies prefer to let their developers wallow in filth.
Getting a company to invest 5,000$/yr into making their breathing revenue machines that cost 200,000$/yr and produce 1,000,000$/yr in revenue is like pulling teeth in this industry. Companies with breathing electrical engineer revenue machines do not bat an eye at 50,000$/yr in making them productive. Think about how much cool and effective tooling you could get with that kind of budget. But if you are not willing to spend even one coffee worth on your tooling, then one coffee of productivity is all you get to have.
It's also cultural.. when everybody in the room thinks a REST api is state of the art, you'll have a hard time selling them more R&D.
It was a shock because in gamedev debuggers are a Big Deal. The vast majority of AAA gamedev happens in Visual Studio’s debugger or some alternative that is at least as capable.
That said gdb and similar debuggers are still too limited IMO, there's so much we could add on top. Well maybe I'm not aware of improved variants.
To be fair, the core components of a low level debug agent are still largely the same, so it is not like the technology is bad per se. It is just that it only constitutes table stakes, a bare minimum foundation, these days. Even looking at 20 year old technology like time travel debugging that allows you to time travel execution back and forth gives a far better idea of what people are missing by using primitive tools.
The DSL was called Game Oriented Assembly Lisp.
> It supports a long term compiling listener session which gives the compiler knowledge about the state of the compiled and thus running program, including the symbol table. This, in addition to dynamic linking, allows a function to be edited, recompiled, uploaded, and inserted into a running game without having to restart.
https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
Found more info on that here:
https://lisp-lang.org/success/graphics/
> Naughty Dog Software used a Common Lisp DSL to write the Jak and Dexter series of games for the Play Station.
> Naughty Dog co-founder Andy Gavin, says the unique capabilities of Lisp enabled fast development and execution of character and object control – something that was needed to fully realize the numerous 3D creatures and devices which interact with the player in real-time (60 frames per second).
> “Lisp was just the best solution for this job,” comments Gavin. “With leading edge game systems like ours, you have to deal with complicated behaviors and real-time action. Languages like C are very poor with temporal constructs. C is just very awkward for a project like this. Lisp, on the other hand, is ideal.”
> As Gavin explains, “With Lisp, one can rapidly develop meta constructs for behaviors and combine them in new ways. In addition, Lisp allows the redefinition of the language to easily add new constructs; particularly those needed to deal with time-based behaviors and layering of actions. Contrary to popular belief, there is nothing inherently slow about Lisp. It is easy to construct a simple dialect which is just as efficient as C, but retains the dynamic and consistent qualities that make Lisp a much more effective expression of one’s programming intentions.”
https://all-things-andy-gavin.com/2011/02/02/making-crash-ba...
I wonder how he works these days.
> But the craziest thing I did was create a new programming language – with Lisp syntax – for coding all of the gameplay. It had all sorts of built in state machine support (very useful with game objects), powerful macros, dynamic loading etc. It was also highly irregular and idiosyncratic, and in true Naughty Dog fashion “powerful but complicated.”
As for how he works these days, I see in the About page:
> Briefly in 2008, and then full time from the second half of 2009 on, Andy has been concentrating on novel writing.
On the contrary the complexity web development, distributed systems emerges from many communicating processes, that execute in different environments.
A sophisticated debugger might help the former category more than the latter. There’s no shortage of monitoring products that act on the distributed systems level.
I've worked with expensive software before (embedded stuff, few K$ per seat) and it it was horrible - missing features slow, crashes a lot, horrible SDK designs. Yes, maybe it did generate some magical code (I have no way to check this) but the dev UX was terrible.
I am working with clang & command line build system now and this is so much better. If someone offered me an expensive IDE I'l do all I can to make sure I don't have to work with it.
The syndrome seems to track with FP and async and possibly with "teaching" programming. FP and async typically don't work well with today's debuggers. Back in the day nobody was taught to program -- they figured it out by doing. When you are taught by someone who themselves never "did" something about the practice gets lost, imho.
\end{curmudgeon}
Do you mean the 1940s and 1950s? Because by the 1960s, Comp Sci was taught as a discipline and by the 1970s and 1980s anybody who wanted to work in the industry was expected to get a BS degree as a minimum (PhD if you were serious).
Sure, there were still people learning on their own, but they were either smart enough to improve significantly, or amateurs who would always produce spaghetti code which the pros (read: Comp Sci grads) ended up rewriting.
Modern IDEs have lots of visual features, specially tailored to support production languages. Inspection panels, tooltips with variable values and function definitions...
Surely we could add more spatial visualization tools to them. But the step up from printf to a debugging inspector panel was on its own a great advance in building a mental model of the program state and making sense of its behaviour, which is what visual programming is all about.
For programmers interested in advanced visual tools tailored to software creation, see Bret Victor's essays on notations:
But, for some reason, none of these tools catch the attention of the most popular IDEs. I hypothesize that there are major roadblocks to implementing this in a generic and useful way.
The first one is the variety of languages, frameworks, and build tools. For example, analyzing a TypeScript+React code base is not the same as analyzing a TypeScript+Vue code base, even when both use TypeScript and the TSC API is very easy to use.
The second roadblock is that useful visualizations also depend on the characteristics of your system, and creating them is not easy.
Maybe things will change with the addition of AI to analyze code bases, but so far, all the tools I’ve seen are either very niche or very short-lived.
A major bottleneck for people learning programming is developing a mental model of what is in the memory during the runtime and what it looks like. The most successful introductions to programming start by making drawings or small games since it's more intuitive for beginners to iterate and have immediate feedback.
It captures everything for a given request - each method call, params/argument/return values, class name/method/line of execution.
You can visibly see which method called which, as each method call is nested relative to its parent callee, in a single consolidated timeline.
"We need visual programming. No, not like that." https://news.ycombinator.com/item?id=40937119
It only really works with C-with-Classes style C++. But, the fact that it works at all is pretty neat.
This worked (with the original ChatGPT 3.5) about as well as asking it to write code if you did the documentation first. Not great, not terrible, misses more precisely when you want the most help.
I should try again with 4o.
And good debugging support doesn't need to be some fancy "Minority Report" style 4D animated UI. Just give us good type rendering support, an easy ability to chain breakpoints and set conditional breakpoints, "watch variable" support. Some other things: "where did this value came from?", "watch for value use".
Go is an especially bad example. Its debugger doesn't even allow custom type rendering, so if you have something like a custom generic map container, you can't inspect it at the debug time.
I kind of gave up on step-through right around the same time I started writing multithreaded code as well.
It's fine when you're working in a nice segregated part of the system, where everything is well-defined. But it's not always the case.
Also, single-stepping is a nice mental tool, kinda like rubber-ducking. You single-step and explain to yourself what is going on.
But in the past 6 months I’ve started to use graphviz and other tools to quickly document code paths for others
Its mostly just a demo but I think something like this a standard tool would be really helpful. https://github.com/ando818/Svelte-Logger
On a related topic, what would really help debugging in most languages is more consistent functionality for built-in introspection, which allows devs to write sophisticated analysis tools. Just being able to access things like the call stack, argument count, etc. is super important, but unfortunately every language has its quirks about what can actually be accessed, and in what contexts (function vs. method), etc.
In fact, later when it existed in FB if you called it in production code it would cause an exception for your users (console didn’t exist unless you had Firebug installed).