To the brain, reading computer code is not the same as reading language (2020)
news.mit.edu
news.mit.edu
> While the programmers lay in a functional magnetic resonance (fMRI) scanner, the researchers showed them snippets of code and asked them to predict what action the code would produce
I have a suspicion that writing code and understanding that written by someone else are different skill sets.
I don’t think this is about understand what is written, it’s about figuring out what that which is written means.
I’ve realized that I sometimes read code in more of a “story” mode, where I’m just looking for the gist of what’s happening and what it is “about” rather than the nuts and bolts of how it actually works.
It makes the code read more like a story from top to bottom.
Some like to organize their code such that any term always appears before it is used. Like a math proof or something. I find this somewhat pleasing but unhelpful for understanding the “story” you mention.
If you look for cweb by itself, you might get weird results, because there are only so many four letter acronyms / words. Look for 'literate programming cweb' or just 'literate programming'.
Maybe that’s just a photo of the crime scene.
Yes, it drives me nuts that e.g. Python makes you put the definitions before their use!
The one thing I'd say almost never works for reading code, though, is the thing people seem to insist is best practice: scattering everything into a hundred separate files with no clear order at all!
Trying to understand that code is like sitting down to an encyclopedia and then being asked to somehow use it to answer to the ultimate question of life, the universe, and everything.
The worst is when you grep the code for "main" in an attempt to at least come to your own idea of order, and it turns out that a bunch of files have their own main functions for various unit tests in files the build system just happens to ignore :/.
Given the strict formal requirements on order of languages like Pascal, Knuth had to come up with a clever preprocessing scheme to satisfy both the reader and the compiler. (See his tool, https://en.wikipedia.org/wiki/Web_(programming_system) )
Modern languages are a bit more flexible, but can still benefit from some re-ordering.
> The worst is when you grep the code for "main" in an attempt to at least come to your own idea of order, and it turns out that a bunch of files have their own main functions for various unit tests in files the build system just happens to ignore :/.
I used to use tools like grep a lot, but I found that my productivity really improved a lot when I set up an environment that supports 'jump to definition' and 'jump to references' etc.
> Python makes you put the definitions before their use!
Reading code without running it in your head feels like reading prose without actually comprehending it.
It's also sometimes unnecessary to understand the high level picture of what the code does (like "oh, this computes the GCD using Euclidean algorithm, I had to code this in uni once")
So, most people read in a higher level fashion and understand just fine. It's really like reading math. When you read 23443 + 54454 * 454534, do you need to work out the computation to understand what this does?
A lot of code doesn't _do_ anything. And you can still read it.
Eg I'm thinking about type declarations, or constraint based programming, etc.
What you are referring to is simulating the execution in my head to figure out which execution flows exist, figuring out which cases aren't covered, what the function calls actually do, what the significance of those cookies is, etc. That's similar to how I read contracts. But unless I have to write a literary analysis or somebody asks me to find all the plotholes and inconsistencies I don't read normal prose that way.
It might be interesting if there are similarities between programmers tracing execution flows, lawyers reading an unfamiliar contract, and editors finding inconsistencies in short novels.
> predict what action the code would produce
So "this code looks at the pathname, then some cookies, and then either calls the signin function or returns True" seems to align with that.
How could you run code, when you don't know the inputs?
You can reason through how the code could run with some example inputs, but that's just one of many tools we have in our belt to comprehend code.
In eg Rust or Haskell just looking at the types is pretty useful. Or in linear programming, you might want to think about the constraints expressed, but not actually do any arithmetic yourself.
I wasn't arguing here that any existing compiler shows a human level understanding of code. Just that there are many ways to understand code, and executing in your head is but one of tools in your arsenal.
Quoting again the part that that prompted the discussion:
> While the programmers lay in a functional magnetic resonance (fMRI) scanner, the researchers showed them snippets of code and asked them to predict what action the code would produce
The question wasn't about whether the programmers understood the code but whether they can predict what the code will output.
I hypothesize that either the programmer recognizes the purpose of the code or they don't.
If they do recognize, their brain will probably engage with the code differently, in a more high level way, taking into account the bigger picture; but if they don't they probably need to execute it step by step to have any chance of predicting what they do, which is akin to doing a calculation.
It also depends on whether the code is a tangled mess or well structured; how descriptive are function names and variable names; whether the code has comments; etc.
And of course in real world code there is a mix of both modes of reasoning. And the point is, those two modes of reasoning might as well activate different regions of the brain. (or maybe not?)
Well, it also depends on whether the suspect that the code has a bug.
Normally, we try to understand code by taking shortcuts through function names and structure, and fill the rest of in via pattern matching.
But if I suspect that a piece of code has a bug, I work hard to turn off these higher levels and concentrate on what's actually written.
Interpreters run your code, but they don't 'understand' it any better than compilers, which don't run your code.
In a sense, compilers typically 'understand' your code better, allowing them to do more optimisations.
(The boundaries in practice are more blurry.)
Here's an exercise for you: how much did Bill spend at the grocery store?
Bill and Tom went to the show store, and Bill bought a black pair of shoes for 23.56. Tom got a pair of red shoes for 40.30. Then they went to the grocery store, and Tom bought a loaf of bread for 3.50 and a gallon of milk for 5.99. Bill also bought a loaf of bread but didn't get any milk.
Anyone can tell that answering that specific question uses a different part of the brain than answering "what did tom and Bill do?"
And further uses a completely different part than: "modify the story to add a barbershop visit for Bill after the shoe store" which will require parsing the grammar to inject an extra story and keep things consistent.
Big picture !== Small details !== Writing !== Editing existing
The study feels incomplete if the four types of understanding are not being equivalently asked for both a story and the code.
I think two important differences right there are that programming languages are more like "constructing" some sort of entity - a formula, an algorithm, a data structure, an architecture of different components, etc" than it is about communication. Consequently, a lot of "thought" when programming is about the implicit properties of that thing being constructed and less so about the syntactic specifics.
You can do the same in human language, e.g. if you write a recipe, a legal text, a mathematical definition, rules for a board game or instructions for an LLM, but human language can also be used for many modes of communication that have completely different purposes.
I think a comparison of brain activity between reading computer code and reading formalized human language - or between reading formalized and non-formalized human language would be more interesting.
one of the least productive programmers I've worked with was obsessed with Groovy
That would make sense: if you are unfamiliar with programming, it makes sense that the unfamiliar problems zone activates.
It would be interesting to study this, to know if this zone activates because of the unfamiliarity or if it still activates with experience while dealing with code.
It could be that the zone will activate regardless (i.e. because when you are reading code, you are somehow always dealing with unfamiliar stuff - because this is why you are reading code in the first place usually) or not (i.e. because reading code is now familiar).
> But doesn’t that invalidate any comparisons with natural language processing, something that the test subjects do have decades of experience in?
It could: the test subjects would probably need experience in both natural language processing (which they have) and programming for the comparison to be fair. We most likely won't have "as familiar in both stuff", so we'll most certainly have to just do with "familiar enough with code".
(Though I have to say, I'm a self-taught programmer with an academic background in theoretical linguistics as well as university-level experience of several languages quite different to my first language, and this result doesn't not surprise me. There are some obviously analogies to be made at some levels, I think the similarity between human languages and computer code is frequently overstated and a remnant from a more primitive linguistics that was quite narrowly focused on grammar.)
It wouldn't even occur to me to think of them as belonging to the same category.
I also notice that some junior programmers I've worked with tend to argue for coding styles in mainstream languages that read almost like sentences. Not sure if this is a generational thing or an experience thing.
But that said, I see no reason that someone couldn't learn to parse notX as fluently as I parse !X. I suppose it's one of those fashions that change with the times. Probably a big reason that I stick to good old C a lot of the time.
The exception, imo, is functions that take boolean arguments. Languages should make it easy to show what they mean, and devs should take care to explain them.
sort(reverse=true, numeric=false) is much easier to read than sort(true, false).
(Silly example, it's meant to illustrate the general point, not to say anything about the signature of a sort() function.)
Then on the opposite end of the spectrum wrt new things, we have Rust, which comes off to me as pathologically terse and inscrutable - certainly much more difficult for a newbie to parse than SQL.
Unwrap a loaf of bread
Pull out two pieces
Put them side by side
Spread peanut butter on the left one
Spread jam on the right one
Etc.
If it ever comes to that, it should be close to pair programming with AI and it should allow a mix of AI generated and human written code.
Like unsafe in Rust, we should be able to say "I know what I'm doing here, stay out of my way". Then let AI verify human written parts for correctness by means of tests or any other methods.
I think we sort of do this already, but it is the humans who verify the AI generated code and I believe we are a long way from swapping the roles.
Let's hope the results are somewhat better than previous attempt ala COBOL.
Regarding verification, in my opinion the foundation for current LLMs is, similar to humans, not far from total hallucinations. I am not sure if actual verification is really possible in that direction. At most "well, 99% of the others have been doing it that way for the past ten years".
What will people do during the remaining 7.9h?
#426 https://lexfridman.com/podcast/
ETA: I believe Evelina Fedorenko, from this paper, is Edward's wife.
My experience differs:
I find the way how programming is typically (also self-)taught much more fitting to my brain than how natural languages are (also self-)taught.
From my experience I do believe that if natural languages were taught in a different style, it would be much easier for a programmer brain to get good in them (I also consider it to be plausible that there might exist a way on how to teach programming languages that is more natural to "language-minded brains").
I (try to) read code from popular open source projects. It's a bit like sight reading in music because the idea is that I go in blind and try to make sense of it.
This rings true to me:
> The researchers saw little to no response to code in the language regions of the brain. Instead, they found that the coding task mainly activated the so-called multiple demand network. This network, whose activity is spread throughout the frontal and parietal lobes of the brain, is typically recruited for tasks that require holding many pieces of information in mind at once, and is responsible for our ability to perform a wide variety of mental tasks.
There is language of course, but I'm not working hard to process the language so it wouldn't show up as an fMRI signal. I am often holding several things in mind at once, including things like what the code does, what the authors are trying to do, what the author's background might be, how similar software works etc.
I did study math, and it also doesn't really feel like reading or doing math.
I find it interesting that for language I find reading easier than writing but for code or math it is the opposite. When reading code or math it almost feels the same as when I’m learning a foreign language and trying to read it. I have to “translate” it into the my internal “objects”.
It takes a while to “load up” all of those things in our minds, and it’s a bit like a balancing act, you need to keep balancing them to prevent the whole thing from crashing down
Then if you get interrupted, it is like loosing balance and things falling into a mess. Afterwards you need to sort of pick everything up again, which takes time and effort
More than a couple of those in a row are incredibly frustrating
Pg talks about this in his essay of the maker schedule vs the manager schedule: (2009) https://paulgraham.com/makersschedule.html
It's not that my brain is better at caching states, I think it's to do with being a better coder: meaningful comments, better proficiency in using the IDE, knowing when to not implement something from scratch,
A (rare enough) person in chronic coma, regaining consciousness on a sedative-hypnotic drug zolpidem, which lasts for a couple of hours as long as the drug is active, can also show parietal lobes (and a bit of dorsal frontal lobes) activation (as showcased probably in Hamilton's Pharmacopoeia in a series on zolpidem).
Thus, according to modern neuroscience and brain imaging, reading code is basically the same as regaining consciousness on zolpidem while in coma.
I was wondering if there was any correlation in the ability to sight read music and to quickly understand code.
I played a few musical instruments as a child and teen but never got good at sight-reading.
I've heard that musical ability and programming ability are often correlated but I could never have made it as a working musician. I did make a good career out of programming though.
My guess is if I were to try it now I'd be a lot better at it than I used to be.
I do, however, rather consciously think of it as a form of improvisation. And I am much better at improvisation than I was at sight reading.
Narrator voice is a specific voice, cadence, pattern etc. that your internal voice takes on when reading a narrative, rather than the narration of your experience. For me, I have a different voice when reading/writing technical documentation compared to reading/writing fictional works, which both are different from my day to day internal dialogue.
Also when I'm trying to read code I feel distracted and annoyed by any background noise where I can recognize words, like speech or music in a familiar language. And I can't talk while coding, it messes up what I'm typing. Also I really prefer english-like languages, like ruby, to more sigil or symbol loaded, like rust, or full of abbreviations and operators, like Haskell. But I'm sure other developers have other brain pathways activated.
I've found that my descriptions and explanations of how things work generally follow the same cadence as this narrator voice, which tends to help me explain things succinctly and transfer knowledge quickly.
I do agree it's distinctly different from the narrator voice for fiction books. I would assume that's due to the presence/lack of emotive language between the two types of writing.
Perhaps this is a similar phenomenon to the inner monologue that some people have but others don't? Or the ability to imagine various levels of detail of objects without physically seeing them? The mind is a strange beast.
I think this implies that the narrator voice isn't really a "sound echo" but rather your comprehension at work. Just like when reading text.
1. Are return values identical for identical arguments?
2. Are there side effects? What are those?
3. "Congratulations, this narrator here found a story that can be told in isolation!"
4. Are the variables used in close locality to their declaration
5. Are all these loops doing what I think they do? (This one requires iterations of temporary code rewrites)
6. Is the story told with high efficiency? Can some side effects be prevented?
But I could never do this if I was reading something. It's impossible to read something and sing something else at the same time. But it's perfectly easy to do while coding.
I wonder if the authors (neuroscientists) discussed the premise with programmers and linguists to formulate the hypothesis. I mean, I would say that the question should have been: are the same regions of the brain activated when doing math/chemistry/physics/neuroscience(so meta!)/science and coding, instead of when talking/writing/reading and coding.
Maybe there are people who experience coding differently - please, let me know if so, but I would be so much surprised.
To me, both spoken/written natural languages and programming languages feel a little bit like math. :-) (no joke!)
I thus often tend to analyze subtle aspects of words just like one does code audits for hidden backdoors (admittedly - native speakers of English agreed - this is easier and more yielding to do in German, my native language, than in English).
So as a follow up to this research, I'm really interested to see, does the brain read legal/formal documents more similar to how it reads code, or more similar to how it reads natural language. Or is it somewhere in between? Does it change when we introduce some syntax constraints on the legal documents? How much rigid syntax does there need to be before the brain switches from one mode to the other?
In my late teens I saw a contract my dad had brought home, and that one at least already had some basic constructs - I saw analogues to variables, logic/conditionals (as used in a programming language rather than natural language), and possibly even subroutines.
And that's just what I remember, it wouldn't surprise me if there were already more such commonalities. It would certainly explain why legal documents are so difficult for the layperson.
Reading language is different, I move my eyes left to right, top to bottom, to consume someone else's ideas. It's a much more passive process than reading code.
I don't usually read code from front to back like you do with natural language.
If I'm doing red green refactor, the red and green phases are all about logic and constraints, but the refactor step is all about communication - I focus on making the code easy to read for others.
But if I'm reading code, I just want to know what it does.
Só, I didn't make a study, but I'd argue that writing code is more similar to writing natural language than reading code is to reading natural language.
If the code is describing a math function then it seems to me reading it will activate the same parts as math.
What if they will read a business logic that is about say users creating accounts?
I always heard that "you'll never read for pleasure again" after law school, but, if anything, I read for pleasure more now than I ever did, and my work "reading" feels like something completely removed from it.
fMRI is not a silver bullet, do your due diligence everyone. I'll wait for this to be replicated before I read anything into it.
I mean; I have a graduate degree, and I’m proud of the work and I think it could very practically be replicated… but will it? Probably not.
If you claim to invent cold fusion probably, but for a study about coding brain I can’t see it being even useful to replicate.
But pragmatic programmers / software engineers, try reading the average PhD's python code. Usually it is a mess.
Knuth may be an exception, rather than the rule.
Not the same as logic? What?? Code is literally logic itself. Code that works with all supported parameters is logically correct code by definition. It's often impossible to prove exhaustively but it can be inferred with very high confidence in many cases.
Producing correct code is the goal of all good developers. Just like producing correct equations is the goal of all good mathematicians.
> Previous studies have shown that math and logic problems seem to rely mainly on the multiple demand regions in the left hemisphere, while tasks that involve spatial navigation activate the right hemisphere more than the left. Working with Marina Bers, a professor of child study and human development at Tufts University, the MIT team found that reading computer code appears to activate both the left and right sides of the multiple demand network, and ScratchJr activated the right side slightly more than the left.
She didn't say anything about what code is in relation to math and logic, she was speaking about how the activated regions in the brain are different between the two tasks. And per the study they are different, or, as she said, "not the same".
I'm not sure if this is what they are implying but I find the statement "reading computer code is the same as reading language or math or logic" to be an odd null hypothesis.
Computer programs can be written in a proof/solver language which would get closer to math/logic. This puts things on a grayscale IMO.
do(thisThing).and(thisOtherThing).until(someStopcondition)....
but when this breaks down it becomes quite ugly.
I wonder if brain-wise reading code is similar to a judge reading through various laws to make a judgement?
Is it also similar to reading a math proof? Are all these things different?
Anyway I don't think it's a remotely interesting finding. Anyone that has done any programming knows that it is not like natural language. We even explicitly know that from the complete failure of languages that try to be programming languages and natural languages like AppleTalk.
The fact that you use English words in your identifiers and comments in your code doesn't make it English any more than writing a paper about maths and using `m` for mass means maths is linguistic.
https://www.baeldung.com/cs/clean-code-formatting#vertical-f...
I don’t agree that “anyone who has done any programming” knows it’s not the same. The number of comments to the contrary in this thread is evidence enough. The parallels between natural and programmatic languages are not just undeniable but often useful, and how the human brain models its understanding of code is orthogonal to that utility.
I know 2 dyslexics who coded for a living. Both reported troubles reading code same as literature.
I got much better about putting white space to spread things out from working with them.
> the researchers showed them snippets of code and asked them to predict what action the code would produce.
That is only (a small) part of programming (the simulating in your head). I'd argue that biggest part of programming is organizing and communication.
Wikipedia's definition [1] is what I had in mind for language:
> Language is a structured system of communication that consists of grammar and vocabulary
Programming languages are structured, have grammars (often described formally using a grammar definition for some parser generator) and vocabulary (the set of functions and types you can use, possibly modules and package, basically), and they are used to communicate (with the machines, and with fellow developers).
The "file format specification" would be a (subset of) the specification of the language, not the language itself, and yes, a file format specification can describe a language (but I believe we only say this for stuff intended to be read and written by a human - so ZIP would not be a language, but HTML would be).
I do agree that programming languages and human languages are not the same thing, they have different characteristics, traits, use cases, behavior… and that's why this study compares them. And that's also why we need to make the distinction. And they also have common stuff, hence the same word. But even if you disagree that they have a relevant amount of common stuff, polysemy is a thing, so it's still "fine" to call English a language and C a language. Quotes around fine because I'm actually only describing what happens: we do call them both "language", that's a fact.
> I'd argue that biggest part of programming is
It might be, but the paper studies and focuses on programming languages, not the more general practice of programming (which is worth studying, but studies always have a (restricted) scope).
> remember the literate programming movement
Where's the confusion in / what's wrong this? Some think that it's better to narrate in code and we call that literate programming. I don't share the vision, but everything looks clear to me.
Also, the vocabulary works very different, in a PL almost all of the vocabulary is defined "at hoc", as you go.
I mean I don't have a problem calling them a language, people should just be vary not to use language ideas for PL.
I wrote this: "with the machines, and with fellow developers"
> A programmer does not communicate with a computer, through a PL
I disagree with this. The programmer describes the steps that the computer needs to perform, to the computer. Or the programmer describes the state which the computer should reach.
The communication is one way (except, arguably, during debugging sessions), but a communication does not need to be two ways.
> He or she writes it down, to later be executed or interpreted, so the computer can "communicate" with the user, through the User interface (not the PL).
You seem to set yourself in the end user perspective, but the developer is also a human, and I'm in the developer's perspective here.
> people should just be vary not to use language ideas for PL.
Agreed, some / many ideas applying to natural languages don't apply to programming languages. But some do.
> the vocabulary works very different
Absolutely. There are strong differences.
(of course, it's fine we don't agree, we are arguing on definitions)
If we look at the debug session or bash scripting, the closest thing of a two way communication. It doesn't "answer" in code.
I see code closer to something like a Floor Plan an architect may use or a CAD drawing from an engineer. Both of them also communicate, but they are not considered languages. They also have a rules (grammer?) and patterns (vocabulary?).
In my first comment, I have broken down the definition of language I found and (supposedly) successfully applied each part of it to programming languages (communication being only one of the features).
Where is my mistake?
Either the definition is wrong, or incomplete, or one of the matches I made is erroneous, or I missed a match.
> an architect may use or a CAD drawing from an engineer. Both of them also communicate, but they are not considered languages. They also have a rules (grammer?) and patterns (vocabulary?).
Tough one, maybe those drawings are indeed expressed in a language, just that we don't usually see it as a language / think of it like this. That would apply to map symbols as well [1].
edit: Just noticed French Wikipedia [2] actually cites cartography as a language without a syntax, so that would check out:
> la cartographie est un exemple de langage non syntaxique
("Cartography is an example of non-syntactical language")
French Wikipedia broadly has the same definition for language as English Wikipedia, but makes the grammar part optional.
We could argue that the examples you cited are indeed not languages using the English definition because they don't have grammar (if rules you mentions are not grammar - and indeed you would probably not need a pass of syntactical analysis to "parse" a CAD drawing)
I think it's clear for the both of us that PLs are not the same as languages. I would just go a little further and calling it a misnomer. For me that is a helpful framing, especially once you go into visual programming.
In this sentence, "is" describes membership (a bit like isinstance in Python, or instanceof in Java or JS). In a definition, this is not the same "is".
In a definition, "is" is the definitional "is", and what is after the "is" is supposed to be a comprehensive list of things you need to match to be called the term preceding the "is" (a bit like "class" or "struct" when you define a type)
Given the definition "A dog is a domesticated animal that has four legs and barks", everything that is an animal, domesticated, that has four legs and that barks is indeed a dog, and something that misses even one characteristic is not a dog (though I made up the definition, it is likely wrong).
Again, polysemy. Not all uses of be are equal. So:
> A language is ... does not mean ... is a language
Indeed, though not for the reasons you had in mind when "A language is ..." is a definition of language: both things around your "does not mean" are indeed true (when you were probably hinting at the second part being false, given your dog example), but the first "is" is definitional, and the second is membership. Both are true because the definition actually defines membership for each individual object described by the definition, and by putting the list of requirements of the definition at the beginning of the sentence, you are describing a particular item that trivially matches the definition. Almost like I did in my example (except I used "everything that", instead of "a").
> I think it's clear for the both of us that PLs are not the same as languages
As natural languages, yes. Which, granted, we would call just languages in everyday life.
> visual programming
How is this related?
I respect your opinion that language in programming language is a misnomer but have not been convinced so far and the more you are writing, the more I'm convinced that your internal definition of language is just too restrictive and I'm also not seeing how this restriction/framing actually helps.
I'm a simple person. You give me a definition of A, you give me something S, if S has all the characteristics given in the definition, S is definitely a A according to this definition.
Now, we can discuss the definition and how it's not entirely (or at all) satisfying, complete, appropriate or useful, and how it could be improved, or how the trick of using qualifiers to further restrict the definition does not cut it.
On this last point, I, so far, have not felt the need to change the definition of language I know to express my ideas clearly, and have been happy to use qualifiers like "natural" or "programming" when needed and unclear from the context. In particular, I have not encountered a situation where calling a programming language a language lead to confusion or bad consequences.
That is sort of my point, if you think of a PL as a language, visual programming is not related. You are drawing a line somewhere between PL and visual programming. If you think of a PL as a specification, it is the same thing just presented differently.
Visual programming certainly deals with a (visual) programming language, which is still a language. A language doesn't need to be textual. It only needs grammar rules and vocabulary. For instance, taking the example from [1]:
- the "set" block and the "for" block are elements ("words") of the vocabulary of "Snap!"
- 'an identifier i needs to appear in a "set" block before appearing in a "for" block' is a grammar rule of "Snap!" (I would guess)
- 'a "set" block can appear in the body of a "for" block' is also a grammar rule of "Snap!"
You can even build a 1:1 mapping between VL and TL, where VL is the visual language and TL is an equivalent textual language, so the notions are essentially the same.
For me, where the notion of language can be criticized is that for natural languages, the grammar rules and the vocabulary set are ever changing and not always well defined. The rules + vocabulary thing is a model at best.
[1] https://en.wikipedia.org/wiki/Visual_programming_language
How about something like editing an image in Photoshop? There's a vocabulary (the commands) they get applied to the image. There are also rules (constraints). One could argue using photoshop is communicating with a computer, what to do with an image.
So while I say language is a misnomer for PL, you say it's a misnomer for natural languages. ;-)
My views on this are highly shaped by my university lessons on formal languages [1], which I also taught a bit teeny bit. You would have text in mind, but the mathematical concepts are actually more generic than this. One of the exercises was modeling a PIN electronic door lock with a finite state machine (how to unlock without unlocking too early?).
These formalisms look like computer science stuff but are actually directly taken from Chomsky's work [2], a (bio)linguist who was/is actually concerned with human languages (which another commenter implicitly mentioned [3])
I guess this stuff is probably valuable to study natural languages, given the correct scope, since it was made for it (xD). It sure is useful in computer science in any case.
One direct result of his work is the (in)famous stack overflow "You can't parse HTML with a regex" post [4] (since regexes can only define regular languages, and HTML is not a regular language - although I believe the answer is a bit wrong, you can find a regex to parse the regular subset asked in the question, though you need to watch for cdata sections (in XHTML) and script and style tags (in HTML), at which point maybe just use a proper HTML parser unless you are dealing with well known content with no such exceptions)
[1] https://en.wikipedia.org/wiki/Formal_language
[2] https://en.wikipedia.org/wiki/Linguistics_of_Noam_Chomsky
[3] https://news.ycombinator.com/item?id=40480913#40483454
[4] https://stackoverflow.com/questions/1732348/regex-match-open...
(s/vary/wary/, I suspect.)
Why not? I thought of crafting a system as telling a story about what the system was going to do, whether or not it involved an interaction with users. The syntax and structure is extremely precise compared to literature, but if it doesn't tell the story clearly, it's less likely to be a successful effort, both in use and for maintainers.
The mediocre ones write their code for the computer/compiler/interpreter. When it works, they're done.
The good ones write their code for human readers to understand, including their future self.
I have a theory that burnout is more common when you keep shifting mental states. Developers who have a phone that keeps ringing throughout the day (or slack messages or urgent emails) are burned out at the end of the day more often than those allowed to be in one state.
It's an absolute nightmare to understand. He may have been in the "coding zone," but he chose variable names that make no sense, abstractions that don't map on to any actual entities, and left no coherent documentation.
I think the organization and communication layers are absolutely critical, and any apparent velocity tradeoff is just accumulating technical debt, because there's probably an entire team who is perpetually on the hook for supporting what you wrote, and actual long-term velocity over time depends on their ability to understand it.
Removing the large team removes those layers and it's why a two person team can do the work of 12.
Adding in documentation, training materials and providing support can happen at the end of the development cycle freeing you to support the project 100%.
It's not that you remove communication or documentation but do it at select intervals. Don't do a full scan of a table everytime you need a value found in the index.
https://news.ycombinator.com/item?id=40474484
it makes it possible for me to cope with projects which require multiple files or which exceed a certain threshold of lines/complexity.
>With LP, I can quickly look through a PDF, using hyper-linked ToC or index, and control/command click at a point and then have the editor open at that point.
I was referring to:
1) Org mode and Emacs. LP was introduced as a concept in 1984, and org-mode was created 2007 or close to that. 20+ years later till we finally got a good formatting option for text + code. One can export to PDF or HTML, but i see no reason why not read the code directly in org-mode. It is better actually.
2) A good programming language with types. That's Rust and it was created in 2015.
3) LLM's to automatically generate descriptions of code. Type annotations are one of the most important hints for LLM's to generate accurate descriptions. Untyped or weakly typed code, like, Emacs Lisp, Python etc, does not help LLM's to be as good as possible.
4) A good LLM which is accurate enough, plus cheap enough and fast. That's Llama3-8b on Groq, in which they provide very fast access to Llama3.
I am currently researching how i can take a Rust project 100000 lines, and transform it to a Literate Programming org document. I have created a small program to cut Rust source code to chunks in an intelligent way [1]. Now i am trying to figure out, how to provide structure to the LLM, by extracting type signatures with rustdoc and feed it as context to the LLM with each query.
It will take some months before i have something good, but based on my calculations, an 100000 line program will be transformed to a giant org literate programming document, in an hour, costing close to, 50 cents.
Point being that LP will make a comeback, it's gonna be quick and it's gonna be cheap.
Langchain-rust for the moment supports only 6 languages, and not even one lisp, like Elisp, but it's tree-sitter integration means it can be extended easily to many many other languages. Also it doesn't support Groq for the moment just OpenAi, Claude and Ollama.
We are getting to Literate Programming being very practical, we are not there just yet, but it's coming soon.
I am researching also how to anticipate in changes in the code, by retrieving git logs, and invalidate some description of the code, which may have fallen behind. There are several wrinkles that need some work.
My needs/usages are far more prosaic, and I am barely a programmer, so the basic .dtx and package which makes documented .tex amenable to use with other languages and a couple of decent plain text editors is working for me.
I'd love for there to be broader support --- I considered quarto and it seemed quite promising, and I wish that LyX had a mode specifically for this, and for a while I was so desperate I was actually considering using GitBook's file inclusion feature, having all the code in discrete little files, and maintaining a series of batch files to concatenate them into working versions.
Alternately, I've been curious if there is some sort of intersection between Literate Programming and Visual Programming ---
>What does an algorithm look like?
That's a million dollar question. Can code be represented in a visual way? Bret Victor tries to give an answer to that. I don't personally think that it can, but there is sure a way, to represent some parts of a program in a visual way.
People use literate programming today in Emacs, in case you are interested. I strongly recommend to try Emacs, LLM's can write Elisp for you, to automate parts of your workflow.
Unfortunately, while I like emacs-style keyboard bindings in Mac OS X, I prefer to use a stylus and often write things out by hand and use handwriting recognition to convert to text (hence my interest in visual programming languages) --- very much envied a co-worker (who was one of two folks I've met aside from TeX conferences who had read TAoCP) who just started it up at the beginning of the day and got all his work done in it during the course of the day.
What do you mean? Formalisms to study languages (natural or programming) are tools.
They're a notation, yes, but one with a precisely linguistic form. Language is an appropriate word to use here.
Maybe "Spoken languages" is a better term for languages like English? I find speaking computer code out loud just doesn't work except for some particular lines of code. "At dataclass class QueueItem magnitude colon int equals one..." is really not clear, if I have to communicate such code in words I wouldn't read it directly and instead describe it, e. g. "QueueItem is a dataclass with the attributes...").
Still, it's interesting to consider why the "language areas" of the brain apparently are not used for reading or writing computer language (or for other different types of mental activity for that matter).
Some processing areas of the brain are obviously dictated by connectivity and adjacency (visual pathway connects to visual cortex, auditory pathways connect to auditory cortex), but reading of both prose and computer code comes in visually, so why does one get processed by Wernicke's area, and one not?! The best I can guess is that multiple areas of association cortex might be competing to recognize/predict visual inputs, and those areas best equipped to do so will win and perhaps suppress the others.
If this suggestion is true - that via competition the best suited area of brain gets used, then it begs the question what are the key differences, pertinent to the brain, between written prose and computer code that cause this differentiated processing?
That, I feel, is the greatest barrier for math teaching in our education system.
The need to compress everything into a line or two, supposedly for ease of transcription and speed.
Hopefully VR and AR can bring about a novel way to teach math which is much more visual/extensive and less about signs and the need to compress
Much like some people get their best intuitions under the shower whereas others do it while walking, other still have them while listening to music...
and
> to fine tune the meta-language mind map of coding it has
I have difficulties to understand / unpack these parts of your sentence, what do you mean?
When I'm reading language (prose), so much is happening -- what is the writer's overarching point and how does this fit in, parsing tons of ambiguity (a word can have lots of meanings), picking up on the writer's "style" (short and to the point? verbose and flowery?), sometimes enjoying the rhythm and even "musicality" of a sentence, visually imagining a "picture" if it's literature, and so forth.
When I'm reading code, it feels much more like following Lego set instructions, or a math proof. And I'm going slowly. It absolutely feels like I'm using a completely different part of my brain from when I'm reading.
Yes there's a little bit of overlap -- variables can use long descriptive names, and I might be reading a non-fiction article about economics that uses some of they math-y concept-building I use in reading code. But that's about it.
Technically both have a "grammar", but they written language grammar has a flexibility and artistry and ambiguity about it, while computer grammar is rigid and logical and unyielding. Interpreting them feels like they use very different parts of the brain indeed.
Would love to see if this is the same when reading different patterns of code - functional vs object oriented. Where do product specs or technical specs lie?
All of this feels more pseudo-science to me - the distinction between language and math is not well understood at the moment.
It can be somewhat ok to solve HackerRank type problems for 30 minutes in such MRI generated field. IRL programming sessions can take up to several hours. And let's see what quality of code would be produced or interpreted for hours in an MF 10000< times stronger than natural. Thus 80% of such neuroscience is non-scalable fragile bs.
I could be convinced that parsing a language at that resolution isn’t how language processing works for general use, but that’s like saying diagraming a sentence isn’t speaking a language. Technically true but missing the point.
They express their thoughts in a simple way yet accurate, they are not getting confused, they do not use a million "you know" and "like" hand waiving and getting lost in the weeds.
Stating that PLs are not absolutely the same as natural languages is true, but no association at all? Or very little association? Difficult to believe.
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html
(Possibly the above might seem too glib at first. So, reasoning further...: )
But this is confusing, since natural language should -by necessity- also be Turing complete and thus be able to express anything expressible in the known universe. https://en.wikipedia.org/wiki/Turing_completeness
Would that mean that some form of Sapir-Whorf would apply, where in the case of some languages, a completely different part of the brain is activated even? https://en.wikipedia.org/wiki/Linguistic_relativity .
I think -intuitively- the best way to see it is some amalgamation of weak-form Sapir-Whorf and Turing equivalence should apply: Not all data structures are equally good at expressing the same things: Some are better than others, depending on the domain.