basically in software there's an inherent tradeoff between flexibility (which winestock calls 'expressiveness'), comprehensibility, and performance. a glib and slightly wrong way of relating the first two is that flexibility is when you can make software do things its author didn't know it could do, and comprehensibility is when you can't. bugs are deficiencies of comprehensibility: when the thing you didn't know your code could do was the wrong thing. flexibility also usually costs performance because when your program doesn't know how, say, addition is going to be evaluated, or what function is being called, it has to check, and that takes time. (it also frustrates optimizers.) today c++ has gotten pretty far in the direction of flexibility without performance costs, but only at such vast costs to comprehensibility that large codebases routinely minimize their use of those facilities
comprehensibility is critical to collaboration, and i think that's the real technical reason lisp programs are so often islands
dynamic typing is one example of these tradeoffs. some wag commented that dynamic typing is what you should use when the type-correctness of your program is so difficult to prove that you can't convince a compiler, but also so trivial that you don't need a compiler's help. when you're writing the code, you need to reason about why it doesn't contain type errors. in c, you have to write down your reasoning as part of the program, and in lisp, you don't, but you still have to do it, or your program won't work. when someone comes along to modify your program, they need that reasoning in order to modify the program correctly, and often, in lisp, they have to reconstruct it from scratch
this is also a difficulty in python, but python is much less flexible in other ways than lisp, and this makes it much more comprehensible. despite its rebellion against curly braces, its pop infix syntax enables programmers inculcated into the ways of javascript, java, c#, c, or c++ to grasp large parts of it intuitively. in lisp, the meaning of (f g) depends on context: it can be five characters in a string, a call to the function f with the value g, an assignment of g to a new lexical variable f, a conditional that evaluates to g when f is true, or a list of the two symbols f and g. in python, all of these but the first are written with different syntaxes, so less mental processing is required to distinguish them
in traditional lisp, people tend to use lists a lot more than they should, because you can read and print them. so you end up with things like a list of a cons of two integers, a symbol, and a string, which is why we have functions like cdar and caddr. this kind of thing is not as common nowadays, because in common lisp we have defstruct and clos, and r7rs scheme finally adopted srfi-9 records (and r6rs had its own rather appalling record system), although redefining a record type in the repl is pretty dodgy. but it's still common, and it has the same problem as dynamic typing, only worse, because applying cadr to the wrong kind of list usually isn't even a runtime error, much like accessing a memory location as the wrong type in forth isn't
this kind of thing makes it significantly easier to get productive in an unfamiliar codebase in python or especially c than in lisp
40 years ago lisp was vastly more capable than the alternatives, along many axes. smalltalk was an exception, but smalltalk wasn't really available to most people. both as a programming language and as a development environment, lisp was like technology from the future, but you could use it in 01984. but the alternatives started getting less bad, borrowing lisp's best features one by one, and often adding improvements incompatible with other features of lisp
as 'lightweight languages' like python have proliferated and matured, and as alternative systems languages like c++ and golang have become more expressive, the user base of lisp has been progressively eroded to the hardest of hardcore flexibility enthusiasts, perhaps with the exception of those who gravitate to forth instead. and hardcore flexibility enthusiasts sometimes aren't the easiest people to collaborate with on a codebase, because sometimes that requires them to do things your way rather than having the flexibility to do it their own way. so that's how social factors get into it, from my point of view. i don't think the problem is that people are scratching their own itches, or that they have less incentive to collaborate because they don't need other people's help to get those itches thoroughly scratched; i think the social problem is who the remaining lisp programmers are
there are more technical problems (i find that when i rewrite python code in scheme or common lisp it's not just less readable but also significantly longer) but i don't think they're relevant to octo