Hacker News Reader

Comment by matheusmoreira | Hacker News Reader

What's wrong with bash though? Targeting bash specifically has massively improved my productivity and decreased the incidence of easily avoidable mistakes. Portable POSIX shell scripting is hell on earth but bash scripting with shellcheck can be surprisingly pleasant. I had the same experience with portable makefiles and GNU Make.

For example, I managed to create a surprisingly good test suite with bash and the rest of the GNU coreutils:

https://github.com/lone-lang/lone/blob/master/scripts/test.b...

It even runs in parallel. Submitted a patch to coreutils to implement the one thing it couldn't test, namely the argv[0] of programs. I should probably go check if they merged it...

View on HN

> Portable POSIX shell scripting is hell on earth but bash scripting with shellsheck can be surprisingly pleasant.

Assuming you meant `shellcheck`: you know it works for POSIX compatible shell too right?

What's wrong with bash is that people invariably end up requiring features that aren't available in some deployed version, and then you've lost a lot of the benefit of writing in shell in the first place.

Similar to shellcheck, shunit2 works just fine for running unit tests in posix compatible shell.

> Assuming you meant `shellcheck`

I did. Edited my comment, thanks. I don't know why I'm so prone to that particular typo. My shell history is full of it.

> What's wrong with bash is that people invariably end up requiring features that aren't available in some deployed version, and then you've lost a lot of the benefit of writing in shell in the first place.

But I've gained quite a lot too. Bash has associative arrays. I just can't go back to a shell that doesn't have that.

Shell scripting makes it simple to manage processes and the flow of data between them. It's the best tool for the job in these cases. So there are still reasons for scripting the shell even if one is willing to sacrifice portability.

I just can't believe it's 2023 and we're actually praising a shell for having arrays and dictionaries. Or that there are still multiple shells in use that don't. Or that people still ask questions like "What's wrong with bash though?" with a straight face, as if they don't know.

Now how long are we going to have to wait until somebody invents a way to do named parameters? That will revolutionize the computer industry! I guess it's way to much to ask for a built-in json or yaml parser, all we can hope for is maybe a stringly typed sax callback based xml parser after another 20 years from now, because dom objects in a shell would be heretical and just so unthinkably complicated.

Why are people so afraid to just use Python? Shell scripting and cobbling together ridiculously inefficient incantations of sed, awk, tr, test, expr, grep, curl, and cat with that incoherently punctuated toenail, thumbtack, and asbestos chewing gum syntax that inspired perl isn't ever any easier than using Python, especially when you actually need to use data structures, named function parameters, modules, libraries, web apis, xml, json, or yaml.

This answer is pretty convincing:

https://stackoverflow.com/a/3640403/512904

Thank you! That's a great analysis, and I love the design of PowerShell, which addresses most of my arguments against bash. The Unix community (especially Sun Microsystems) has traditionally had this arrogant self-inflicted blind spot of proudly and purposefully cultivated ignorance in its refusal to look at and learn from anything that Microsoft has ever done, while Microsoft is humble and practical enough to look at Java and fix it with C#, look at bash and fix it with PowerShell, etc.

Here's the summary of a discussion I had with ChatGPT about "The Ultimate Shell Scripting Language", in which I had it consider, summarize, and draw "zen of" goals from some discussions that I feel are very important (although I forgot about and left out PowerShell, that would be a good thing to consider too -- when I get a chance I'll feed it the discussion you linked to, which made a lot of important points, and ask it to update its "zen of" with PowerShell's design in mind):

The Zen of Python.

https://peps.python.org/pep-0020/

Discussion about Guido van Rossum's point that "Language Design Is Not Just Solving Puzzles".

http://lambda-the-ultimate.org/node/1298

https://www.artima.com/weblogs/viewpost.jsp?thread=147358

Discussion of Ousterhout's dichotomy.

https://en.wikipedia.org/wiki/Ousterhout%27s_dichotomy

https://wiki.tcl-lang.org/page/Ousterhout%27s+Dichotomy

Email from The Great TCL War Part 1, started by RMS's "Why you should not use TCL".

https://news.ycombinator.com/item?id=12025218

https://vanderburg.org/old_pages/Tcl/war/

Email from The Great TCL War Part 2, started by Tom Lord's "GNU Extension Language Plans".

https://vanderburg.org/old_pages/Tcl/war2/index.html

Summarization of the important points in those discussions that apply to The Ultimate Shell Scripting Language, resynthesized into a "zen of" list.

Discussion of Support for Declarative and Procedural Paradigms and how it applies supporting standard declarative syntaxes including json, yaml, and xml (which bash still doesn't and probably never will, and PowerShell does of course).

Suggestions for some more "zen of" and design goals, specifically focused on addressing weaknesses or design flaws in popular languages like sh, bash, tcl, python, perl, etc.

Discussion of how "Simplified Debugging and Error Handling" can be balanced with "Macro Processing and Syntax Flexibility", better than debugging C++ templates, Lisp macros, TypeScript code compiled to minified JavaScript, etc.

Discussion of how JavaScript / TypeScript fall short of those goals.

Discussion of visual programming languages for interactive shell scripting as well and general purpose programming.

Discussion of layering visual programming languages on top of textual programming languages.

https://donhopkins.medium.com/the-shape-of-psiber-space-octo...

Interoperability of text and visual programming languages with LLMs for efficiently and reliably analyzing and generating code.

https://docs.google.com/document/d/1QJ98QwC2ubsTNKOFAzUAw6Zy...

Discussion of using Python to build a visual data flow node based shell scripting language on top of Blender (that just happens to support 3D, image processing, video editing, GPU programming, machine learning, Python module integration, and everything else that Blender is great at).

https://www.youtube.com/watch?v=JOeY07qKU9c

Discussion of how to make efficient use of token budgets when using LLMs with text and visual programming languages.

Here is the condensed discussion with the bulk text I had it analyze omitted, so you can more easily read the summaries and recommendations and "zen of" manifestos:

https://docs.google.com/document/d/1wKhdEoLWCZX9TNaftQxLp6ot...

Here is the entire unexpurgated discussion including all the email messages and articles I had it analyze, if you want to see what it was considering (they're interesting discussions to read in their entirety if you're interesting in that kind of stuff, but ChatGPT is excellent at summarizing them and highlighting the important and relevant points):

https://docs.google.com/document/d/1RTxeYjZ2vZsNU4xTj0ZhiXXj...

With the rise of LLMs I think now is a great time to reconsider the design of text and visual scripting and programming languages, with LLM interoperability and compatibility in mind.

ChatGPT:

Designing a text representation for a visual programming language that interoperates well with Language Models like GPT necessitates careful planning to ensure the representation is both meaningful and efficient. Here are several considerations and strategies to optimize the text representation:

Token Efficiency: The text representation should be concise to make efficient use of the token budget of the language model. Avoid verbose syntax and prefer shorthand notations where possible.

Syntax Simplicity: A simple and consistent syntax helps the language model predict and generate code more accurately. Overly complex syntax can lead to increased errors and confusion for both the model and the user.

Standardization: Using standardized syntax and well-known programming patterns can help the language model better understand and process the code since it has likely been trained on similar patterns.

Semantic Density: Each token should carry a significant amount of meaning. This can be achieved through the use of expressive keywords or operators that encapsulate complex actions.

Modular Design: Break down the text representation into modular components. This approach allows the language model to process each part in isolation, reducing the complexity and cognitive load.

Contextual Keywords: Use keywords that clearly indicate their function within the language, making it easier for the language model to provide contextually relevant suggestions.

Window Size Awareness: Design the text representation to be legible and understandable within a typical window size, avoiding lines that span too long and require horizontal scrolling.

Minimize Ambiguity: Ambiguity in text representations can lead to multiple interpretations by the language model, resulting in inaccurate predictions. Clear and unambiguous syntax helps maintain consistency.

Commenting and Documentation: Incorporate support for comments and documentation within the text representation. This not only aids human understanding but can also provide language models with additional context for generating code.

Escaping Visual Elements: For elements that are inherently visual and do not translate well into text, provide an escape mechanism that allows these elements to be described textually in a way that the language model can process.

Alignment with Model's Training Data: Align the text representation with the structures and syntax that the language model has been trained on. This can include following common coding standards and naming conventions.

Iterative Refinement: Use an iterative approach to refine the text representation based on feedback from testing with the language model, ensuring that the design evolves to better suit the model's capabilities.

By considering these aspects, you can create a text representation for a visual programming language that is both language model-friendly and conducive to an efficient programming experience.

Agreed. If you don't test your script in other shells and on other platforms, why advertise compatibility. And not just the script, but also the programs called.

I got burned by supposedly "portable" scripts by the whole Ubuntu dash disaster, and again when I started using Mac OS X, and then again once I used Cygwin and msys2 on Windows.

I do keep portability in mind when writing shell scripts to ease porting later, but without testing there's really no way to be sure "/bin/sh" is right. And some of the Bash features such as arrays are legitimately useful.

Reply on news.ycombinator.com