Sherver: Bash lightweight web server | Hacker News Reader

Sherver: Bash lightweight web server

github.com

163 pointsturrini66 comments

Here's my small webserver:

    #!/bin/bash

    while : ; do cat conference.txt | nc -l 80; done

Here's the story behind it:

https://www.go350.com/posts/finding-a-hacked-server/

I liked the story and the impromptu web server!

Two comments:

- using "ping -a" creates an audible ping so you can find the server if you're by yourself without having to look at the screen.

- No need for cat:

    while :; do nc -l 80 < conference.txt; done

TIL. thanx for -a.

Interesting. I don't hear anything when I use the -a flag.

Let's check with "reveal codes":

    $ ping -a 127.0.0.1 | less

    PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.

    64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.029 ms^G

    64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.023 ms^G

Looks like you need a terminal that has a BEL sound.

It just echoes a Bell character to your terminal. Make sure your terminal emulator supports it, and is configured properly: https://en.wikipedia.org/wiki/Bell_character

Windows Terminal shows a "visual bell" in the tab bar whenever BEL is emitted (even though I have it set to "audible" in advanced settings of the profile, hmm).

Edit: Uhm, right. I have also silenced all system sounds in Windows.

Whoa, nifty.

Why does firefox shows the "chose an application to open this link" when you access the http://localhost link, but shows the content correctly when you access the 127.0.0.1/0.0.0.0 ip-based version?

Browser expects to see some HTTP headers before the content. Adding HTTP version with response code, content type declaration and empty line dividing headers and content should do the trick:

    HTTP/1.1 200
    Content-Type: text/plan;

    1st line of content

I could read stories like this forever, they are educational and engrossing, thanks for sharing

for a more complete solution, check out Bash On Balls, a Rails clone written in bash:

https://github.com/jneen/balls

> I didn't want to install and configure Apache or nGinx. In fact, I didn't want any configuration.

Personally, in that case, I'd use:

python3 -m http.server

There are many similar options. I think that awhile ago there may have been a list of one liner web servers posted to HN. Either way, I've been using python's for years and it's just muscle memory now.

Big list of http static servers:

https://gist.github.com/willurd/5720255

That's my goto solution for a simple server if I'm testing something locally but it has issues dealing with concurrent requests in its simple form.

For example I ran a simple hello world test service on Kubernetes once using Python's http.server.

Just having 3 Kubernetes health checks happen on regular intervals would routinely cause the server to respond with errors. I think what ends up happening is if you have 2 requests coming in within a short enough period of time one of them will fail. You have to use a more complicated threaded http server if you want a zero dependency solution.

`npm install -g live-server`

Also works great (but a bit bulkier sure).

They reinvented CGI scripts but didn't actually implement CGI. Could have implicitly supported a lot of apps by default. There's some pretty decent CGI libraries for Bash, too.

Personally I use busybox's httpd (which does support CGI) for a small portable web server.

Just an idea, along the lines of a single binary webserver: one may use Fossil SCM [1] to easily spawn a webserver ('fossil server', or even 'fossil ui'), no need for root (port 8080 by default).

The pages could be served as part of a project (all version-controlled), or as documentation pages, or as wiki. Styling is somewhat limited, but could still be done creatively.

Additionally, a static website or even a whole application (PHP or the likes) could be rolled out via /ext, the 'extension' path.

[1]: https://www.fossil-scm.org

It is a socat server, not bash. Basically a bash script calling socat.

Definitely not “pure”.

Someone beat me to it. When I think of pure bash, I don't really think of invoking anything that isn't a builtin or a grammatical feature of bash. If socat is fair-play, almost anything you can spawn on a shell should count then.

edit: on top of that, I already found a perl script utility in the repo: https://github.com/remileduc/sherver/blob/master/scripts/uti...

I think I'll just write something similar and wrap Apache with a shell scrip, claim that it's a web server written in bash, post it here and see what reaction I get, just for shits and giggles.

Here: https://github.com/remileduc/sherver/issues/1 that way you can have HTTP/1.1 support even with arbitrary output lengths. I was planning on sending this to the linked repo so the owner doesn't need to use HTTP/1.0.

EDIT: changed link to the github issue I created with inlined utility.

Jokingly, "sudo service apache2 start" may also be considered pure Bash :-)

Exactly!

~~Right. "Pure" bash would mean they somehow got sockets working in the scripting language itself, which is not possible unless you're on a system with `/dev/tcp` (which isn't many these days).~~

I am wrong! See response.

/dev/tcp is a Bash construct (mentioned as "pseudo-device file" in the manual): https://tldp.org/LDP/abs/html/devref1.html

Oh huh, TIL it's only a Bash thing. I stand corrected - I thought it was a Linux thing that was removed for security reasons.

Thanks for correcting me :)

Bash can't do listen sockets, so I don't think a pure bash web server is possible. You can make an "pure" gawk web server, though it works terribly https://gist.github.com/willurd/5720255#gistcomment-3143007

Ok, we've impured the title above.

There’s always https://github.com/cemeyer/httpd.sh .

This seems like an odd usage of cat and command substitution to me:

    HEAD_TEMPLATE=$(cat <<EOF
        <title>Sherver example</title>
        <meta name="description" content="Sherver example">
    EOF
    )

https://github.com/remileduc/sherver/blob/master/scripts/ind...

It achieves multiline block text in bash, which is otherwise a PITA to reason about. It can keep certain scripts cleaner, especially since `cat` is often inlined, without process spawning.

> especially since `cat` is often inlined, without process spawning.

That would be surprising. How would that work? `cat` is definitely not a bash built-in.

Cat is not a specified built-in but many implementations do build it in to speed up scripts that use it quite heavily.

Other examples include `which`, `time`, `echo`, etc. They're usually promoted to built-ins when it's clear that's the intended command to run, but there are still analogs on the PATH if need-be - I'm not sure exactly how bash would detect a non-standard executable there to know when it's unsafe to promote to a built-in, though.

For example with echo:

    $ bash -c 'echo hi'
    hi
    
    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /* 24 vars */) = 0
    
    $ strace bash -c '$(which echo) hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7fffe36c6cd0 /* 24 vars */) = 0
    execve("/usr/bin/echo", ["/usr/bin/echo", "hi"], 0x7fffe2e16ff0 /* 25 vars */) = 0

In fact for me, `cat` always gets inlined, even if I call it with its full path or with `which cat`.

There are cases where the built-in and the actual executable differ - the most prominent case being `time`. The built-in does not have the command line arguments to print out things like pre-empted scheduling (forced context swiches).

    $ time --help
    --help: command not found
    
    $ /usr/bin/time --help
       Usage: /usr/bin/time [-apvV] [-f format] [-o file] [--append] [--verbose]
          [--portability] [--format=format] [--output=file] [--version]
          [--quiet] [--help] command [arg...]

You had:

    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /\* 24 vars */) = 0

Something seems off. strace(1) prints the trace to stderr not stdout, so you should need |& not | to catch it with grep. At least, that's how it works here.

Also, cat does not appear to be inlined for me:

    rascul@smarts:~> strace bash -c 'cat /etc/os-release' |& grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "cat /etc/os-release"], 0x7ffe7b5334e0 /* 88 vars */) = 0
    execve("/usr/bin/cat", ["cat", "/etc/os-release"], 0x562c447076f0 /* 88 vars */) = 0*

Yeah good catch, I forgot it since I didn't copy 1:1 the command line into vim when I formatted it for HN. I use fish + powerline so the command line doesn't copy correctly. In fish, it's `&|`, which would look weird anyway. I just forgot to add the `&` after the `|`.

The output was copied 1:1 though.

The inlining is an implementation detail and I've seen it happen differently across different platforms.

> The inlining is an implementation detail and I've seen it happen differently across different platforms.

Do you have more information on this? I would be interested to read more about it. As far as I was aware (and my knowledge is certainly not exhaustive), the only thing kinda related is as noted in the bash man page:

> The command substitution $(cat file) can be replaced by the equivalent but faster $(< file).

I don't, sorry. I just know it's something some implementations opt-in to (especially drop-in replacements to bash). It's behavior I've observed quite a bit and have leveraged it in some cases to improve the performance of some critical bash scripts over the years. I'd also be interested in some prose about it.

Are you sure?

    strace -ff -e execve /bin/bash -c 'x="$(cat <<< "test")"'
    ... snip
    [pid 311214] execve("/usr/bin/cat", ["cat"], 0x56280da76fc0 /* 3 vars */) = 0

Yep, I'm sure. It's not a guarantee (nor is it specified, to my knowledge) that the inlining happens. You also used a subshell there, which might make a difference.

The original poster is using a subshell. Your example uses echo, which is both a built-in AND an external program.

They are not comparable. I've never seen a shell inline cat, probably because it would make no sense given there are POSIX builtins to do exactly what cat does.

It of course makes sense since `cat` is widely used for the purpose of the original code, just not in a subshell. It's used often to output blocks of text to stdout/err for multiline output, namely --help text.

strace doesn't lie here, and I've seen it substitute cat in trivial cases quite often.

Indeed strace doesn't lie but you have yet to provide an example of a system where what you're saying about cat is true. You've shown a number of examples which do not actually back up the claim you are making, and have not responded to other posters pointing out the flaws in the examples you did provide.

I'm genuinely curious to see an implementation but I suspect you've misunderstood something in the past and your statement was incorrect.

"echo" and "time" are explicitly bash built-ins, "which" and "cat" are not, and you will always get a process spawned there.

Your second strace is misleading, you didn't trace child processes.

    $ strace bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffddfcdaee0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x55eb565cb0b0 /* 59 vars */) = 0
    
    $ strace -f bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffd09b3e228 /* 59 vars */) = 0
    [pid 81493] execve("/usr/bin/which", ["which", "echo"], 0x5560d172e0b0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x5560d172e0b0 /* 59 vars */) = 0

> They're usually promoted to built-ins

There’s no runtime “promotion”. You can read about it all in bash(1).

I believe the pure bash way of assigning a multi line string to a variable, without dealing with quoting, might be as follows:

    IFS='' read -r -d '' HEAD_TEMPLATE <<'EOF'
        <title>Sherver example</title>
        <meta name="description" content="Sherver example">
    EOF

https://stackoverflow.com/questions/1167746/how-to-assign-a-...

I mean, if it's got no single quotes:

  HEAD_TEMPLATE='
  <title>Sherver example</title>
  <meta name="description" content="Sherver example">
  '

Yes this is indeed simpler, however I noticed in some of the code single quotes were also used [0] so I wanted to use an example that would work for that.

[0] https://github.com/remileduc/sherver/blob/master/scripts/ind...

If this is pure bash then so is

``` caddy file-server --browse --listen :80 ```

Kudos for scripting a great hack in BASH.

Since the title reads "pure bash" however I was expecting to see an implementation using builtin bash features such as /dev/tcp.

Have you considered solving this problem using BASH builtin support for TCP?

I recall that bash can't create a server socket, because bash doesn't bind().

Plan 9's rc-httpd is quite useful, supports cgi and serves many plan 9 web sites, many of which run werc http://werc.cat-v.org/

It can also run on Unix via plan 9 port (p9p) or derivatives.

https://github.com/9front/9front-test/blob/master/rc/bin/rc-...

Ah, no. Bold claim unrealized. Immediately I see many unnecessary forks and externals to do things that bash actually can do itself.

For the occasional service where you are essentially just wrapping an existing binary, a bash server behind uwsgi can be totally sensible.

Fantastic. For a more sinatra-inspired take, see also https://github.com/guigo2k/ganesh