Hacker News Reader

Comment by goombacloud | Hacker News Reader

To spare you the trap you can open the file as FD in your bash process (e.g., exec {my_fd}>"$TMPFILE") and then directly delete it before doing anything else, and use the /proc handle to access it ("/proc/$$/fd/${my_fd}")

View on HN

One more fun experiment: avoid the tempfile on disk/tmpfs by storing data into the pipe connecting two processes, where the first one can already exit for the pipe to give EOF when reading beyond the buffered data.

  true | sleep infinity &
  STDINPID="$!"
  TMPPIPE="/proc/$STDINPID/fd/0"
  echo hello > "$TMPPIPE"
  echo write more > "$TMPPIPE"
  cat "$TMPPIPE"
  echo write again > "$TMPPIPE"
  cat "$TMPPIPE"
  kill -9 "$STDINPID" # clean up pipe

Now I would like to avoid the additional process by using the current shell process instead.

Still needs to spawn a temporary process but it can be killed early:

  true | sleep infinity &
  STDINPID="$!"
  exec {mypipe}<"/proc/$STDINPID/fd/0" # hijack FD
  TMPPIPE="/proc/$$/fd/${mypipe}"
  disown "$STDINPID" # suppress killed message on stderr
  kill -9 "$STDINPID" # clean up process now already
  echo hello > "$TMPPIPE"
  echo write more > "$TMPPIPE"
  cat "$TMPPIPE"
  echo write again > "$TMPPIPE"
  cat "$TMPPIPE"

Seems "sleep 60" is safer and still good enough unless the system is completely overloaded.

Since now, multiple concurrent connections will work after my patch. (Need to add subshell) i will need to consider using fd as output cache. Instead of temporary files.

This here is a simple echo server that uses tr to uppercase:

  exec {checkfd}>/dev/null
  CHECKFDPATH="/proc/$$/fd/${checkfd}"
  (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) &
  STDINPID="$!"
  WRITER="/proc/$STDINPID/fd/1"
  
  while IFS= read -r LINE; do
    # only echo lines if we didn't close the connection yet
    if [ -e "$CHECKFDPATH" ]; then
      echo "$LINE" | tr '[:lower:]' '[:upper:]' > "$WRITER"
      if [ "$LINE" = "bye" ]; then
        exec {checkfd}<&-
      fi
    else
      echo "received while closed: $LINE"
    fi
  done < <(nc -q 1 -l 8080 < "$WRITER")
  if [ -e "$CHECKFDPATH" ]; then
    # close checkfd to close writer pipe
    exec {checkfd}<&-
  fi

This variant here works a bit different:

  exec {checkfd}>/dev/null
  CHECKFDPATH="/proc/$$/fd/${checkfd}"
  (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) &
  STDINPID="$!"
  disown "$STDINPID"
  READER="/proc/$STDINPID/fd/1"
  
  { while IFS= read -r LINE; do
    echo "$LINE" | tr '[:lower:]' '[:upper:]'
    if [ "$LINE" = "bye" ]; then
      echo exiting > /dev/stderr
      break
    fi
  done < "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; } | { nc -q 1 -l 8080 > "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; }
  exec {checkfd}<&-

No need of tr here in your example.

The fd will only be used to stor it temporary since, we first need to sent headers, and the headers need to be sent before the bofy. The server base os working fine now with accept and a patch request os already sent to bash. So probaböy on the next release accept will be full usable.

That's an awesome technique, thanks for it! It is Linux-specific though.

It's also useful for signaling between processes to have them continue doing something as long as this proc path exists (the fd can also just be backed with >/dev/null instead of a real file that needs to be removed)

For creating some kind IPC, i would create a named fifo or even use coproc (see man bash)

Normally, the pid and fd could be reused, so you should either make it a known unique path and verify it's still the same, lock the pid via some means (not sure if and how it's possible), or use another technique.

That's super clever, thanks for pointing this out!

Reply on news.ycombinator.com