An lsof Primer | Hacker News Reader

An lsof Primer

danielmiessler.com

169 pointsdanielrm2621 comments

The thing I use lsof for most often these days is watching Flash videos with mplayer (or VLC if you prefer). This allows me to pause/rewind/fullscreen easier. Or sometimes I save the file to watch later. You still have to start the video from within Flash and then pause it so it downloads the video file. Then you can do this:

    $ lsof | grep Flash
    plugin-co 1038   naner   16w   REG   254,2  4442164 7602188 /tmp/FlashXXfHeQqB (deleted)
    $ mplayer /proc/1038/fd/16

Oftentimes the output will give you duplicates for the same temporary flash file (FlashXXfHeQqB in this case) with an extra number after the process id, you can ignore all of that. I'm assuming those are child processes. All you need is the process id (1038) and the file descriptor (16). You can safely ingore the character (w) after the file descriptor.

Back in the day (Flash 9 or so?) the flash player used to just dump temporary flash video files in /tmp and delete them after you close the browser tab. I guess they figured it was too easy to copy the files out of /tmp so the newer flash player deletes the /tmp file immediately after it creates them. This is why we have to go and grab the open file descriptor.

This will work with most but not all websites. Some websites (Hulu) will use a streaming protocol (rtsp) among other things to make the content harder to get at outside of flash.

This works on Linux, I'm not sure about OSX or BSD.

Thanks. I also appreciate watching long presentations in an app such as VLC because I can then watch it at 1.2x or 1.5x speed to save time. After getting used to it a normal speed presentation seems to plod on forever.

Try this script:

  #!/bin/sh

  IFS='\
  '
  for i in `ls -dLtr \`lsof -c plugin-containe -a -u $USER -X +L1 | awk -F ' +' '/\/tmp\/Flash/{ print "/proc/" $2 "/fd/" gensub("[^0-9]", "", "g", $4); }'\``; do
    PLAY="$PLAY $i"
  done
  IFS=' '
  mplayer -osdlevel 3 -fs $PLAY

I have it bound to a shortcut key in my window manager and I installed http://userscripts.org/scripts/show/13333 to automatically pick 720p videos, and pause them.

So I go to youtube in a tab, let it buffer for a bit, hit the shortcut key and enjoy.

Note that it's only semi-recent versions of flash that automatically delete the videos, if not remove the +L1

Just tried it. Consider my mind blown.

If you get an error relating to gensub ("function gensub never defined"), you need to install gawk.

For most Flash video hosting sites, clive¹ is the easiest way to download a video to watch in an external player; sometimes I've had to resort to poking through the HTML source in Firebug, but even that doesn't always yield a useful result. I'll have to keep lsof in mind the next time I hit one of those!

¹: http://clive.sourceforge.net/

In linux at least you can use /proc/`ps -ef|grep flash|grep -v grep | awk {'print $2'}`/fd/

It's a good article. Beware though:
lsof +L1 shows you all open files that have a link count less than 1, often indicative of a cracker trying to hide something
On OS X, lsof +L1 returns tons of files, this is normal.

Indeed - this also indicates files that have deletion pending. A UNIX greybeard warned me about this as I was exploring the getdirentries() syscall.

Thank you. I was a little concerned when my MBP spewed out a list with this option. I tried a little look-up on why that is normal.
From the manpage:
When +L is followed by a number, only files having a link count less than that number will be listed. (No number may follow -L.) A specification of the form ``+L1'' will select open files that have been unlinked. A specification of the form ``+aL1 <file_system>'' will select unlinked open files on the specified file system.
On my MBP (SL) at least, all the files listed with +L1 are from /private/var/folders/
A quick search seems to hint that this is the location to store secure caches and temp files for Snow Leopard.

Why would "lsof +L1" possibly signify "a cracker trying to hide something"? I'm wondering becuase I have lots(!) of results for this command.

Here's something that bothered me for a while: lsof may incorrectly classify your IPv4 connections as IPv6.
$ sudo lsof -i 4:443 $ sudo lsof -i 6:443 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME httpd 12345 apache 6u IPv6 987654321 TCP *:https (LISTEN)
If you see this, the fix is to disable IPv6 by setting "options ipv6 disable=1" via modprobe.

Daniel,
I noticed your contact link is broken in this article (and perhaps all?). The HREF points to:
http://danielmiessler.com/var/www/localhost/htdocs/includes/...

Fixed, thanks.

If we want to kill the process listening on a specific port, we can do that with a simple shell script: http://aaronblohowiak.com/how-to-kill-the-process-listening-...
Similarly, most versions of lsof support built-in filtering of TCP ports by status:
lsof -iTCP -sTCP:LISTEN -P
will show you the open TCP ports without having to use grep (which discards the header.)

Unless I misunderstand it, your shell script can be replaced with fuser, e.g. 'fuser -k -9 1337/tcp'

Oh sweet! I didn't know about fuser. Thanks!

For at least _some_ *BSDs (Net, Free. Don't know about Open, Dragon Fly), see fstat(1) and sockstat(1) for functionally similar tools that ship with the base system. lsof(1) isn't part of base, but is available via ports/pkgsrc if lsof(1) is required or desired.

Thank you for this! I almost feel embarrassed not knowing the extent of lsof's abilities; I've used it for a long time just for basic things (e.g. lsof <file>, that's it!). Very awesome, I can't wait to use some of these in my scripts.

I use lsof a lot in combination with strace. E.g. lsof -p some_process tells you that fd 9 is the socket you're interested in. Then you can run something like strace -e read=9 to see all the reads on that socket.

The one I use most often is:
lsof -i -P
This shows all the open Internet connections and port numbers.

I thought that this site was for "hackers", not UNIX newbies.

Reply on news.ycombinator.com