Distributed grep client in Go | Hacker News Reader

Distributed grep client in Go

github.com

23 pointsirtefa15 comments

I don't know if this is meant to be actually used, but clearly ssh with shared keys and a trivial bash script would be better than just shelling out to grep from go itself. Fairly pointless really.

Without the context of why this was developed, I have to agree with easytiger's point of pointlessness. Perhaps it was just an exercise in curiosity and exploration of the language. If not, I would use pdsh instead http://sourceforge.net/projects/pdsh/ $ pdsh -b -w '10.0.0.1,...,10.0.0.n' 'cat /var/log/your_log | grep "your_pattern"'

Yea i dont mind things like this being posted, but some context would be good. It seems to present itself like a serious tool which might dupe some inexperienced people.

I was not intending to confuse people. This is the first time I posted something on hn. I will give more context when I post something next time. Thanks!

no worries. HN does have quite a big love affair with go.

How long have you been writing in it? I have an idea for autility id like to implement in it for a bit of fun.

Nice one on pdsh! Hadn't heard of it before but will definitely be using it from now on. No more bash for loops with ssh :)

This doesn't really particularly safe to me (assuming that exec.Command allows spaces in arguments):

  cmd := exec.Command("grep", s, logName)

Since you are searching for the expression in just one log file, why not use Go's excellent regexp package?

It started off as a class assignment where it was required to execute "grep" on whatever shell we were running the program on. That's why, instead of using the regexp package, we went for the above (somewhat shitty) approach. But, that's something I am definitely looking into changing!

You might have to work pretty hard to write a grep that's as good as grep. It pulls a lot of swift moves in the name of speed.

exec.Command doesn't run a shell; it just executes the command. So yes, it's perfectly safe to have spaces and other shell metacharacters in there. Whether or not it's efficient here is another question, of course.

So yes, it's perfectly safe to have spaces and other shell metacharacters in there.

You could (for instance) probe content of system files by passing

  -f/some/file

And matching that to the log. If you have some control over the log, you could use this to read content from system files. Of course, it's all indirect, since the file is interpreted as containing regular expressions. But it's certainly not 'perfectly safe'.

I could also read content from system files by using the advanced hacker tool "cat".

I don't see what this has to do with exec.Command.

I've just got a little python script that sshs around my boxes running grep on each, writing it to the a log file with the box name prefixed, and sorting them all by date (as my log lines have date prefixes and this makes sense for me).

Using ssh instead of running a service on each server seems more straightforward to my thinking.

Golang dependency on each machine can be removed by building the executable. The command to execute the server cannot be run remotely. Since the grep is confined to a single log file, it is not very useful. It shows how easy to use go routines to make concurrent requests.

This isn't really 'distributed grep' as far as I can tell. It is simply running grep individually on each machine.

Reply on news.ycombinator.com