Hacker News Reader

Comment by pooriaazimi | Hacker News Reader

I'm not trying to start a theological war about grep/ack here, I'm just mentioning it in case someone hasn't heard about 'ack' before and they (like me) might find it extremely useful: http://betterthangrep.com

It's grep, just better. It highlights the selected text, it shows which files, and in what line the text was found (and uses vivid colors so you can distinguish them easily), ignores .git and .hg directories (among others, that shouldn't be searched) by default, you can tell it to search, for example for only `--cpp` or `--objc` or `--ruby` or `--text` files (with a flag, not a filename pattern), and many many other neat features that I'm sure grep has, but you have to remember and memorize them. ack has sensible defaults.

Why ack? http://betterthangrep.com/why-ack/

manpage: http://betterthangrep.com/documentation/

Oh, and ack is written in perl and doesn't require admin privileges to install.

View on HN

Do you know of any C ports of ack? Ack is beautiful and productive, but nowhere near as fast as grep (orders of magnitude slower, in fact).

    gfind . -type f -exec grep -i mbr {} \; >| /dev/null  
    1.10s user 0.81s system 90% cpu 2.113 total

    gfind . -type f -exec ack -i mbr {} \; >| /dev/null  
    24.34s user 4.17s system 96% cpu 29.678 total

(Yes, I know about the flag to search recursively. This is the most fair comparison.)

I'm not on a machine that I can use freely to test, but maybe you shouldn't use -exec? use xargs instead. I don't remember it being slower than grep that much...

I wrote a mostly-clone of Ack in C: https://github.com/ggreer/the_silver_searcher . Output format and most flags are the same. Besides the speed, most users won't notice a difference.

I spared no effort in optimizing. Pthreads, mmap(), boyer-moore-horspool strstr, it's all there. Searching my ~/code (5.2GB of stuff), I get this:

    ag blahblahblah  1.93s user 3.54s system 313% cpu 1.749 total

    ack blahblahblah  9.75s user 2.79s system 98% cpu 12.690 total

Both programs ignore a lot of extraneous files by default (hidden files, binary files, stuff in .gitignore, etc). The real amount of data searched is closer to 500MB.

Only that, Ack's core strength over time will always evolve and depend on Perl's regular expression and text processing powers.

So re writing this in C will fundamentally mean endlessly growing a language which will look similar to the Perl implementation. Or a Perl DSL.

Not that its a bad thing, I find it interesting though. I would say you better start with a specification.

Ag supports the same regexes as Ack. I use the PCRE library. I only call pcre_study once, and I use the new PCRE-JIT[1] on systems where it's available. These tweaks add up to a 3-5x speedup over Ack when regex-matching.

1. http://sljit.sourceforge.net/pcre.html

If you use PCRE, you do NOT support the same regexes as Ack.

"Perl Compatible" isn't really Perl compatible, see http://en.wikipedia.org/wiki/PCRE for details.

Yes, there are a few edge cases, but hardly anyone uses those features. In fact, 90% of the time, most people seem to use literal string matching.

This looks fantastic. Could you by any chance update your PPA for quantal in the future?

EDIT: The last precise build works just fine, though.

Thanks so much, I was staying with grep precisely because of the performance and perl dependency of Ack. Does the silver searcher compile on win32 as well?

Per the README on the github page, instructions for building ag for Windows are here:

https://github.com/ggreer/the_silver_searcher/wiki/Windows

The author forewarns that "[i]t's complicated".

Since I added pthreads, there's no chance that it builds on Windows anymore. I don't have a Windows machine or VM to test stuff out on. Patches welcome, though!

Looks good, but from the doc I can't tell if it supports the second most useful feature of ack, that is scoped search:

    ack --ruby --js foo_bar

will search only ruby and javascript files, which means .rb+.erb+.rhtml+.js+...

Also exclusion with --no-* is very useful (especially --no-sql).

This is markedly different from 'simply' ignoring irrelevant files, besides the fact that it does not need a 'project' to work (ack --ruby foo_func $(bundle show bar_gem)).

The better part being it is extendable so that I can create --stylesheets covering css+sass+scss+less, or add say .builder to --ruby.

(BTW, love the name/command)

Did you benchmark read() vs mmap()? Most tools seem to go with read() for grep-like io patterns.

In fact looks like GNU grep has --mmap switch and it's a little bit faster in the simple case than default on my Ubuntu system. But -i makes mmap slower. Maybe GNU grep just avoids mmap because of error handling (you get a segfault/bus error instead of an io error return when things go wrong).

Feel free to correct me as I am too lazy to test this, but I do not think that is the most fair comparison. I would do

  ack -i mbr > /dev/null

I think that starts up perl once, not once per file. If so timing should be much better.

ack searches recursively by default; I don't think it can search non-recursively (why would you want to? That is what grep is for)

Also: try comparing grep and ack in a directory tree that has 'garbage' such as .svn or .git directories or .o files.

No, it's not a fair comparison.

For every single file found you start ack again. You compare startup times here. Ack is so slow here because it's a perl script. For every single file you start the perl interpreter, and the perl interpreter compiles and interpretes ack every time.

It is totally not fair that perl has a slow startup time and doesn't run as a daemon. Grep is cheating by not pulling in a huge runtime.

I think the point stands.

First, without knowing the makeup of files he has, you can't tell how much a corner case this is. It could be 100K small files or 10 large ones. Few care about runtimes for small files, but many care about runtimes for large ones.

Also, and probably more importantly, you'd use ack differently in a recursive-find situation. You just "ack" from the top of the tree. The perl interpreter starts only once.

I don't think this is a useful benchmark for typical uses of ack.

For myself I usually use this function below that is useful if you do a lot of code grep.

    function g! { grep -nr --exclude=.git --exclude=.hg --exclude=.svn --include="*.$1" "$2" ${3-"."} --colour; }

And then:

    g! py "some python code"

https://coderwall.com/p/uhzc0a

You can tweak some git grep config settings and get the ack UI and because it's git grep it's got most of the code conveniences too as well as the speed.

http://travisjeffery.com/b/2012/02/search-a-git-repo-like-a-...

Hey! Glad you found my post useful!

So did I. It's excellent.

Now, if we need something 3x-5x faster than Ack... https://github.com/ggreer/the_silver_searcher

This comes highly recommended.

> It highlights the selected text,

    grep --color

> it shows which files, and in what line the text was found (and uses vivid colors so you can distinguish them easily),

    grep -rn --color pattern ./files/
    files/foo.sh:123:    echo "Look at the floral pattern on this dress!"

> ignores .git and .hg directories (among others, that shouldn't be searched) by default,

    git --exclude=.git --exclude=.hg --exclude=.svn

> you can tell it to search, for example for only `--cpp` or `--objc` or `--ruby` or `--text` files (with a flag, not a filename pattern),

You would use `find` in conjunction with `grep`. "Art of Unix Programming", modularity, and all that jazz. Presumably you would just modify your own grep alias or define a function to avoid retyping. The end result pretty much looks like my grep alias:

    alias grep='grep -Ein --color --exclude=.git --exclude=.hg --exclude=.svn'

I still fail to see a reason to use ack, especially when I can assume grep is always available for portability.

...and many many other neat features that I'm sure grep has, but you have to remember and memorize them. ack has sensible defaults.

That's why.

For a similar but much faster tool than ack, which simply wraps `find` and `grep` in the UNIX tradition, see:

http://www.pixelbeat.org/scripts/findrepo

> doesn't require admin privileges to install.

To be fair, neither does GNU Grep - just do `make' (without `make install') and you're good to go.

Not trying to start a war here either, but is it faster?

It's faster than BSD grep, but (in my experience) slightly slower than GNU grep. The productivity boost though, is enormous.

Reply on news.ycombinator.com