Show HN: Grep with colours written in Go
github.com
github.com
And the color themes for the terminal are godawful. No matter how you spin 16 colors, how solarized or other, everyone is kind of chained more or less to that attrocious 16 color pallet, which is always going to be way way higher contrast or low-fi than something like a vim theme that can pick some complementary colors to work with.
Colors feel like my terminal punching me in the face. No thanks.
Also have you ever logged into ubuntu? Holy shit colors were a TERRIBLE idea.
Generally though, I'm not out to point fingers, it's just a fact that not all software is complete and bug-free. We're all people with limited time and we don't have the attention to detail that dealing with the maliciously compliant children that computers are takes.
What I think would be good (for me at least) is not colors, but sort of markup, like bold or underline, which is available in terminal as seen in man pages and PS1. I usually make them bold and compatible colored.
Another bonus is that I don't feel cheated when I log into systems that don't support colors (*BSD, Solaris, etc). Everything just looks normal to me. Even non-syntax highlighted vi (not vim).
If your terminal has a background very close to one of the colors on that palette, that is a theme problem, not an app problem.
So it is possible for tools to choose colours by an exact RGB value, but typically they wont.
The other thing to remember is some tools can have their colours remapped. Even tools people might not expect to be able to. eg you can configure (via environmental variables IIRC) `ls` to have specific colours for specific inode types. However in my personal opinion it is a lot of faff for very little reward.
Do you remember the days of X Modelines? I still have the nightmares.
HN discussion: https://news.ycombinator.com/item?id=3717303
Funny enough, I've found that adding syntax highlighting to even normal English text makes it more readable. I sometimes turn on a random language's syntax highlighting in e.g. Notepad++ even if the choice of colouring is basically nonsense.
In the article you link, apart from the green on grey, the little example paragraph is actually very nice and readable.
Something like http://www.beelinereader.com/individual would probably make things easier for me, though it costs money.
The other issue is that it is too uniform. I like how syntax highlighting tends to make random patterns which kind of looks like pictures in the code - it makes it much easier for me to navigate, and figure out where I was and where I'm going.
This depends entirely on the content of the text. Stories, fiction, non-fiction, reference material ... information where one human is intending to communicate with another human should limit use of color.
For source code, that humans do read but is intended to be parsed and compiled by a computer, I'm not assembling (in my head) the whimsical trials of a protagonist - I need to see structure; color gives me a quicker overview of the structure of a line, function, class, etc.
First: It requires that every command line tool add support for this environment variable. It may not be much effort for each project, but it adds up to a ton of developer time. Even if this standard gains traction, some apps will slip through the cracks. So in the success case, many users will still be annoyed that one or two of their tools print colors when they shouldn't.
Second: As the FAQ on that page mentions, people can configure their terminal emulators to squelch color. NO_COLOR only matters for users who want color in some applications and not others. In that case, are certain applications supposed to refuse supporting NO_COLOR? Will all users want the same set of programs to behave that way? Are users supposed to unset NO_COLOR before running those programs?
At that point, it seems like a more complex version of using some aliases that add --color=never (or whatever the appropriate flag is). Or if the user prefers to disable color by default, they could set $TERM to "xterm-old" and alias their favorite commands to add --color=always. Either way, there's no need for another environment variable.
Really though, this seems like it should be a feature of the shell or the terminal emulator. Does the process name match certain patterns? Pass color codes through. No? Strip them. That would solve the problem for all programs past, present, and future. And total development effort would likely be less than that needed to implement NO_COLORS in every command line tool.
I think it's apparent that this isn't something you're really keen to support in ag in the few times I've seen you talk about it, so feel free to close the PR I have open there rather than leave it hanging without any reply.
Your use case seems perfectly valid to me, but how would NO_COLOR help you? Wouldn't it suppress all color in all programs when it's set? Assuming everyone gets on board, of course.
He can then selectively enable color on the programs he wants by simply unsettling the NO_COLOR variable before execution of the program.
Edit: missing not
Another workaround is to merely pipe the output to a file in certain cases.
If that works for a certain tool, it's author really has no excuse to not implement NO_COLOR env as well.
The program checks if the output is a tty, if it is not a tty it will drop color information as it is either a pipe or a file. The escape codes would look ugly in a file and could get in your way with some text processing programs.
Even if the program did drop color when not sending output to a tty it still is a crappy way to remove color as the pipe would need another program as a receiver such as pager.
You don't need a pager. cat works. Compare
grep --color=auto hacker /usr/share/dict/words
with grep --color=auto hacker /usr/share/dict/words | cat
The same applies to GNU ls, which actually changes more than just colors depending on whether it's printing to a tty or not.Programs that emit colors by default when not sending output to a tty are buggy.
A wild card in this mix is Windows. On UNIX-like systems, detecting a tty is simple. On Windows, it is... not so simple. But the OP's tool, blush, doesn't appear to support Windows at all anyway. (Which is totally cool. I very much understand why you might not.)
I guess the program implemented the tty method you could always alias your stuff to something like blush="/bin/blush | cat" if you decided you did not like color.
While a part of me says piping programs to get the result you want is the *nix way. Another part of me feels like it would just be a dirty hack to avoid color.
^ I have not run the above program, only proven it to be correct.
Are there xterm codes for generating terminal color that don't end in m? It handily covers 8/88/256/24bit. What else is needed?
* Personally I quite like the bold and reset SGRs even when I don't want the term colourised. Bold can highlight text without drawing your attention away from the main body of the terminal.
"\033]11;#53186f\007"
So, bare minimum, I'll need to include # in future. you can /generate/ 24 bit color using only ; and digits...
When filtering ANSI I'm usually streaming the result to something that expects pure ASCII, and tuning sed to only accept two escape codes would be a pain in the neck.
Sadly handling terminal escape codes is one of those impossibly painful jobs. Heck, often even the terminals don't seem to support the escape codes they have documented or the documentation is so poor that you're basically left with trial and error (or a shit load of hex dumps to trawl through if you're lucky). eg I've wasted hours trying to get Kitty specific escape codes working on that terminal emulator before giving up.
I'm investing a lot of effort at the moment learning escape codes because I'm writing a new $SHELL which is designed to have rich media support even in $TERM's which don't support rich content. But I do still have support to convert the $SHELL back into a black and white console (I even have an option to strip colour from the STDOUT (even when pipelined) of all processes negating the need for your sed command. However that feature hasn't yet been committed back to the master branch.
For example, yellow text on a dull yellow background? No thanks.
For years I've just wrapped commands on bash to strip the colour control-codes from stdout.
function monochrome() { "$@" | sed -r "s/[[:cntrl:]]\[[0-9]{1,3}m//g" }
This feels like complaining that Spotify doesn't have a silent mode for people who dislike sound. /shrug
I think there are FAR fewer people who want it off everywhere by default than people who want it off occasionally
having all the arguments to your app be handling via dash args EXCEPT the one for color is non-standard, harder to explain in the docs, introduces an inconsistency to how your app is controlled, etc., etc...
A good place to start would be this: why GNU grep is fast[1] - Starting with the Boyer-Moore string search algorithm and reading through the optimizations done in GNU grep.
p.s. there's an implementation of Boyer-Moore hiding in Go's standard library.
[1] https://lists.freebsd.org/pipermail/freebsd-current/2010-Aug...
In Go-land, you should be able to replace uses of memchr with IndexByte[1], which should be implemented in Assembly on most platforms.
Of course, for any of this to have a big impact, you'll want to take Mike Haertel's advice on avoiding line breaking and stop using bufio.Scanner. :-)
There are a couple of places I wish I would have done better. Using bufio.Scanner actually bothers me a lot. Also in the Read() method it reads everything from all readers into a buffer instead of pulling what it needs to check.
Thanks for suggestions :)
I know ripgrep has a ton of fantastic optimizations by Burntsushi.
You might wanna check it out...before making such statements.
Plus the next time you make sweeping generalisations about the reading comprehension abilities of HN it is probably worth remembering that this is an international community and thus English isn't going to be everyone's first language.
I suppose in that sense it does aim to be fast. Fast for the human to parse.
Here's an excellent write-up on how it works, benchmarks, etc.: https://blog.burntsushi.net/ripgrep/
My desktop is running ubuntu-18.04, is an i5-3570, and has a fairly quick intel SSD.
Running "blush -R -i FunctionName ." takes 15.090 seconds and finds two files.
Running "ag -i FunctionName", finds one file, missing one in .clang-format.
Running "ag -i -u FunctionName", finds two files and makes 0.64 seconds.
So somewhere around 20-25x faster.
I'll give it a try. I normally use a different Golang tool called sift as my grep replacement (which I love so far): https://github.com/svent/sift
Sift's goals seem to be mostly performance (it is super fast), but it would be nice to have some of these more sophisticated coloring features in there as well, as they are useful.
Yes, 'cat FILENAME | blush "some text"' and 'blush "some text" < FILENAME' do the same thing. But, what if you don't have permission to read the file - the former be re-written as 'sudo cat FILENAME | blush "some text"' - the latter form can't. What if you want to build a pipeline? I think its pretty persuasive that 'cat FILENAME | blush "some text" | sort' reads better than 'blush "some text" < FILENAME | sort' - the former reads from left-to-right, the latter reads from the middle, to the left, and then bounces over to the right. Tastes may very - but, I think its a hard sell that such an opinion is clearly wrong.
So, yes, its unnecessary. And, yes, in a script using cat like that can complicate error handling. But, for interactive use, what advice exactly are you trying to convey?
< FILENAME blush "some text"
My actual (light) advice is merely that we're all guilty of inappropriately using IO redirection facilities that punish the performance of our shells. `cat <filename> | grep <expression>` should be replaced by `grep <expression> <filename>`.
No doubt that pipelines are easier to read. The author has a whole section in README demonstrating blush's ability to read STDIN - nothing is lost by using best practices everywhere else. Documentation matters, and it should communicate best practices.