Hacker News Reader

Comment by postfuturist | Hacker News Reader

I ran into a wall when trying to use unix-y tools to do somewhat complex, regex/replace functions for code refactors. Basically, anything inside a single line is easy, but once you cross that line barrier, the complexity increases dramatically. That rendered the changes useless because most programming languages allow you to add arbitrary new-lines between any token in the language.

View on HN

I know what you're saying. I ran into the same problem a while back and ended up hacking a tool that does the kind of structured pattern matching I wanted. Its syntax is a bit awkward, but what the hell - you can find it here if you're interested: https://github.com/nhaehnle/patrex

I also hacked up a tool for structural search. I'd noticed intellij's built-in structural search has a very simple pattern language - you just type the code you're searching for and insert a wildcard like $x$ or $foo$ where an expression can vary. However, its restricted to the languages Intellij can parse.

(in what follows HN is mangling asterisks, so I used 'X')

Tokenizing a language is much simpler though. So I just did that, and let wildcards match like <token>X? up to the next expression boundary at the same nesting level - the boundaries were ',' ';' ')' '}' and ']' when I tried this on java, C and perl.

This turns out to be simple enough and powerful enough to be useful. For instance, your example:

boost::bind(& ${id} $( :: ${id} )+, $( boost::ref( Xthis ) )|ref| $.X )

Would be: boost::bind(& $id1$ :: $id2$, boost::ref( Xthis ) ) (for 2 args - I'd need to repeat for more args like so: boost::bind(& $id1$ :: $id2$, boost::ref( Xthis ), $id3$)

This works because $id1$ is non-greedy, and whitespace is ignored. I did tend to tweak the tool to what I was searching for (I'd have dropped ',' as a delimiter for this one), which I guess is cheating on keeping the syntax simple!

As A sysadmin, I have two general rules about scripting;

1. If it's longer than 15 lines, rewrite it in {python,ruby,perl} 2. Never accidentally rewrite grep, sed or awk.

Reply on news.ycombinator.com