Phan by Etsy – A Static Analyzer for PHP 7+ | Hacker News Reader

Phan by Etsy – A Static Analyzer for PHP 7+

github.com

113 pointsnikolay60 comments

This has been written exclusively with PHP 7 in mind and must have taken a reasonable amount of thought and a lot of hard work.

The source code looks great, and I could see this supplanting some of the other tools out there for PHP static analysis.

Co-author here. Happy to answer any questions folks have about Phan.

This is awesome. Any plans to add autocomplete functionality similar to HHVM's typechecker?

We'd like to get there eventually. Phan populates a sqlite database when analyzing code (if run with the -s flag). This allows fast runs on single file changes or patches. We're hoping to use this for things like auto-complete and finding callers for a class/method/whatever eventually.

For now, we have a primordial Syntastic vim plugin (https://github.com/etsy/phan/blob/master/plugins/vim/bundle/...) for getting errors into the quickfix view.

That's awesome. If a PR was opened that added autocomplete and/or finding callers and/or refactoring (and assuming that the code is good), would you guys be open to merging it? Or would it be delayed so that the core functionality can be focused on?

Definitely.

If you want to start working on that, feel free to open up an issue (https://github.com/etsy/phan/issues/new) to start a conversation about it.

We currently store method/function callers in the sqlite database (see https://github.com/etsy/phan/blob/master/src/Phan/Analyze/Br...), but aren't yet exposing an interface for getting callers. We'll want to store callers for constants, classes, properties and global variables as well.

Meanwhile, have you checked padawan ? https://github.com/mkusher/padawan.php it's a server made to be used by editors for autocompletion of PHP code (like jedi for python) it works pretty well with vim AFAIK

Yep. https://github.com/mkusher/padawan.php/issues/1

Project started after mine. https://github.com/cweagans/theforce

Just ran out of time, so mkusher did his own thing. I disagree with his direction and I think it's overly complex from a user's standpoint, but he disagrees and isn't willing to change it, so I lost interest in that project too.

Using the actual AST from the PHP interpreter is going to be a) much faster and b) much more accurate.

How Phan treat SIMD in PHP7? Does it pick up what is going on with functions as variables?

Phan does not attempt to make sense of things like `$class_name->$function_name()` or `$$v`. I'd be happy to see a pull request that could handle simple strings, but otherwise I imagine the best we can do is emit a warning.

I'm not sure what SIMD refers to. Can you clarify that?

> I'm not sure what SIMD refers to. Can you clarify that?

With a name like "asm" that comes as a surprise... Single Instruction Multiple Data[0] refers to low level processor instructions that are able to work across a range of data, e.g. video cards working on changing the values in a framebuffer with one instruction mapping across an entire buffer in parallel.

[0] https://en.wikipedia.org/wiki/SIMD

That doesn't clarify what the OP meant. PHP does not allow you to issue processor instructions directly.

I'm going to take a crazy leap here based on the commits. asm might not stand for what you think.

What about `$$$v`? Does it handle `$$$v`?

I wrote this when I had just woke up as I was excited, so I had to get a question in. I think what I meant was passing functions into other functions.

I do this for things like:

$user_requirements = [filter1, filter2];

foreach ($user_requirements as $test) { if (!$test($some_data)) { return false; } return true; }

Is there any way to tell the analyzer to make sure that afunction of n arguments is passed in?

Awesome project, Do you know where I can download a php_ast.dll for the latest php 7 for windows?

I'm not sure. PHP7 is pretty new so package maintainers may not have had a chance to put binary releases together. You may need to compile it yourself from https://github.com/nikic/php-ast.

How is performance of Phan? Nested looping checks in phase 2 scans sound intensive and bottleneck-ish.

Analysis takes a lot of CPU and we're not doing any parallel processing. Phan itself is about 30k lines of code and takes about 1.5s for a clean run.

That being said, you should take a look at the '-s' flag for saving state to a sqlite database. After the initial run the stored state can be used to only scan changed files. For a patch with a few files changed it should be about 0.5s to run the analysis.

Am I right in assuming that the name is a tribute to John Allspaw's rec.music.phish posts from the 90s?

https://groups.google.com/forum/#!search/allspaw$20phriend%7...

Clarification on the title of this post; Phan is written in PHP7, but will analyze code written in any version of PHP.

As long as that code can run on PHP7.

PHP gets so much love and money shoved at it, the Ruby programmer in me finds it unfair.

That being said, I'm really glad that this helps millions of PHP devs around the world so bravo to the people working on this.

PHP gets "so much love and money shoved at it" because, for a lot of problems, it's the right tool. You can always ask facebook about it.

It wasn't the right tool for Facebook - Python would have been a better alternative, at least once they graduated from shared hosting. It was good enough that migration was never worthwhile, because the technology wasn't the innovative part of Facebook, but that doesn't make it good.

Could you explain your point? Given it's success, I can't see why PHP wasn't the right tool for Facebook.

Just that Python is a better language for solving those kind of problems. I wrote a vaguely-facebook-like webapp in the same timeframe; later I worked (briefly) at last.fm (which uses, or at least used, the facebook stack) and spent most of my time working on better replacements for PHP code and issues that were caused primarily by PHP. If nothing else look at all the tools facebook produces - thrift and HHVM being the most prominent ones - to work around their PHP problems.

PHP was the "right tool" in that it was the language Zuck knew when it was a one-man project. And facebook's success is a reminder that second-tier technologies can be perfectly adequate to build a successful business on, that business concerns often have a lot more impact than technical ones. It shouldn't be taken as an indication that all their technical decisions were right.

Thanks for the answer.

No, the set of problems where PHP is the correct tool is rather small: 1) existing PHP codebase and 2) small webpages in need of little dynamism.

You're mistaking a language for a framework/library.

So does PHP.

> PHP gets so much love and money shoved at it, the Ruby programmer in me finds it unfair.

Popular has little to do with fair, and a lot to do with being in the right place at the right time.

And network effects mean that being popular can make something desirable for some uses even when its intrinsic features aren't superior. And, anyway, while Ruby may sometimes lose to PHP for this reasons, it (due to Rails) often benefits from this, too.

I think this is the first time I've seen PHP accused of getting "so much love"...

I really don't want to sound negative, I'm sure a lot of thought and work went into this, but one has to wonder - would it not be easier to just use static languages instead of writing static analyzers from scratch for every dynamic language out there?

Often the problem being solved by this sort of thing isn't directly about making it easier to work with a language, but making it easier to hire developers. The hiring pool for developers who are very good with PHP is much much larger than a lot of other languages simply by virtue of it being an easy language to get started with.

This is also why Facebook has spent so much time and effort on things like HHVM and Hack.

2 counterpoints:

* Would you hire somebody who wasn't able to learn a new language in a reasonable amount of time?

* Would you let somebody who just started with PHP (by virtue of it being an easy language to start with) work on your backend?

Not counterpoints. The comment above states that the pool of very good PHP developers is larger. This implies easier hiring (capacity to learn a new language is not relevant, because it works both ways), which, among other reasons, would be because they'll be applying first to PHP positions; and secondly, a very good PHP developer is far from somebody who just started.

Let me rephrase. I am proposing that (barring very specific requirements which webdev doesn't justify) there's no such thing as a "good PHP developer". A good developer is a good developer, period.

A good developer will learn a new language in a reasonable amount of time and will be able to be productive and write good code in it.

A PHP-only (or X-only for that matter) developer is, by the definition above, not a good developer.

So why restrict oneself to some random pool of X-only people?

In general, I agree. But getting to mastery level takes time even for really good developers. Certain jobs require master-level application of that language's constructs for efficiency, security, or other reasons.

What would you call a good developer who knew PHP?

A good developer :)

IIRC, one of Facebook's motivations for sticking with PHP was that it was the easiest language for everybody new to learn, meaning that they don't have to consider language expertise when hiring.

FWIW, they have released cool open source projects on Haskell and ocaml, so it's not like they've never considered any alternatives.

My understanding is that Facebook also had a MASSIVE amount of PHP code written. They then ported the performance bits to C++ and then realized they still had a huge amount invested in PHP, such that it wasn't worth it to switch off it entirely.

I also have a small suspicion that they wanted to stick to their roots a bit. But thats just me.

Imo: the only hard thing in webdev are handling the complexity of scaling and legacy.

In many cases the latter forces you to compromises in what you pick and the first one in compromises when you pick it.

I fully understand when people try to make work what they have.

Eg PHP by itself is a very fast powerful language. The tooling and several aspects (dynamic type etc) are exhausting for larger code bases but this is something which people inhouse can solve without requesting the full company to switch.

If you already have a codebase in a dynamic language, it can be incredibly hard to justify the cost of rewriting (which will likely pause feature development until the rewrite reaches feature parity, etc.) v. the cost of employing a small number of people to improve tooling of your current language. Or even forking your current language (as FB have done with HHVM and the introduction of Hack).

I don't know, how easy would it be to rewrite the entire Etsy stack in a new, static language? My instinct says it would be "not trivial".

...no?

Static analysis has nothing to do with typing. Static analysis is about interpreted code, where you lack the luxury of compiler warnings.

Static analysis doesn't have to involve typing, but it is only about a program reviewing your code for potential issues. Whether or not it is interpreted isn't important.

You make it sound like people don't implement static analyzers from scratch for every static language out there. Static analysis is not obsoleted by using a "static" language. You can't even say that static analysis is easier in a static language (the problems you deal with are just different).

Every language with a non-trivial number of users tends to have a couple (or maybe couple dozen) different static analyzers of varying quality. This is one of the PHP static analyzers.

> You make it sound like people don't implement static analyzers from scratch for every static language out there.

Yes, except those static analyzers are the compilers for those languages and the rules that define crappy programs are in the type system, not in ad-hoc specifications of patterns to identify for static analyzers...

Static analyzers for different languages detect different types of errors. A static analyzer for PHP may try to find type mismatches. For statically typed languages that's obviously not relevant, as the compiler already handles it. On the other hand, a static analyzer for C may try to find use-after-free bugs. For PHP that in return is not relevant, as the compiler/runtime handles it already.

While some compilers do ship with static analysis components (e.g. Clang), static analyzers are usually distinct from compilers. Doesn't matter it for static or dynamic languages.

Type checking is static analysis. Languages like C have crappy type systems, so the type checking static analysis is also crappy. This is a property of the type system, not the universe...

> Yes, except those static analyzers are the compilers for those languages

The first linting tool was made in 1979 for C programs and I remember ads for PC-Lint in the back of programming magazines all the time growing up. Static analysis tools catch more than just datatype errors. They look for patterns that identity common logic errors as well. PC-Lint used to cost a couple hundred dollars, and people were plenty willing to pay that much for extra peace of mind.

What if your type system was rich enough that a "datatype error" was a logic error?

That can't prevent every logic error. Here's[0] something I remember reading that really impressed me with static analysis tools. Running PVS-Studio against the Unreal 4 Engine (which is entirely C++).

It was able to detect a surprising number of logic errors, probably just caused by as tired programmer, or copy-pasting too quickly. For example in the function below, it was able to spot that the last 2 clauses of that comparison are identical. The programmer meant to use a '+' instead of '-' in the last one. I don't think you could have a type system that would disallow this logic error. (If it is possible, please prove me wrong, because that sounds awesome).

  static bool PositionIsInside(....)
  {
    return
      Position.X >= Control.Center.X - BoxSize.X * 0.5f &&
      Position.X <= Control.Center.X + BoxSize.X * 0.5f &&
      Position.Y >= Control.Center.Y - BoxSize.Y * 0.5f &&
      Position.Y >= Control.Center.Y - BoxSize.Y * 0.5f;
  }

[0] http://www.viva64.com/en/b/0249/

PHP has some degree of static/gradual typing actually, so there's that.

I'm guessing PHP still dominates shared hosting but would anyone ever use it on AWS, DO, Heroku, own servers, etc?

Yes.

Yes. Facebook is PHP, as are Etsy, and there's no way they're the only ones.

Heroku, DO, and AWS all fully support PHP, which they wouldn't bother doing if there weren't a market.

Sure, why not? You can still write good applications using PHP. It not just for legacy servers and pre-installed applications (wordpress/drupal). You don't even have to use [framework of the month]. Its just a tool. Good architecture, good practices, and good code are not syntax dependent.

Reply on news.ycombinator.com