Composer: NPM-like package management for PHP | Hacker News Reader

Composer: NPM-like package management for PHP

bergie.iki.fi

75 pointsbergie39 comments

I am one of the guys behind http://pearfarm.org. I mention it just to show I've thought a lot about this problem.

What's interesting is that we all know something like this needs to happen for the PHP community, but the community isn't stepping up and rallying around one. Maybe being sucked into the Symfony world will help it get traction, but I am not sure, only because history has not been on the side of progress of this issue in the PHP community.

PEARFarm hasn't caught on too much (though we use it extensively in-house).

Composer works almost identically to http://pearhub.org, and didn't catch on, either.

Beyond that, there have been more (see phark, etc).=

As for feedback on your idea, my concerns are: 1) Support for legacy PEAR packages doesn't seem to exist. There is a lot of code out there that people already depend on. 2) PHP 5.3 only, while it is the future, still knocks down the addressable market to maybe 1/4 of the PHP world. 3) Does your packaging system support installation of executables? 4) Is there a bundler-style solution? I need to be able to easily install all dependencies in a sandbox. Committing an enormous vendor/ dir is an absurd practice. 5) Does Composer allow the package to not include all files in the git repo?

That said, I would love for there to be a standard, widely-used, useful, easy & practical way to manifest dependencies and install them into a sandbox. I waste way too much time managing this on my own.

1) It does support PEAR, it's listed on http://packagist.org/about-composer

2) PHP5.3, well, so be it. The people stuck in the past are usually not the ones willing to take the plunge into such solution anyway. They can join us when they are ready.

3) Not yet, but we do have some plans for that.

4) You shouldn't commit your vendor dir, you should commit the composer.lock file, which allows anyone to re-install exactly the same version of the dependencies anywhere. And yes this is similar to Bundler as far as I understand it.

5) We also have plans for that, see https://github.com/composer/composer/issues/57 - but this depends on other features that we need to finish first.

1) Ah ok, didn't see that.

Do you PEAR-install PEAR packages? It's important for when there are executables. We wrote another project called comice to make it easy to create PEAR sandboxes and work with them: https://github.com/ardell/comice

2) We will move to 5.3 ourselves soon, I am just telling you the reality. PHP 5.3 broke some BC stuff, making it risky to migrate. I think that's slowed adoption a lot.

3) Not a huge problem that it doesn't install executables, so long as you support legacy PEAR installs with executables (see #1).

4) Cool, I will have to look into that.

See the easy for developers part on why I think this has better chances than PEAR did.

PEARFarm, pearhub, and phark all made it easy to create packages, so I don't think this is going to be the silver bullet to broad adoption.

I've looked through this code - I have several such needs for packaging PHP dependencies - and have decided against using it for several reasons:

- Like most code coming from the Symfony community, it is built Java-style. Not a bad thing, but many things, such as dependency checking or version management, can be done in much simpler ways with less code

- PHP 5.3 only. Thanks, but no thanks. I know everyone is in a rush to promote 5.3, but I feel as though it can be done in such a way as to gracefully degrade and support older versions of PHP quite easily.

- Has dependencies on Symfony projects. That is a great way to have other frameworks adopt your tools! I'm sure this is going away, especially if they want framework-wide support.

That said, I'm jealous. I've been working on something similar for a few months in private and this is already on Github. If they can iron out my above issues, I'll be happy.

- Java-style. You don't like it, I'd argue it's not Java-style, but I know what you mean. I don't think this is worth debating much, it's just opinions :)

- PHP 5.3. Yes. 5.2 has been EOL for almost 1 year now (http://www.php.net/archive/2010.php#id2010-12-16-1), 5.4 is coming fast. I don't mean to be harsh but you will have to get over this one. ZF2, Symfony2, Flow3 are built on 5.3. Drupal8 will be too. Typo3 v5 will be.

- Dependencies on Symfony Components. We are trying to build a package manager here. Something that shold help and promote code re-use, and you would like us to avoid re-using code in the process? I think this is missing the point. Also, you have to realize that most end users will never have to see that, since they will only interact with the "binary" composer.phar file.

I sincerely hope you join the effort.

- Yeah, I just don't like it. But it's a minor niggle.

- Sorry, coming from the CakePHP world, we just migrated to PHP5.2. And yes, using < PHP5.2 is common enough - fucking CentOS - that we should still support it. I also work with Lithium, which embraces 5.3 as much as it can, but I don't see the need for doing so in all cases. So yeah, you can be harsh, but I'll be realistic.

- If all I'm working with is the phar, sure. But if we downport to 5.2, nay. Also, PHP frameworks tend to not depend upon each other's libraries too often. I don't see your CLI implementation being pulled into an official Zend package manager for example.

I'm definitely following along on Github, and have been for quite a while.

Why would it matter how Composer itself is built? It is just a tool you use to manage your dependencies and install your project. And all of it comes as a single phar.

  $ wget http://getcomposer.org/composer.phar
  $ php composer.phar install

Done.

It matters because if you're still on a 5.2 platform you cannot run composer without having php 5.2 and 5.3 installed side-by-side, which is a pain.

It's an aesthetics issue. Not an important one, but I'm personally just not a fan of it.

Pretty nice idea, be good to see if it gets some traction and becomes popular. I suppose the next step is populating it with the most popular packages and trying to get active adoption with developers.

It is in the process of being integrated in the Symfony2 framework, which is one of the largest PHP communities. That should dramatically help adoption.

We have some other plans that will help, but nothing sure. Just talking to various people.

Ultimately though, package management is solving a big pain point. If the user experience is good, and it is easy enough for everyone to use, I would dare say that it can not fail. And don't get me wrong, things are not perfect yet on that front, but we are trying hard, and we welcome any feedback we can get.

Why are you using JSON, exactly? What's the harm in allowing comments in the package description file?

YAML is much better suited IMO, and it has traction.

I really don't think you'd need comments in the package file, but that's not why we picked JSON.

JSON was nice and easy because it's provided natively by PHP, unlike Yaml. Also we were inspired quite a bit by NPM, and I guess that played a role, implicitly.

Also, it is a fact that every format has haters, and we couldn't have picked something that makes everyone happy. So we picked one, and I'm sure people will be able to live with it.

Now I do recognize the fact that some people may not be used to JSON (though I find it quite awkward - even narrow-minded - for web devs), and for that reason I added some syntax checks to warn users of common mistakes in their JSON file, because while PHP has native support, all the info you get from it for malformed JSON is a null return value and "Syntax error" in json_last_error(). We also plan to have an init command that will allow you to build a package file interactively.

> I really don't think you'd need comments in the package file, but that's not why we picked JSON.

What's the harm in letting me decide whether to add comments? Also it's the first place users often look; why not allow me to add useful information? Also, comments have been a big part of programming for decades, and not allowing them without a suitable replacement for annotations is a big step backwards.

> JSON was nice and easy because it's provided natively by PHP, unlike Yaml. Also we were inspired quite a bit by NPM, and I guess that played a role, implicitly.

Yaml can be done in pure PHP so it should not be hard to embed a YAML parser, and I think it would be worth it, to provide something that programmers have taken for granted for decades. Also yes NPM is successful but that doesn't justify copying bad design decisions from it.

> Also, it is a fact that every format has haters, and we couldn't have picked something that makes everyone happy. So we picked one, and I'm sure people will be able to live with it.

This one's easy. Ignore the haters.

> Now I do recognize the fact that some people may not be used to JSON (though I find it quite awkward - even narrow-minded - for web devs), and for that reason I added some syntax checks to warn users of common mistakes in their JSON file, because while PHP has native support, all the info you get from it for malformed JSON is a null return value and "Syntax error" in json_last_error(). We also plan to have an init command that will allow you to build a package file interactively.

With JSON you get experience from it but still trip over it all of the time. Libraries choke on trailing commas, as well they should, because it's a serialization format, not an editing format! Using JSON as an editing format is wrong. Here's a pull request I sent about a trailing comma issue that prevents a repo with 176 watchers from working. https://github.com/atmos/smeagol/pull/15/files

...but you're not creating a new a bad design decision, you're copying a bad design decision, that was copied from somewhere else. And I commend you for coming up with this, as I think the pros outweigh the cons.

"Also it's the first place users often look" — err, no, the package file is for computers, the README for users.

OK, one of the first places users often look. It contains information that's not commonly found in the README, such as dependencies. And I often add comments to dependencies when working with Ruby On Rails, which fortunately uses ruby for its dependency file.

so… for npm, the javascript package manager, using json was a bad design decision?

Yes. Dependency information and script commands are there. Both ought to be able to have comments. Node.js has the ability to load JavaScript, and since it's an installed environment, YAML isn't too much overhead. Both support comments. It can be serialized to JSON if needed.

It can be used more appropriately as a serialization and storage format with npm. It would be a matter of having a script for pushing a package that generates a package.json file from javascript or YAML, and adding it to .gitignore (but not .npmignore) so people see the file with comments enabled.

Given that the parent comment says this is being integrated into Symfony2 (if I read it right), it seems strange to introduce another config format to handle it.

As if a mixture of PHP arrays, YML, annotated comments and XML wasn't quite confusing enough.

Not that I have a problem with JSON itself. It's my serialisation format of choice in PHP as the extra verbosity of its native serialisation seems unnecessary given the lenience of the language.

I actually like having config files as PHP - that way you can not only include comments but hack in logic if you ever have to.

PHP config files are much like PHP templates. It sounds handy to be able to do the odd quick hack in there, but on large scale project it always ends up as a gigantic mess in my experience. Quick hack over quick hack, it slowly turns into an unreadable maze of possibilities.

That being said, for packages I really don't think this applies, because it's static declarative information, I can't imagine why you would need "quick hacks" adding logic to that.

We used to do that with Midgard. Just like embedding in logic into your templates, it gives you a lot of flexibility but ultimately ends up with you or your users shooting yourself into the foot.

For reference, here is one of the old, PHP-based config files we have: http://trac.midgard-project.org/browser/branches/ragnaroek/m...

Using homogeneous architecture has tremendous advantages over using a patchwork of different formats and languages.

Quick example. If you're using native PHP templates and configs, installing APC gives you instant template and config caching with zero effort. To get the same functionality for other formats you would need to roll our your own caching, introducing bugs and complexity along the way, but, most importantly, reinventing the wheel.

That's just one out of myriad examples.

Also, if you have more than two nested levels of arrays (regardless of the language used) in a giant uber-config file, you are the one who is shooting your users into their feet.

Don't code XML in PHP. Refactor. Use includes (which JSON doesn't have, BTW). Use autoconfig pattern and align your configuration file with classes that use them. Use sensible conventions over configuration. Start the file with return statement instead of magic name:

  <?php
   return array(
     'var' => "val",
   );

Nothing will ever scale if you don't even try to design it in a sensible way.

I didn't know that you could return a value from an include like that, that's incredibly handy (and makes me wonder why almost nobody does it that way).

I love this strategy, it's great for config. For example in the RocketSled microframework I mentioned earlier[1] you have a config file to instruct the system which class to execute by default in which you just put:

<?php return 'Hello World';

Being able to include a file like that in your project which returns a value has the advantage (in my opinion anyway) of not cluttering up the namespace with lots of defines(), of not requiring any magic naming convention, and of not having to call out to any external functions or edit config files that are part of the core system's version control.

[1]https://github.com/iaindooley/RocketSled

Admittedly, the example above is quite convoluted. Most of the config files looked like this, which is almost what JSON could look like if it was based on PHP syntax instead of JavaScript:

http://trac.midgard-project.org/browser/branches/ragnaroek/m...

Compared to other formats, it looks like it's missing braces at the top and bottom of the file. YAML gets around it by not requiring trailing commas and not reminding people too much of another format.

Jesus you expected a human to produce that file??? No wonder they shot themselves in the foot! My solution to that wouldn't be a different config format, but less config.

There are advantages and disadvantages. I recently moved to JSON config files for my projects, because they are easy to update from inside the app and because they can be easily validated.

One small, unimportant and low priority suggestion would be to change the layout of the browse page (http://packagist.org/packages/) to a more "app store" type layout. I think something like the ubuntu software center or new chrome web store would really look nice.

Obviously i understand that these codebases dont come with fancy artwork but most have a logo at least that can be used.

This looks really cool. I've recently been similarly inspired by watching Node.js guys (Substack in particular) and the penchant for very targeted re-usable components.

The strength of PHP is that there's so much code already written that monolithic frameworks are a step backwards, but a good package distribution system is required for this to work.

I recently released a little front controller style microframework that has a default autoloader with a simple way to register your own:

https://github.com/iaindooley/RocketSled

and an associated package manager that basically just has a list of github repos and a config file which specifies dependencies:

https://github.com/iaindooley/RocketPack

Since I don't want to constrain myself to using only namespaced code or one particular autoload implementation I won't be using Composer just yet (indeed I'm an advocate of a "lazy" namespacing policy) but I look forward to following the project's progress via github.

My goal is to use tools that don't "have an opinion" - that allow me to use whatever is going to get the job done most quickly. Things like coding standards, policies and aesthetics are moot when it comes to delivering code on the day, especially if your code is highly modular and compartmentalised.

Regarding Substack code, I've made a PHP implementation of the DNode protocol: https://github.com/bergie/dnode-php

With this you could actually call Node.js code from your PHP apps.

And yeah, php-dnode is available on Composer :-) http://packagist.org/packages/dnode/dnode

Oh cool! I was interested in dnode-php, you've already done all the work :)

/me follows

This is neat :)

I spent a lot of years in php, and the lack of a serious code-sharing story was certainly a pain point.

Using JSON (and being strict about it) is a good idea. If people really want to use Yaml, then they can write a composer.yaml, and have a build step that dumps it into composer.json. Courier does this for node projects that want to write their package metadata in coffeescript.

I wish the same were available for python. I know there's pip/virtualenv, but they just don't come close to node's npm or ruby's bundler.

git submodules can handle the basics of this decently well.

You can include any git-hosted project pretty easily with `git submodule add`

You can easily switch which commit/branch/version of the project you want to have included. So if you want to be using the latest version of Project X + one special bugfix that you made, just fork the project on GitHub and switch your submodule to point at your latest commit.

`git submodule update --init --recursive` to update all the package/submodules in your project at once.

Interesting, how does it compare to PEAR?

Reply on news.ycombinator.com