Drupal: 15 years old
buytaert.net
buytaert.net
Thanks, Dries, Jimmy, and John!
Are you talking about their node system ?
I talk about this a lot and I must admit I rarely use words as diplomatic as naive...
That has to be the first time that I'm labeled 'diplomatic'. I have different words as well but I see no upside in using them, those who choose to pick drupal after many years of broken processes, lack of an upgrade path between major versions, abandoning large numbers of users on broken and unsupported code anybody that chooses drupal today really deserves what they get.
Drupal wants end-users (not limited to developers) to be able to create new types of entities, add fields to those entities and store complete revision histories for those entities as they are created and edited. This should be achievable within a GUI and without needing to edit code by hand. It also wants developers to be able to distribute code that give these end users new types of fields to use.
It achieves this by creating two new tables for each additional field. One table stores just the current value of the field, and the other table stores the historical values of the field in previous revisions. An entity like a 'blog post' with three fields will therefore have a base table plus three additional tables for the current values, and a revisions table with three additional tables for the historical values. When loading an entity, these field values can be loaded via SQL joins, or (depending on the field type) via additional queries.
As this is expensive, entities can be cached, and the next attempt to load the entity will pull the complete object from memcached, Redis etc., and this is how most production sites would do it. Obviously in the case of zero caching, where you have entities with many fields (say, 30) and where you need to load many entities, you can quickly end up with many queries - hundreds or even thousands! But almost all of those go away with caching enabled.
Is this a bad design? It would certainly be possible to make the database schema line up better with the entities, which would reduce the number of queries - a single 'blog_posts' table could be queried, rather than hitting the 'node' table and then the additional field tables. But you would lose a lot of the end-user flexibility - the ability to define new entity types, add and remove fields or reconfigure those fields (changing their size or even their cardinality). Instead of adding new tables, you could add and remove columns from an existing table, but this brings other problems - your tables can become very large, and altering tables containing existing data is slow. Mutating a single table when you want to add/remove a field is a more dangerous tool to expose to an end-user than just adding/removing additional tables.
If only developers can do these things, then yes, you would expect to design your tables for optimal performance. There's a large body of knowledge on SQL optimisation that a developer would be able to bring into play. But if this requires giving up the ability of end-users to modify entity types via the GUI, it no longer achieves the original design goals. Drupal trades off best practice database design against delivering certain tools to end-users, and mitigates the performance impact with a fairly robust caching system. To me, that's good design, given the constraints.
You might argue that Drupal's design goals are wrong, or that those design goals make Drupal inappropriate for some use cases, but I don't think any of that makes it a fundamentally bad design.
EDIT: fixed some grammar in the db schema description
This is pretty much the epitome of bad design. It creates a nightmare data model. The only advantage is that you can get away with this in the most brain-dead way possible when adding/removing fields without having to create a proper migration.
The goals are fine, it's the implementation that sucks, nothing in the SQL standard would stop you from doing this properly but drupal purposefully chose to throw away vast amounts of performance and interoperability for their end users in return for a marginal speed up in their own development processes, presumably because they simply couldn't think of a better way of doing this.
It's amateur hour, and drupal is by far the worst CMS out there for these and many other reasons.
That's a pretty big advantage if you're an end-user who doesn't know how to create migrations. Yeah, you wouldn't do it that way if you were writing the code from scratch yourself, but you're not.
The 'nightmare data model' is only really a problem if you think of SQL as your level of abstraction. If you use the entity APIs, the data model looks fine. The whole point is that you shouldn't be poking around in the DB most of the time (I will admit that this is a principle frequently violated).
Entity storage in Drupal is pluggable, meaning that SQL databases are not the only possible storage. Tightly integrating SQL-specific design patterns would break pluggability (though again I would admit that SQL databases are by far the most common storage engines). Maybe making entity storage pluggable might be a bad idea, but it's not obviously bad design.
I'd imagine that other systems which can use SQL databases as dumb storage (Datomic does this, for instance) will probably not produce beautifully optimised tables and queries, because they're doing their optimisations at higher levels of abstraction. Drupal gives up thinking about tables and columns and instead thinks about entities and fields, which is how efficient caching of these entities is possible in the first place.
Sure, you or I could make something much more efficient by giving up a lot of this flexibility, but then you'd be looking at a different product with different design goals.
Such a migration should be created by drupal, not by the end-user in a proper design, you know, like everybody else does it.
> The whole point is that you shouldn't be poking around in the DB most of the time (I will admit that this is a principle frequently violated).
DB's tend to become the place to hook into other systems rather than to create yet-another-dependency on whatever framework is currently powering the front end.
Having a brain-dead schema means two thing: there will be no migration path out of the present CMS without a complete overhaul of the DB and it will make interoperating with other software accessing the same DB all but impossible.
> Sure, you or I could make something much more efficient by giving up a lot of this flexibility, but then you'd be looking at a different product with different design goals.
I don't think drupal had any actual design goals, and I think the lack of experience designing such systems is what lead to this mess, not any possible design goals they might have had.
What is sad is that they seem to re-invent several wheels with each new re-incarnation of drupal but they keep their brain-dead db architecture because they've double down on it so many times that to admit this was an error this late in the game would destroy whatever confidence people may still have in drupal. It's just sad.
Anyway, if you're a new user looking for a CMS do your future self a favor and stay as far away from drupal as you can.
> I don't think drupal had any actual design goals
You dont THINK? Well maybe you should LEARN and then you will KNOW.
Actually Drupal is very good (the best framework I know) for migration and interoperating with other software. But you don't do that by connecting DBs directly like you seems to want to do... (which is wrong with any framework/cms)
I ran a drupal based business for years and I know the platform - and it's shortcomings - very well.
>> I don't think drupal had any actual design goals
> You dont THINK? Well maybe you should LEARN and then you will KNOW.
Well, I could have used stronger language there but I see no need. Because whatever I would have said you'd have found fault with the form rather than with the content. Believe me, I've spent enough time repairing and porting installations of drupal from one release to the next to know for a fact that their design goals are about as flexible as a rubber band and that they fail to deliver on the most basic ones (performance, ease of maintenance, backward compatibility and ease of upgrade).
> Actually Drupal is very good (the best framework I know) for migration and interoperating with other software.
Well, let's say our experiences probably are not identical.
> But you don't do that by connecting DBs directly like you seems to want to do... (which is wrong with any framework/cms)
For better or worse, DBs tend to be a relatively standard item and of course there is a 'right' way to do things (which is different for every platform), but having done several re-builds of large platforms we could probably agree on having a sane database schema and reasonable naming scheme to be a plus in such situations.
So from a pragmatic point of view this 'wrong' way is often the only way to achieve certain goals, and with drupal this way is nailed shut to the point that you are not only locked in to drupal, you are most likely going to end up being locked in to a specific drupal version (due to their modules usually not making it past an upgrade).
FWIW at some point in time I hosted thousands of drupal sites so I think (see, I'm doing it again, so feel free to take another cheap shot at me) that I have enough experience and familiarity with drupal to see it for what it is.
Now admittedly these are not sites with millions of visitors, however several run on a single 2GB RAM VPS. And, yes, there is the problem with version upgrades, which in the past I've handled by a site redesign every several years, and paying someone to handle the data migration / mapping.
From my point of view Drupal 'just works' for me (and several other web guys I know), clients are happy, hosting is cheap, and site performance is fine. In a world where there is always something to do, I still don't intend to look into it further than this, or fret over changing CMS because there is no need to.
And after reading your critics, you seem to really don't understand how Drupal entities work.
I've provided that (1) drupal is inefficient, (2) drupal sucks when it comes to backwards compatibility, (3) that drupal has a totally broken approach when it comes to data base schema design and finally (4) that other CMS's (such as Symfony2 and Laravel, and Yii as well) in the PHP ecosystem show that it is entirely possible to avoid most of these pitfalls.
You say that 'drupal is how CMS's should be done, that's why all others fail' when in fact most of these others are succeeding where drupal is losing ground to all of the above and to the likes of wordpress and joomla (though those have less functionality).
If you feel that drupal is how a CMS should be done then I'm perfectly ok with that but don't make it seem as if that is anything more than your personal view with which I disagree for the above reasons.
And even if you're limited to the PHP eco-system, there are lots of alternatives.
One thing I did not mention, but which I'll be more than happy to comment on now is that another thing that bugged me about the drupal ecosystem is that it seems as if it is altogether too much self-centered, unwilling to absorb best practices from the rest of the industry and a fairly toxic community which tends to be openly hostile to anything and everything that is not 'drupal'. I don't like that particular aspect of drupal one bit and it is this kind of attitude that is the reason why drupal to this date sees such a huge level of abandonment when it comes to modules, past version and large numbers of sites marooned on that code.
It shows a total lack of respect, care and understanding of the kind of issues that the users of these systems have to deal with.
I understand perfectly well how drupal maps its external interface to it's data model and when I tell you that this approach is broken beyond repair it is not my understanding that is deficient, it is drupal, when seen against the backdrop of properly designed software, of which we -fortunately- have plenty of examples.
If you wish to ignore all that that is entirely your freedom, but recognize that this is akin to a religious point of view, you support it because you are invested in it rather than because you are aware of the alternatives.
A good data model and the cruft that drupal produces are not even on the same scale when it comes to maintainability and efficiency, if you feel like arguing that point with actual facts and data to back it up I'd be most interested but I fear you'll end up coming short or having to admit that maybe the drupal way isn't all that good after all.
2. Drupal 8 is built on top of Symfony2
3. Saying Drupal is self centered show that you don't know Drupal community (They team up with Symfony and follow now many Symfony paradigm for example...)
4. Sorry but saying Drupal has a totally broken data model is very exaggerate, you may not like it, you can have critics, but it's a perfectly valid model, which works perfectly.
5. You don't provide much information either...
6. All cms/framework with a large module ecosystem have trouble with backward compatibilty, Drupal is not the worst. For example WP have trouble with compatibily between module for the same version of WP. Drupal don't have this issue because their guidelines are strict.
7. Drupal is loosing ground to Joomla ? For someone with high standard, it's funny that you mention Joomla and WP as better replacement to Drupal. Very funny. Because, yes, I know these alternatives and they are really poorly designed and not just the data model (I dont know why you said I don't know alternatives...)
With all my love and respect.
Now, please tell us how this one would be an improvement over drupal for the newbie user.
- Loose coupling. Drupal has been becoming increasingly API-centric and storage engine agnostic. Focusing on the physical storage schema for a DB engine misses the point of the architecture.
- Performance(!) Supports schema changes on existing large datasets (migrations are only starting to appear in Drupal 8, and elsewhere bring their own complexities when you want zero downtime). Unsurprisingly there was some gnashing of teeth when this 'table per field' approach was introduced and as a result it was decided to build denormalized read layers on top (materialised views and bundles). However, it turned out that they showed no perf improvement over Drupal's object caching whilst adding complexity to the design and were therefore abandoned. (The fact that this performs as well as materialised views speaks very positively of Drupal's architecture and code quality.)
- Multiple revisions of any data.
- Multilingual capabilities.
- Support of tree type data models.
https://www.drupal.org/node/1035804#comment-6860070
On the whole I would call this a thoughtful and reasonable design choice, and one that is supported by real-world implementation on millions of sites at all scales.
Better yet, avoid the php ecosystem completely and go for a more mature one, even the python and ruby frameworks have better long term prospects than most php based stuff.
The biggest downside is that you'll find it harder to find programmers, the upside is that you'll find fewer 'absolute beginners' and interviewing will be a bit easier (fewer respondents, higher overall quality).
I'm writing this as a very long time PHP user that also spent considerable time on the drupal offerings.
That said, I hate Drupal with a passion (I don't get to make the choices at my current place and it's D.O.D Government work) and would rather work with WordPress if a CMS based on PHP is absolutely necessary, their user interface is much, much more friendly - and as a programmer I hate views, panels and all that other cruft I'm forced to work with because "that's the Drupal way."
Edit: Oh yeah, unless they've recently changed it, they store PHP serialized data in the databases, and full urls for the website, so moving domains is quite the fun times and you need a special plugin to do so (even as simple as going from dev.example.com to example.com breaks the serialized strings if you just run some queries on the database). The fact that they store whole urls with domain and everything in the database like that has always skeeved me out.
I've been working with Drupal for 7 years now and have worked with lots of other frameworks and langs in that time. I've had a hot/cold relationship with Drupal for the whole time, and have lately (since D8) had a substantial thaw with my attitude toward Drupal. It's nice because in the last several years of hating it but getting to know it really well, I've come out on the other side being able to get things done in very little time.
So back to the original question - what would recommend instead (and frameworks don't count)? It's got to be something that a non programmer can build their project with.
I agree it has gotten better on many fronts since the old days but plenty of my old reservations against drupal still (surprisingly, I expected those would be major focal points for improvement) hold today.
> It's got to be something that a non programmer can build their project with.
That's a huge restriction, and in fact, if a non-programmer can build a project on a self hosted platform then you're essentially asking for trouble with respect to security and future maintenance. I don't really believe such a system could be made if it is at the same time required to be secure and reasonably scalable, unless it is hosted and kept up-to-date and secure by another party.
The fact that wordpress.com is raking in the money is an indication that for simple websites this is definitely a market opportunity, and given the frequency with which self-hosted wordpress sites get hacked the 'non programmer' restriction would seem to be the cause of a bit of a problem. It's like asking for a car that can be constructed by a non-mechanic.
I think that's a good analogy actually, but that's the market. Honestly, I think to try and move into the market that WP has so tidily sewn up would be a waste of time and energy so Drupal has aimed for the higher end of the market - large institutions and companies with content mgmt needs. Also, in my (admittedly biased) perspective, Drupal provides vastly more interesting work opportunities than Wordpress ever could. That's why I chose to invest my time in learning it all those years ago.
> if a non-programmer can build a project on a self hosted platform then you're essentially asking for trouble with respect to security and future maintenance
Granted, but programmers are at least as likely to spawn security holes by being given a more powerful weapon. Relatively simple things like XSS and SQLi can be taken care of for a user who's not touching the code while they build their project. The strong params feature in Rails was a result of a gaping security hole in what I would consider to be the bellwether of good framework design. Mistakes are made.
So you still haven't named a better alternative ;)
In my opinion, I sort of agree - Drupal is the worst CMS, except for all the rest of them.
An extremely customizable cms for non-programmers as far as I know does not exist, including drupal, so we don't actually agree, it's just that you've limited the scope to the point where there is only the one alternative left and that alternative suits your particular use case.
Agreed that drupal is a more interesting environment than wordpress, many of the same restrictions and gripes apply to both platforms and drupal seems to attract the occasional wordpress user because of increased flexibility.
I will say that I think it's a noble goal to allow non programmers to build something powerful, but in my experience it translates to programmers being able to build features insanely quickly rather than handing that power over to non programmers. Maybe it's different in other sectors.
There is a whole eco-system around drupal of consultants who know just a little bit more than their clients to get them back out of the trouble they're in.
Agree with everything you said except this, PHP has recently undergone an renaissance in development, 7 is a huge step forwards (proper AST, the start of an optional type syste, 100% speedup, half the memory usage - I've actually seen these changes on stuff I'm running), composer and the PSR's also helped a great deal.
I expect the split on python/ruby/php to remain pretty constant, Python stuff remains excellent but they aren't doing anything revolutionary in terms of the Web story, Ruby with RoR has made some excellent changes in the new release but in terms of impact I don't think they exceed anything available on PHP.
I'll happily agree that PHP is a horrible language but it's also incredibly productive to program in if you avoid the warty parts (getting easier to do) and approach it like a Python (which I also am) programmer would in terms of writing good maintainable code.
The comment on developers is accurate, there are a lot of PHP "programmers" who fall short of the average in the Python or Ruby community however the total pool of good PHP developers is likely (I don't have metrics so I'm guessing on that one) larger than the total pool of good Python (web anyway, python is everywhere) and Ruby web devs just on community size.
I don't regard having to winnow the chaff a big problem, I'd sooner have more good applicants (in absolute terms) mixed in with a lot of bad applicants than having way less good applicants in a different language.
That said your advice on using Symfony2 or something else over Drupal is extremely valid and I can say that as I used Drupal for a couple of years and ran into all the usual issues (worked fine for non-logged in, fell over with a few hundred logged in users), as a nice way to generate largely static content for end users it's just about acceptable but people seem determined to not use it for that, whether you can blame Drupal for that or the end users is an interesting question I guess.
I now don't use Drupal anymore, although I might occasionally whip it out for a site that only requires a few admins to log in...