37signals doesn’t like you sorting stuff. Are you just remaking a spreadsheet? | Hacker News Reader

37signals doesn’t like you sorting stuff. Are you just remaking a spreadsheet?

n8.tumblr.com

166 pointsnate67 comments

I love sorting and filtering.

When I'm looking for something, I often remember where it is by different criteria (date last visited, date saved, keywords, frequency of use, numerically, alphabetically, etc) so I get frustrated when my options are limited. But I do understand why designers want to do without.

I hate even more when things are sorted badly. For a long time Google Bookmarks were sorted alphabetically by default. Every single time I went looking for an old bookmark, I'd navigate to the Google bookmarks page and then I'd have to click on 'Date' and then on 'Bookmarks'. (Random tweets and other nonsense was included with my bookmarks by default, too. You had to choose 'Bookmarks' to view only your bookmarks.) 99.9999% of the time you're looking for a recent bookmark, not one that begins with a zero or 'a'.

To this day I'm baffled as to what confusion of ideas lead to that interface. They fixed this in the December 19th update/redesign but I'm sure it persisted for at least a couple of years.

fck, finally! I've complained about this same issue for years! Thank God is Christmas! :P

I think the author is missing the point. Some data just looks and acts beautifully in tabular / spreadsheet form, and if you have an app where that is the case, then by all means, use a tabular layout.

Does that mean you are just remaking a spreadsheet? Cmon. No. There is all sorts of added value for using a webapp that has a tabular data view. Maybe it's nicer to use. Maybe it comes with choices pre-filled out for you for certain columns. Maybe you provide user/team visibility, or sharing, or all sorts of other stuff that a simple spreadsheet could never provide standalone.

The point is, don't shoot yourself in the foot because 37signals forgot to put a tabular view of their data in. That means nothing. You should be thinking -- does a tabular view add value to your app? Yes? Great, then provide a tabular view for your users.

Yeah, this is a stubborn hipster design decision, which the post author is trying to propagate through hero worship.

> You should be thinking -- does a tabular view add value to your app?

I think the point of the article was that the author never thought this, but presupposed the answer was always 'yes'. It wasn't until looking at alternative implementations that this question popped up.

I've done the same myself. I once implemented sortable live lists (lists that may be updated with added/removed items in the background, with only part of the list shown and the rest retrieved from the server). I wondered, 'how should this work?' and cast around for other examples. At first I used the (old) Google Reader model: two lists, one is 'All items', the other is 'New items'. But then I realised that, actually, sorting doesn't add anything, and indeed might confuse people who expect predictable order in their lists. Problem solved: don't allow sorting, instead provide a canonical order.

Sorting in a paged list (as seen on most web pages) isn't very useful unless you intend to page through the whole list. If you're only going to look at one page, you're probably better off with a search for the attribute(s) you're looking for rather than trying to find a sort that puts those results up top.

Developers love to put in sort controls because they cover lots of use cases and don't take a lot of thought to implement. And mentally, they tend to think about sorting a lot anyway. But my experience with looking at data from non-developer users has always been that they rarely use sort options when available (and search filters, either, btw. ) You see about 5% of users employing those controls.

So the developers think they've solved certain use cases by providing sorting, and users don't see those cases as covered, because they don't see that they can use sort to solve them. It's better to just not put in sort, reduce complexity and find other ways to cover those cases that users are more likely to employ.

I am quite sure you have fallen into the Google group think of no sorting + no folders. Every time someone at google has the balls to challenge that dogma, like replacing "add label" with "move to" in Gmail, correct usage soars. The search & labels only fallacy has led to some terrible interfaces, such as the Google Docs home page.

I'm quite sure you're wrong on that, since I know exactly how we reached that conclusion despite originally thinking otherwise. I worked on the issue at another major net company where we were dealing with a _lot_ of lists being viewed by very large numbers of users. While I'm unfamiliar with the results of research at Google, both analytics of production services and follow-up user labs showed very little traction for using sort to reorganize sets of information among non-technical users.

It's just not how regular people think about manipulating information - in the real world, sorting a set of any interesting size is impractical if not painful. You can literally ask a loaded question - "is there any way you could sort those to make XYZ easier?" and folks will avoid that in favor of other method.

That is surprising, as I have data points that show people prefer clicking "least difficult", or "most popular" vs search or an alternative categorical method. I really think it comes down to context. You use the phrase "interesting size" and that is relevant, as I only consider sorting of stuff that is generated by a human, not humankind. I fully accept the interface should be different for human data versus humankind data. I think google products often assume every list is infinitely long, and it can lead to poor interface choices.

By "interesting size" I mean "large enough that it's no longer convenient to page through it." Generally, for less than that, sorting's not necessary because you can just scan through the list visually, and for longer than that, the sort generally becomes useless anyway. It didn't seem to matter if the list was 100 long or 100,0000 long.

We saw less than 10% of users ever employing the sort controls; that's not zero, but it's low enough that you can't depend on them to cover use cases.

The problem with your examples are that you're not really showing "sorts" from a user standpoint. What a user is really doing is effectively filtering for the "show me the N least difficult", not "sort by difficulty, ascending." Also, sorting on a relatively scaled value make more sense, but a lot of the time the values in a column don't have that kind of relationship. What's the "least zip code" good for?

It's hard to say why the other applications don't have sorting, but I think it's a stretch to think there is some design genius behind it. It could be for pretty mundane reasons.

I agree you don't have to reimplement spreadsheet features, but make sure you have a CSV dump if you think your users would appreciate that.

I remember a place where I worked where they'd show users a big table with dozen of statistical numbers and percents everywhere. I kept saying to just show one number (The relevent %) because users didn't care about all these numbers. In fact, it'd make everything harder to understand.

But, "the leay way says to do something and iterate based on feedback" they said. So, a couple of months and iterations later, it was all redesigned with a single number: the important percent.

A trick to help focus on the important stuff is to ask: "If I ask a user that doesn't care at all about my product and is extremely busy, what should I show to make it interesting (Or convenient)?"

ha, I would wager that there is a simpler reason: sortable tables in HTML are nearly impossible to implement.

once you have a lot of data, you want fixed column headers. stacked sorting (column A first, then B), a lot of things suddenly become needed additions.

and where do you sort? front end or back end? how much data are processing?

just look at google spreadsheets and you'll see how sub-optimal it is.

Data tables (jQuery addon) does a pretty good job with HTML tables.

If you can push all your data to the front-end all at once, yes. DataTables makes this pretty straightforward.

Otherwise, his point stands. DataTables has hooks for this stuff, of course, but that's just a way of saying "DataTables doesn't itself solve this problem".

I rather like the fact that DataTables doesn't try and solve the whole problem (front end and server back end) - otherwise you end up with things like ASP.Net server controls and DataGrids which are really just an attempt to port desktop development approaches to the web - creating an unpleasantly leaky abstraction along the way.

I find that using DataTables as front end and hooking it up to a backend is pretty straightforward and results in a nice separation of concerns - let the server/database worry about paging, sorting, searching and filtering. Of course, if you have a relatively small amount of rows then just let DataTables do all the work - but at least you have the choice.

> If you can push all your data to the front-end all at once, yes. DataTables makes this pretty straightforward.

Paginagtion(ajax or page reload takes care of it). I use slickgrid https://github.com/mleibman/SlickGrid; pagination is straight forward, filtering takes some work. But do the hook once, and wrap the table in jquery-ui css framework's ui-widget, and you have a good looking, working table.

> but that's just a way of saying "DataTables doesn't itself solve this problem".

slickgrid is sortable, but filtering needs work. In fact, "slickgrid doesn't do everything" is part of its philosophy.

I have come to opposite conclusion that of author's. Having an excel sheet replacement in your repertoire is important, especially when you are replacing enterprisy stuff.

Data tables was too slow for us over using 500+ rows. We went with SlickGrid... which is faster but just all over the place.

I found DataTables was much, much faster when I passed a JSON data hash - I think the DOM manipulation is what normally kills it. We've got some tables of several thousand rows in a small internal app and performance is fine, even in older IEs. (I think I tested up to several tens of thousands before IE started to be unreasonably slow - YMMV, of course.)

Just add some JS and call it a day. http://www.kryogenix.org/code/browser/sorttable/

It is not difficult at all. You sort on the backend where your data is using your RDBMS then you display it.

I'm confused as to why this is hard. I code primarily in java with JSF for views and this is possibly the simplest thing to do in any of my projects. Using something like primefaces I generally never even need to write a line of javascript unless i need to customize some action and I can use theme roller / stock css for appearance.

Example: http://www.primefaces.org/showcase-labs/ui/datatableComplex....

This is working client side, if i want to do server side its as simple as implementing a single class that pushes the sorting/filtering/paging off the to the database:

Example: http://www.primefaces.org/showcase-labs/ui/datatableLazy.jsf

It can't really be much harder to do this on other stacks can it?

JSF is a bit of exception. I haven't 'really' searched but its harder to do this in say Struts. What rule of thumb for the data size do you use when choosing between client side and server side sorting?

I wouldn't say there is a hard rule of thumb and there are three levels of operation possible, at least with primefaces:

1) All row data sent to client side, all paging done on client

2) Backing controller contains a List with all data in memory, client only gets 1 page at a time via ajax updates.

3) A truly 'Lazy' data model, ajax request for a new page of rows, controller references a lazy data model that fetches just that page worth of data from the database tier and sends it to the client.

In practice I really only use #2 and #3 normally. Unless the application has a bad internet connection as a requirement I assume its faster or negligible to fetch pages via ajax vs loading them all into the client up front. As far as choosing between #2 and #3, for my applications its usually obvious, the table either contains 5-25 items (go with 2) or thousands of items (go with 3).

In the middle ground i guess it comes down to loading: Can you afford to use more memory in the web tier or would you rather keep that memory usage down and suffer slightly long update times? Assuming your database is well designed it should only had maybe 10ms onto your response time to just grab the page from the database vs already having it in memory.

just look at google spreadsheets and you'll see how sub-optimal it is.

How is it sub-optimal?

One of my jobs is to make a UI that gets this right, so I'd appreciate hearing what you (or anybody) find deficient in the way online spreadsheets do this.

excel is the reference. it is so ridiculously optimized, it is not even funny.

outside of spreadsheets, take the itunes simple list view as a baseline. the 'simple' things are important. double-clicking the column border resizes the column to fit the longest text, etc.

sortable tables in HTML are nearly impossible to implement.

This is definitely a solvable (and solved) problem.

and where do you sort? front end or back end? how much data are processing?

If you have all the data loaded clientside, sort it there. Otherwise, just make a call to the server to query for data compliant with the new sort order. Both methods are fairly low cost.

Sorting on the back end is generally not cheap unless your dataset is very small (or unless it's indexed, but with arbitrary sorting it wouldn't be).

This may just be a case of giving credit where none was necessarily due. The absence of sort may just be a victim of 37signal's own design philosophy of only including features that users want and repeatedly request thus causing them to float to the top of the queue. In the grand scheme of things, High Rise and Base Camp users probably have other more pressing feature requests / complaints that they would rather get implemented pronto.

In my opinion, Google put the search vs. sort debate to rest a long time ago. For large, multifaceted datasets, search with faceted filters is almost always the right approach (again IMO).

Only for data that is subjective - given a search term which site is most relevant? Who knows.

But for questions like - allow me to examine the cost-distance-availability of hotels sorting is essential.

And indeed, Google is moving in this direction: they are increasingly returning inline results for factual (less subjective) data. E.g., https://www.google.com/search?sourceid=chrome&ie=UTF-8&#...

Was there ever a serious "debate" regarding search versus sort? They clearly complement each other. Google, by failing to support meaningful sorts, removes a convenient way to access information.

For example, in my old mail system, I could often home in on a particular message by sorting the relevant messages by size (simple communications < code exchanges < zip files containing all project files). Gmail's search-only approach-- while no doubt theoretically pleasing to a company that cut its teeth on search-- feels like half an implementation to me.

I can't argue with that. To be more precise than my original post, Google may not be the best example here, since it implicitly sorts based on the vague notion of relevance, which is where the magic lives. I'll amend my original point to include an implied sort order that aims to please "most of the people most of the time".

I always understood the sort in "Search vs Sort" to mean manually categorising documents/things into boxes or folders. Not sort as in an SQL order by clause.

Right, it's search vs classify, not search vs sort.

Once something is "put into a box", the "box" is just another attribute of that thing. I don't think it matters whether the user is classifying stuff arbitrarily, or the system is implying those attributes.

This post highlights the difficulty in making data accessible in a "traditional" web app. One of the most intuitive interfaces for accessing and manipulating tubular data is the spreadsheet, but very few web apps look or behave like a spreadsheet.

I would argue that there is an inherent friction between usability and accessibility of information. Adding options to sort, filter, search, or selectively show or hide fields might make data more accessible but result in reduced usability overall. The exact point where the tradeoff happens isn't clear to me, but 37signals tends to err on the side of usability, at the expense of accessibility. Of course, one could argue that in a data-intensive app, accessibility _is_ usability, and vice versa.

> Why wouldn’t a user just use a spreadsheet then?

Spreadsheets are hard to share and hard to version-control, and the data is cumbersome to access programmatically.

Google Docs spreadsheets handle all of these things.

I think that it is better to question why developers still create so many simple CRUD applications. Where is the next generation MS Access?

Zoho?

I am aware of things like Zoho and Oracle Application Express. These mostly seem like MS Access moved to the web, which is not a bad thing. There is obviously another step that needs to be taken to allow a large number of simple CRUD applications to be built by anyone who is technically competent but not a developer.

My day job is using Application Express. And yes, it's explainable as an Access-for-websites, backed by an Oracle database.

I've actually been working on exactly this for quite some time now. You can check it out at http://loggur.com.

I should note that a number of things are currently disabled on the site (like the mobile version and invite referrals), but the current information on the site should give an idea where Loggur's headed. A few things are disabled because just last week I decided to convert the entire backend to Node, mainly for Socket.io... because the idea is to let users edit and use apps together in real time... probably using a turn-based system or variations thereof. I was going to wait until after beta release to do this but decided it would probably save a lot of time and effort in the long run to knock it out while the code base is relatively small.

And I actually just now (re)hacked together the invitation system right after I saw your comment. Gotta seize the opportunity, even if it only means another 5-10 sign-ups! The plan was to wait until after the app builder is complete and use Loggur to build Loggur (i.e., recreate the majority of the current site), as it wouldn't take long at all; but the opportunity came up to do a shameless plug.

And I thought I was done coding for the day!

Barely noticeable text: "Help us get featured on", followed by the logos of TC, Mashable, etc.

Cute :-)

> I think that it is better to question why developers still create so many simple CRUD applications.

Because clients keep throwing money at us.

Because people don't want a machine they can learn to use in order to solve their problems. They want their problems solved right now.

Yet spreadsheets are incredibly popular.

I am not claiming that every computer user should want to create simple CRUD applications. A few technically competent non-developers might wish to do so though. Many of us who have been around for a while have had to deal with someone creating an MS Access application that is eventually used by everyone in a department. Companies would often then pay for a "professional" reimplementation of the MS Access application that would take months or years and cost a fortune. Often the end result was something that did not work well and was hated by users.

> "Where is the next generation MS Access?"

In Sharepoint.

This is very accurate.

Back in the 90's, everybody wants app with DB capability. Thus MS Access fills the need. It's cute and perhaps less barrier to enter by non-technical people.

Fast forward to 2010, everybody wants an agile web-app that integrate nicely with BI, Access, Excel, Word, with version control and what not, with CMS capability, with alerts, RSS reader, portal like capability... here comes Sharepoint.

It's a really good idea to question the fundamentals of presentation. People, however, are really good at looking at tables and detecting patterns (at least, according to Tufte), and, since we're unlikely to anticipate all the ways people will use data, it's a good idea to make sure that, in addition to the good ideas we come up with, we give users options to muck with data to their heart's content.

we give users options to muck with data to their heart's content

The answer there is probably to let the users pick their own tools too. If someone wants to chart data in your system for a PowerPoint presentation, the answer might be to give them a CSV export, instead of building a chart drawing tool into your application. (Of course, that assumes that your monetization strategy doesn't rest on restricting access to your data...)

Are you sure that people are good at finding patterns in tables? If I give you:

x = {1..5} and y = {3.0982, 3.3928, 3.8838, 4.5712, 5.45500}

can you tell if that's linear, quadratic, or something else? Probably not. But if I show you a graph:

http://www.wolframalpha.com/input/?i=.0982*x**2%2B3

it becomes pretty obvious. A table is not the right way to display this data.

Sure, graphs are often better than tables.

That's not relevant to the point of whether tables should be sortable, if you already provide a table for some reason.

A more relevant task:

Given the data

Can you tell if the columns are correlated? With sorting, you can, easily.

37signals is pretty dogmatic about their design: what features _37signals_ believes their products should and shouldn't have. Just because they don't have the feature doesn't mean lots of users don't want it.

Furthermore, sorting large data sets is a huge performance challenge. I think that's the main reason why you don't see it in Gmail. Not because sorting is not desired.

Some of the stuff they do isn't that far from sorting. The screenshot where you can filter to "last 7 days", "last 30 days", etc. is a discretized alternative to sorting by age that then doesn't sort within the discrete buckets. May or may not be a better UI; depends on what people use it for and what people would've used sorting-by-age for.

Most would call that a filter, not a sort.

It's a filter, yes, but a filter based on picking different cutoff points in an ordered column: 7 days, 30 days, 90 days, etc. I often find myself using these things as a roundabout way to sort a column, where first I get the last 7 days, then the last 30 and skip the ones I'd already seen, etc. Though it really depends on the app whether users would do that. I recall doing that most recently on an apartment-search site that offered that kind of filter but no sorting: I wanted to see the most recent listings, so first I saw the most recent listings, then loosened the filter to the next-most-recent category, etc. Would've just preferred a Craigslist style sort so I could continue scrolling down instead of doing the filter-loosening thing.

You could embed a Google Spreadsheet (http://googlesystem.blogspot.com/2006/11/major-update-in-goo...) and use Google Apps Script to feed it data from your database (http://code.google.com/googleapps/appsscript/service_spreads...).

Here's an example of how to connect a Google Spreadsheet to Neo4j running on Heroku (http://blog.neo4j.org/2011/12/neo4j-labs-heroku-neo4j-and-go...).

I'm developing an electronic parts inventory web app because I needed a better way catalog, order and share parts. I couldn't think of a better way than a spreadsheet to display the dozens of parts a user might have. Each part has an ID from the schematic, name, value and description which are displayed using a dojo data grid (Very happy with dojo grid btw). The rest of the attributes are hidden until the user needs them. There is a lot of data and comparisons being done and spreadsheets, it seems, were my only choice. The electronics industry hasn't solved this either. Take a look how ugly and overwhelming the product search is from Digikey, Mouser or Newark.

You've made some good points.

I don't know how complex your data set is, so just remember that if you have an extremely complex set of data, give me the user a way to find something based on different criteria. If an item has a date, a number, attachments, etc give me a way to sort through them so that when I kinda know what I'm looking for I can find it.

Sorting is useful when order is more relevant than the exact results. Many sites just throw away this use case, because it's inconvenient.

is this what's called minimalist design?

Providing obvious, useful functionality has no more relationship to the concept of a cargo cult than providing a cup holder in a car.

Don't deny your users the 80% (more like 99%) case just because there's a complicated tool out there that, with a bunch of bloat and UX problems, can also provide the 20% (1%) case.

He actually proved the opposite, that sorting functionality in such apps is far from "obvious and useful".

37Signals doesn't provide it in widely used apps, and very few are complaining. So sorting is more like the 20% 1% case.

Following whatever 37signals does is what seems like cargo cult. They do so many things wrong, and yet designers like this come and say "If they are doing it that way then it must be the right way".

I'd rather be allowed to choose rather than forced to accept

That's one problem I see with UX/UI designers; sometimes they don't really think about what the end user wants.

And this statement is just bull-crap:

"All this got me thinking about if I’m truly designing a solution to a problem if all I’m doing is replicating the features of a spreadsheet in a web app. Why wouldn’t a user just use a spreadsheet then?"

Because you're building an application not a spreadsheet viewer.

Reply on news.ycombinator.com