Our approach to subjective programming questions | Hacker News Reader

Our approach to subjective programming questions

blog.slant.co

110 pointsnatosaichek59 comments

Cool! You have "discovered what Stackoverflow users were doing," and where Stackoverflow stomped it out like a plague, you have made an offering for it. Good job!

I'm still amazed at Stackoverflow's failure to follow their own stated philosophy ("discover what your users are doing and enable it") in this particular area.

Pretty much for any question you find via Google that you are really interested in, you will find an SO question that fits the bill and has some big long jibber jabber filled message about being closed for such and such reason.

The strange thing to me was that when Jeff left was like when it went full tilt. It's like Mao died and the rest of the party big wigs realized they weren't being the best communists they could be and just started going overboard to stomp any possible conflict with Maoist ideology.

One of the most frustrating parts of some early experiences I had with some programming classes, was a lack of "meta". So, we're learning some specific constructs... but what is the fucking context!

This is one reason such topics come up, whether on StackLatest or wherever. The new "team" members want to know where the fuck to start and what it all means.

I'm put in mind of Organic Chemistry I. A great class for "weeding out" before majors are declared. At least as I experienced it, a boatload of often "arbitrary" memorization... A real slog, when you couldn't see the forest for the trees.

Mind you, some significant percentage of "What's best?" posts are indeed people just too clueless or lazy to tackle any element of the question themselves. But... that's what voting and moderation are for.

StackOverflow seemed to be dealing with this pretty well. I don't see what the need for the change (in attitude, if not stated policy) was.

I think many people, like me, have been waiting for someone to address this obvious niche that has been crying out for attention.

There's also an interesting path to monetization: it's pretty much models the buying process, esp for the enterprise. Also generalises to purchasing decisions, in general - it's relevant to all transactions. i.e. the whole economy.

However... very hard to do well. It's good start, but nailing it will take a lot of time, effort and luck. Starting in this very focussed and narrow niche is a great way to build understanding of what's needed and how to do it.

EDIT I found the site slow to load (actually got the wait or cancel dialog). Because such a site thrives on as many users as possible, it may be more important to make it accessible to more people at the expense of prettiness (I assume the pie-charts are partly to blame for the slowness). It also biases the answers and votes (maybe a good thing - target that niche?); might explain why emacs got few votes.

The presentation of information is too "detail-heavy" (i) the list of questions includes 3 alternatives and even some lines about each. The typical user wants to scan the topics to find one of interest. (though I guess having a some context is good, since that's what SO does; but only to clarify the question, not the answer).

(ii) in the actual answer, there's no summary of the results [EDIT whoops, there is a summary; somehow I went directly to one of alternatives at first]; no comparison of the alternatives; and no indication of what context each tool best suits ("job" for the tool); instead, each page is all about one alternative. I would guess that something like a grid might work better, with the tools listed vertically, and the attributes horizontally (with the "job"/context first). The user can then scroll down to examine each point in detail (or maybe drill-down, with expanding sections, for truly secondary/tertiary content, like comments/debates). Attributes could be voted on.

In short: organise it to suit the user-task, not the subject matter.

> I found the site slow to load (actually got the wait or cancel dialog).

Sorry about that, we are under some heavy load and didn't have time to do even the most basic speed optimisations (gzip etc). Its very high priority.

> The presentation of information is too "detail-heavy" (i) the list of questions includes 3 alternatives and even some lines about each. The typical user wants to scan the topics to find one of interest. (though I guess having a some context is good, since that's what SO does; but only to clarify the question, not the answer).

Totally agreed. I know how to fix this, will be done soon.

> (ii) in the actual answer, there's no summary of the results; no comparison of the alternatives; and no indication of what context each tool best suits ("job" for the tool); instead, each page is all about one alternative. I would guess that something like a grid might work better, with the tools listed vertically, and the attributes horizontally (with the "job"/context first). The user can then scroll down to examine each point in detail (or maybe drill-down, with expanding sections, for truly secondary/tertiary content, like comments/debates). Attributes could be voted on.

We have this page: http://slant.co/topics/what-s-the-best-jabber-gchat-client-f... that shows ALL the options and an overview of what they are suitable for. To further improve this we are working on a comparison matrix style feature that is approximately what you described here.

Would it be possible for you to email me? (stuart@slant.co) I'd love to be able to show you some 'in the works' features as they are being built for some feedback.

Another thing I particularly like about threads comparing two or more different technologies/frameworks/languages is when someone says "X does Z badly, and here is why Y does Z better." Slant seems to not be able to facilitate this very well.

Good point. We plan to add something along the lines of "counterpoints" ("X does Z badly"), and we have other viewpoints ("Y does Z well"), but directly pointing users toward solutions that solve those counterpoints ("[take a look at Y, because] Y does Z better") isn't something we'd considered. I'll definitely keep that in mind as we hash out the counterpoints idea a little more. Thanks kindly.

The strange thing is if SO actually decided to cater for this, which would be pretty trivial, they would win this game by default. It's basically just ocd run amok.

Gotta say, at first glance I really like this approach..

Here are a few things that are going on in the back of my mind.

> I'd love to see a time based filter. For example, consider the evolution of JavaScript libraries. For simplicity's sake, let's take the shift from prototype to jQuery. What you guys are creating could effectively answer this question: 'When did jQuery surpass Prototype as being the premier library, and why?' The filter would have to be periodic (since votes are going to be spread out), but I would find it fascinating to glean the shifts in developer technologies. Think Backbone, ember, angular, etc..

> I see a lot of ULs in place of what seem to be 'features'. While this may only be relevant for things that are capable of being compared (apples-to-apples so to speak), I imagine significant value will be come out of a resulting table of comparison; that is, if each bullet point could somehow be categorized, tagged or curated in some way. This may be hard to achieve (and may not be relevant in some cases), but you could effectively build out a report highlighting things that _are_ and _are not_ available (instead of having contributors write long lists that then users have to parse and reconcile). Does that make sense?

> In the blog post you state "we aren’t actually trying to tell you what the best programming language to learn first is" -- I appreciate that you don't want to suggest the 'best', but I imagine that's how most developers (particularly newcomers) are going to interpret the results. I'm curious if such a forum would influence new developers for the better or worse.

> Sign in with Facebook? What about stack exchange/github? I don't know what you guys think about leveraging reputation, but it would be nice to bring my existing recognition - score and tags (albeit slim) - along with me. Even integrating LinkedIn could be interesting with skills. Just a thought.

Awesome feedback.

1: That is a really cool idea and something I have been thinking about since day 1. I want to be able to show a graph that displays the relative change in opinion over time, annotated with the addition of new citations to the Viewpoints. So when someone updates the Ember Viewpoint with new information and the citation for it, you can do some basic "cause analysis" on what changed,when and why :)

2: Absolutely makes sense. We are still debating the best way to do this, but one simple solution is an editable comparison matrix that shows the simple features/specs that the Viewpoints have. The ULs would then be used to further flesh out the more significant points that can't be explained in matrix format (for eg how nice the UI is for a particular app)

3: Again, we debate this point all the time. What do you think about the phrasing "What are the different programming languages suitable to learn first?" or something along those lines?

4: Awesome idea, we are going to add additional login options soon, but the idea of bringing in external rep is very interesting!

I agree that the time-based view will be important, as the answers do change over time. One way to approach is to show votes in last 12 months, instead of all votes ever. Side benefit of this is that it encourages users to stay active and engaged with the site (to maintain and update their views as they evolve).

Maybe you could present the different options in reverse order of votes? (filtered a bit so that spammy answers aren't always first.) Changing the order of presentation can be a good way to stem cascade effects from seniority/etc. For example, the US supreme court votes in reverse order of seniority to help prevent cascading effects. (That way less senior justices can vote Wo undue influence...)

As the author of the recently closed/opened/locked question about brackets [0] on Programmers.SE thanks so much for creating something like this. I'm really enjoying all the additional detail on why something is/isn't the best. Keep it up!

http://programmers.stackexchange.com/q/188455/54164

Thanks!

Wow, they closed and locked that? That type of question is pretty much exactly what Programmers.SE was intended for.

But unless I'm not understanding Slant's mission correctly, this type of question wouldn't work there: it's for things like "What programming language should I learn next?" or, a little closer to your closed question, "Should I use curly or straight brackets to denote scope when designing a programming language?"

At first blush, this looks fantastic and I'm excited for it's potential. Particularly in my experimental side-projects, I'm constantly looking for some guidance from others.

Closed StackOverflow questions infuriate me to no end - not least because that's where the experts are that are eager to help!

Thanks for the comment and the content you added Toby!

I like a lot of this approach. But, please replace the pie charts.

http://en.wikipedia.org/wiki/Pie_chart#Use.2C_effectiveness_...

Humans aren't that good at telling more than very rough ratios in angle rather than length. Yes, when the differences are large you could tell at a glance the overall distribution of answers, but when it is close, you lose that. A horizontal stacked bar would achieve a similar visual flair but would be much more intuitively accurate.

It would be great if you could let users indicate preferences between answers so that you could produce an ordered graph of which answers are better than others. I'm thinking of something similar to the less wrong textbook recommendations[1] with it's rules:

   1. Post the title of your favorite textbook on a given subject.
   2. You must have read at least two other textbooks on that same subject.
   3. You must briefly name the other books you've read on the subject and explain 
      why you think your chosen textbook is superior to them.
   
   Rules #2 and #3 are to protect against recommending a bad book that only seems
   impressive because it's the only book you've read on the subject.

AFTER EDIT: I'm thinking this would be particularly useful if you've narrowed your choice down to a few options and wanted to see comparisons between those options.

[1] http://lesswrong.com/lw/3gu/the_best_textbooks_on_every_subj...

I think it's great to have a centralized place where you can find answers to subjective questions; however, the current format makes it difficult to digest __why__ something is considered better than its competitors.

It would be useful to have a set list of pros and cons of each entry.

Hey mate!

So the idea is to have the list of Pros/Cons! For example checkout this one:

http://slant.co/topics/what-is-the-best-search-engine-for-we...

It tells you what its good for, as well as the "cons"

Edit: Also topics like: http://slant.co/topics/what-is-the-best-javascript-drawing-l... make it clear what the best use-case is for the various options. Do you want something a little more like a "comparison matrix"? Thats also in the works to help compare specs/basic features.

I do like the content so far, but this isn't really a Q&A...it's more of a WikiAnswers, isn't it?

The good thing about StackOverflow is that, when in doubt, you can always use a user's score/comment history to gauge their credibility (though yes, that is an argument by appeal to authority...)

With Slant, the tracing of comments/submissions is as hard as it is on Wikipedia.

And the "the answer must be in article-ish form" has its disadvantages, of course, not least of which is that some of the best answers come through repartee, as they have in StackOverflow.

Also, I'm not sold on the font. It's not great at small sizes and you may want to stick with a sans-serif.

Hi Mate,

> With Slant, the tracing of comments/submissions is as hard as it is on Wikipedia.

We are building an "authority" SO style karma system, and unlike Wikipedia you can easily see who is a contributor to a post. This is something we agree is really important, but just didn't make the cut for the Alpha product. Please also note that your edits need to be initially approved by the existing contributors, which helps keep quality high and helps you trust the content.

> I do like the content so far, but this isn't really a Q&A...it's more of a WikiAnswers, isn't it?

Well there are questions asked and answers listed.. but you're right in that it's not really Q&A. Someone the other day told me it was "Q&Os" (Question and opinions)

Seeing the vote breakdown for alternatives could be one of the best features of this site. Can users vote easily without leaving comments ?

Being able to get a quick overview of community sentiment on what's the best library/tool to use for XXX on/in YYY could be very helpful when one is researching alternatives.

You can vote without leaving comments, but you have to create an account to do it.

Maybe it's because I am not signed in, but it looks like you are missing a lot of the tools needed to spot sockpuppets / shills, like history of votes / edits for a user, and list or voters for each answer.

If your site gets popular you will be flooded with astroturfers and over-enthusiastic project promoters.

Hey mate,

We are recording all that information and have put a lot of thought into how to combat those problems. Unfortunately they didn't make it in for the Alpha, but its definitely something we are aware of and are actively working on.

Thanks for the feedback.

Regarding sockpuppet accounts and the like, we do have that information - some of it will get more visible as we build out profiles and moderator tools, it's just been on hold while we focus on more user-facing features, since it hasn't been an issue yet. Account validation e-mails are also on the way.

Spam is definitely a concern, but we hope the fact that most content requires acceptance from another user before getting visibility will put up some barrier to entry, and in the cases where it's not sufficient, it should help us get some information on associated accounts. In cases where that fails, there's always manual moderation.

Regarding astroturfing, we welcome it, to some degree - we don't expect each viewpoint to maintain NPOV, we just want to make sure that each viewpoint gets visibility. Cons/criticisms will eventually broken out into a separate feature to reflect this.

I like idea but... the first question that interested me was

"What the best 2D Game Engine" http://slant.co/topics/what-is-the-best-2d-game-engine

And I'm confused by my options on slant. What is a 2D game engine? Can we even discuss that? Example: Phonegap, should I add it? People make games with it. They also make non-games with it. I didn't see a place to bring up this discussion (meta-discussion?)

Next: I've been looking at 2D engines recently so I thought about adding entries but I don't know the engines enough to do anything other than add them to the list. Except apparently I'm not supposed to do that? But even that is useful info (ie, to know what the choices are)

Anyway, not sure if that's useful feedback.

It always seemed to me that StackExchange's own http://programmers.stackexchange.com was intended for this very concept. From the FAQ:

What about subjective questions?

Subjective questions are allowed, but subjective does not mean “anything goes”. Please keep it professional at all times. If this is a question you'd be uncomfortable discussing with your colleagues in a work environment, it's probably not appropriate here, either.

For example, here is a question re: CSS pre-processors that seems pretty subjective: http://programmers.stackexchange.com/questions/181536/are-th...

Until your highly subjective question gets closed:

http://programmers.stackexchange.com/questions/188455/why-do...

The problem with that question is that there isn't really any answer to it. It basically just ends up being that at some point someone made an arbitrary decision to do something that way and a bunch of other people copied them.

I think this is awesome, especially since I asked about this exact thing just hours ago before you posted.[0] I'm really excited and I can't wait to see you take off!

[0]http://news.ycombinator.com/item?id=5286601

Thank you!

This is awesome. I browsed stackoverflow when it first started and just wondered when the first wave of overzealous "editors" would take over the site. It took a while, but those people killed spirit of the site for me.

This looks great.

IMO, a lot of the difficult things are subjective. Good luck.

I was just thinking of this idea over dinner tonight, when a friend was giving his recommendations on PHP and Ruby frameworks and their pro/cons. "There should be a site for this type of knowledge!" I thought. And --boom!-- I get home, and here it is on HN.

Looks like a great start. One thing that strikes me is that it's really hard to see how useful the data is on first glance. I kind of like that the pie chart is front and center, but for me it's useless without showing the total number of votes.

> useless without showing the total number of votes.

Hover and you can see the % and on the Viewpoints it shows the votes they received but obviously to be even remotely trusted we need total votes. Consider it on the to-do.

This is super awesome. You're getting criticism of the font. But I have to say that I really like it. I just created my own issue and like the UI all around.

For karma, I'd recommend that each person gets points in proportion to the number of upvotes combined with the number of characters they added to an answer that appear in the final answer.

Also, I think you need some sort of comparison or overview for each issue. And next to the pie chart I think you should allow for a 140 character final recommendation, listed individually for each user. E.g., "Dan recommends SilkJS because it's easy to write, fast, and better than Node"

Great ideas! Thanks kindly.

I really like the idea, and what few bits of criticism I had you have already seemed to address.

I just wanted to compliment you on something specifically that I though was really great: the registration process. Very similar, no confirmation fields, and, most importantly, I was signed in immediately after clicking finish. It didn't take me away from or reload the page I was on, and it didn't ask me to confirm my email. The small things really matter.

Guys, I really like what you are doing, I think there is a market opening for this type of site. Couple comments: (1) You probably ought to get tags implemented ASAP so it's easier to find stuff in broad categories (2) Some ppl might not be too thrilled with your lifting so much content from SO. I realize you're probably just trying to seed some content, but still. Might leave a bad taste for some ppl.

Thanks! Agreed on the SO point, I actually just tried it out on the "Haskel" topic and maybe a couple others with it all fully cited but stopped as it didn't feel right...We have over 300 topics right now with only those few from SO.

Working on point #1 right now.

I like this, but I'm having serious issues reading the content in a manner I'd use it.

It's the font choice, size, and line-spacing to start with. It doesn't lend itself to quick skimming, and to succeed in this segment you really need to nail this aspect -- because this is what we all want... quick answers.

Really good points and I do agree that we need some work on the presentation. Thanks for the feedback.

Please please please don't pull a quora and force people to have to register to see the vote counts.

Thanks you guys, i feel like you are the guys invented democracy to save us from SO tyranny.

Small Suggestion: All caps in Title makes the title very harder to read. so instead of WHAT IS BEST EDITOR can you change like What is Best Editor.

Looking pretty good. A few things that I noticed:

- "Read More" and "Seem n more viewpoint(s)" go to the topic page when I expected them to just load some additional information.

- "Read More" link is still present on individual topic page even though it goes nowhere.

Totally agree and we have plans to fix both points :) Thanks for the feedback! Was there anything else you noticed that was confusing?

Great site, I've signed up.

Suggestion: Disambiguate questions where voting on multiple choices is recommended. It makes sense to be able to choose more than one answer on a what-are-your-favorite-... question, but not on a yes/no question.

I like the idea, but it crashes Opera.

Really sorry about that :( it's very much in an Alpha state. Will get this fixed ASAP.

Thanks for the feedback. One of the projects that our front-end engineer is currently working on should fix this issue, though it may not be live for a few days. Very sorry for the inconvenience.

Beautiful and informative. Great job.

Yes, yes and yes!

Why limit topics to stuff where there's no objective answer? There are a lot of people (including people with 20K rep+ on SO) who quit SO in disgust and who would just love to share their knowledge on both objective and non-objective question.

Pivot immediately. Do both objective and non-objective : )

Instant winner IMHO, going to help people there immediately!

Well you just made my day, thanks so much.

On the objective comment, the format actually does work for objective questions just fine without us changing anything. We just excel at subjective questions so thats how we are positioning ourselves.

I think people want the ability to do both, but there was a huge need for a better way to answer objective questions. In the pre-SO world, forums and bulletin boards served subjective questions mediocrely, but objective question abysmally.

Solving the objective question problem is and always has been SO's mission. It's a hard problem, and they solved it well- at least partially by firmly avoiding discussion questions.

So, while I think there's totally room and a need for a better solution to discussion questions (and there are several active attempts: discourse, slate, quora, etc), I hardly see it as a fault of SO's that they pushed away subjective questions.

I found SO (over time) to give less and less useful information: the best information was always in the quasi-subjective questions. It's nice to know how to use something, but really, I want to understand why. The SO/SE theory is against those questions. I have 19K rep on SO, 5Kish on P.SE, and I barely contribute anymore.

So I am planning on creating a Slant account and experimenting when I get home.

I feel exactly the same way (not with the high rep you have, but a decent rep that I worked hard to cultivate in the past).

I think there are a good number of us who feel this way.

It is frustrating for a company to say it wants to "discover what its users are doing and enable it", and then to see that company squash so many individual pieces of excellent content those users have created.

Reply on news.ycombinator.com