How to talk to an open source project as a large scale or interesting user
blog.powerdns.com
blog.powerdns.com
We pride ourselves on being one of the friendliest MTA communities out there, so to find people who give up and quit using it is always sad. Many times we've contributed custom code to companies for free to help them out, but we can't do that if you are silent. So speak up, companies, even if it's only in private messages.
This looks good for what we want to do, let's test it out. Cryptic error. Google for 24h for a solution, can't find one. Ask on official mailing list / forum / bug tracker. Getting ignored as usual, or dismissed rudely. Never use that product ever again.
I'm a CTO and I do that process more often that I would want to, sometimes I don't get that far because Install instruction sucks, and usually that's enough to deter me.
A concrete example: Since The Guardian opened Scribe (their CMS WYSIWYG editor) I had it in the back of my mind for weeks because it looked really interesting and close to what I had been looking for. We were doing CMS upgrades recently, so I wanted to take the occasion to plug in Scribe.
It took only 2h for me to put Scribe on my "never use ever again" list. It all look pretty neat at glance, quick install, nice plugin system, yadda yadda,... However, their one line install simply doesn't work, their is no list of dependencies anywhere, and simply no documentation. Logged an issue notifying them of the poor quality of their setup instructions. Got a reply 7 days later, which starts a discussion. But now it's too late, still have personal interest for Scribe, however my company has already moved on.
I'm all for OpenSource, have some myself (though, more of a "here is what I did recently" type of guy), however if you want to pretend to be a serious library/framework/piece of software, that includes decent documentation with clear install instruction for most common distributions.
Plus, dialogue is good, but most of the time I need it right now, or in a very short time frame, so more often that not I will simply drop back to another less interesting piece of library (or nasty one) that simply works, because that is what matters, it works when I need it. (Again with the Scribe example, no matter how much I dislike TinyMCE, I ended up using it, because one line of JavaScript is all it took for it to work).
But absolutely we could all use some improvement in these areas.
Documentation is pretty boring to write,and not as fun as feature-du-jour, but may be there could be some guidelines about what a basic documentation should have, specially with GitHub it could be straight forward to have a Markdown template for basic doc (Installation, FAQ, Requirements, Contact points, ...), I know I use one.
It's funny, because the answer is right in the article. If your customer:
* is large and/or interesting * has a legal/security/marketing/ team that's worried about sharing information publicly * has an IT dept that doesn't know the whole company is depending on a single open source product * feels they have to use gmail accounts and made-up names to talk to you
Then maybe a public forum really isn't the place to expect your customers to ask for help.
I would suggest creating a partnership program for businesses like Cloudflare - swap tech support for publicity (which will then help drive adoption)
Giving away a good portion of your infrastructure in a public forum and asking for configuration help, IS just asking for trouble.
The "large and interesting" deployment line was stated and reiterated several times as being "important" to the level of help that you'll receive from them. I fail to understand HOW this will garner more users to your service.
Through PR/publicity.
"Used by Google, Amazon, Facebook, etc." makes an impressive case when you're choosing which project to use.
Nothing discourages me more from using a service if you differentiate the type of customer support you provide to a client based on their size or whether they have an "interesting" setup.
In the grand scheme of things, Google, Amazon and Facebook are less normal, so if you're tailoring your support model around those types of clients, then it's not for me...
I guess the thing is, you don't want to make that too widely known. It's more of an optics thing.
I would not entirely place the blame on PowerDNS here, it also points out that how our Corporate citizens can do better for the eco-system by being active partners with the Open Source projects they rely on, instead of hide and snipe mailing-listers.
Another reason why employees at big corporations are loath to contact open source projects is that they assume they won’t get help, and need to pay for support. And, while it is true that someone needs to pay the bills here, we realise that for many users, a support agreement is just not going to happen. Corporate IT might not even know the whole company is relying on an open source project. Questions might be asked. For such organisations, we are fine with providing free help on the public mailing lists. But to give proper weight and context to your issue, it helps if we know who you are.
I think corporate owners are in the best position to offer money for services. They are usually looking specifically for some sort of support contract they can pay for! (If Corporate IT doesn't know, that company has bigger problems anyway.) Something like a VIP program would be a great way to get these types of users in the door. Feed all that money back into your open source project - if you're big enough you probably have project support costs anyway (hosting, etc).
I agree with your second point as well, and I think it reinforces my point, because you can take those internal people that you are dedicating to the task and make them the active partners to the open source project.
It might be more productive to think about "How an open source project should talk to a large scale or interesting user".
I was deploying open source software in large corporations more than a decade ago[1] and I formed the impression that open source projects could do a LOT more to engage with and support their users. It sometimes felt like the developers just didn't care about users. Documentation was often sorely lacking[2] and "support" typically came in the form of mailing lists that were full of users trying to figure out how the software worked, and swapping advice and tips - often a case of the blind leading the blind.
The end result was that corporate users would sometimes choose commercial software, despite the cost, because it came with decent support and/or had functionality that the open source alternative lacked.
These days, I think that a product manager is an essential part of any open source project team.
1: e.g. http://mail-archives.apache.org/mod_mbox/tomcat-users/200009...
2: e.g. https://www.mail-archive.com/linux-raid@vger.rutgers.edu/msg...
Really? I see no entitlement, merely a very common annoyance that FOSS projects are traditionally very bad at support (and your frankly, shitty, attitude is part of that, and part of the reason many enterprises consider community open source projects to be absolute last resort)
If I had a penny for every time I had a question ignored in IRC or mailing list ghost towns, I'd be running my own Y Combinator by this point. Those that will actually take the time to help users out, or projects with sufficient and good enough documentation that the "hand holding" as you put it, isn't necessary, are like refreshing oases in a vast desert.
Reasonable requests (not demands, requests) for basic things like documentation and functionality are met with snark and hostility.
Thanks for contributing to the problem, I suppose.
a bargain (for you)
and ps -- just so we're clear, the package I've contributed includes the mandatory per-user function help (each with runnable code examples), plus a 10+ page vignette discussing common usage, implemented inference algorithms, and parameter setting suggestions.
The vast majority of email I get starts with the implicit sentence [I'm a lazy prick who can't be arsed to read any of the docs, runnable code examples, or surrounding discussion. So if you could answer questions from someone who couldn't do the briefest amount of research that would be fabulous.]
Those emails are closely followed in quantity by the emails best summarized as "I'm gonna be super vague here but something didn't work. I neither wish to share the exact invocation of your package nor data, and certainly neither of those without prompting, but something didn't meet my unknown expectations and I want you to fix it now!"
Climb off your high horse and put away the moral outrage, please. I am not your enemy.
Your edit only continues to speak to the bad attitude I mentioned earlier.
> the package I've contributed includes the mandatory per-user function help (each with runnable code examples), plus a 10+ page vignette discussing common usage, implemented inference algorithms, and parameter setting suggestions
That already puts this package at the 99.7th percentile, I'd guess, in level of documentation among open-source projects. x0x0 does not have a bad attitude. A bit of anger born of frustration, okay, but open source developers with bad attitudes do not write documentation at all.
And to answer your question, the primary purpose of the IRC channel and mailing list is to let the users support one another. Only when a question arises that even expert users don't know does the developer need to step in.
If you're willing to pay for support, an email to that effect would probably be welcome. Otherwise, you're already getting more than you paid for.
I share x0x0's surprise that that's not obvious.
That seems to be the model that Red Hat went with, for instance.
<rant>
In other words, don't say you provide support if you don't intend on giving it. At least a message along the lines of:
This is provided as is, where is, if it breaks you get to keep both parts
As it stands, now I've wasted time, and now I hate your project for being an unsupported piece of shit, and now I hate the developers for fucking lying about it.
This is a literal experience I'm fighting with right now.
</rant>
Also, Open Source has come a long way in the past decade. Maybe it is time to revisit that initial impression?
For HN'ers who are still learning the finer points of English this title shows one of the many weird inconsistencies of the language.
Normally, when trying to say 'a thing' where the thing starts with a vowel we change the 'a' into 'an' so 'a axe' becomes 'an axe', 'a umbrella' becomes 'an umbrella'.
However this is a notable exception: when pronouncing 'user' there is an unwritten 'y' e.g. it is said 'yoozer', thus we wouldn't normally add another consonant here so it should be 'a user' not 'an user'. If you say it out loud you can hear it (and hopefully feel the extra tongue work that, traditionally, we try to avoid).
Another common exception is 'Hotel'. It takes 'an' rather than the expected 'a' as for some reason we drop the 'H'. For me it rolls better off the tongue with 'an' but this may purely be due to conditioning.
'Herb' is a difficult one as in the US it's pronounced 'erb' while in the UK it has a hard 'H'. Thus in the UK we use 'A herb' — I assume the US says 'an Herb' — correct me if I'm wrong.
It's nonsense like this that makes me pity you brave fellows who attempt to master our pidgin of a language!
Edit: change 'pigeon' to 'pidgin' :)
Yep.
I'm not sure about the rest of the world, but in Australia I don't think I've ever heard 'an hotel'. It's always 'I'm going to a hotel'
Saying 'an Hotel' makes me pronounce 'Hotel' in a very uncomfortable way.
The south and the "posh" (there is quite some overlap there as there is a perception that the south is more posh and the north more rough) use "a hotel" which is also technically correct, as you move north the "h" gets dropped so "an" becomes correct, though if you are writing that you should have a apostrophe to indicate the dropped letter (an 'otel) - in fact the 'postrophe is sometimes even pronounced by way of a very small pause (this is often a clue when decided if someone has a particular accent or is just imitating it).
EDIT: Wow, five overlapping replies in two minutes, all correcting the "hotel" thing... Just goes to prove the aphorism that the best way to get help on the internet is to post an incorrect answer (perhaps Cloudflare should have tried that...)
# yes a hotel an 'otel
# no an hotel a 'otel
Sorry for being pedantic but since we are on topic I thought it would be relevant to bring it up.
As you can see from the rest of this conversation (and the way my points for the article has swung violently from -1 to +6 and and is falling back again) I'm clearly no expert despite having been born to it! :)
Regarding herb: in written word it is "a herb" for correctness, though few in the north pronounce it exactly that way as the "h" is dropped to produce "an 'erb" or sometimes (though this makes you sound uneducated due to certain stereotypes) more like "a nerb" (this time with the harder "a" as in "an".
It's nonsense like this that makes me pity you brave fellows who attempt to master our pidgin of a language!
It is common to respond to people apologising for their bad English with something along the lines of "don't worry, I deal with worse from people who have been here for generations". We are not being patronising here (well, at least some of us aren't!): sometimes we genuinely do have to internally correct/translate more when talking to someone from 300 miles away than we do when talking to a Pole/Spaniard/what-ever who is not (yet) particularly fluent in English.
If you make the word boundary more pronounced, you get "a hotel" with a clearly noticeable air-burst, and a clearly distinct "h".
Same process as with "herb" etc. and some other words that can be pronounced both with and without aspiration in English.
This is much the same as in French, where if the preceding word end in a vowel (with certain exceptions) and the current word starts with a vowel sound, you can usually elide the last vowel in the preceding words except when the "h" is aspirated. My French teacher liked to say that the French "thinks" they are pronouncing the "h" in those few cases (aspiration usually refers to cases where there's a strong burst of air in pronouncing the sound, but in French "aspirated h" does not actually involve fully pronouncing an "h", though there may be some aspiration)
(I may confuse this - this is based on recollection of my French classes 20 years ago...)
(I think it's 'west country'... I'm not quite up-to-speed on my UK regional dialects)
You base a/an on the sound of the first syllable rather than what the first letter is.
An hour. An herb. A hotel. An HMO. An REI. A one-off. An MRI. An fMRI. An SEO. A DOE building. A user. All of these are for US English pronunciation.
http://www.theguardian.com/guardian-observer-style-guide-h
http://www.telegraph.co.uk/topics/about-us/style-book/143530...
http://www.economist.com/styleguide/a#node-21532141
I've generally used the Economist in my past writing and they say:
An
An should be used before a word beginning with a vowel sound
(an egg, an umbrella, an MP) or an h if, and only if, the h
is silent (an honorary degree). But a European, a university,
a U-turn, a hospital, a hotel. Historical is an exception:
it is preceded by an, the h remaining silent.
In modern vernacular usage, "to beg the question" is
sometimes used to mean "to raise the question">
> An herb.
Erm...
A Herbie toy.
An herbal tea.
It's possible that in the eastern or southern US, they say the H in "herb".
Whew, that was easy. Next time, try some of the 100 million people living on the east coast.
Your snark is rude and uncalled-for.
I'm just curious so that I can try to imagine the accent.
I live in the southeast, we say things like reckon, yall, and wander, but "erb" is still the accepted way to pronounce herb 'round here.
Yes I do, because YOU are such an hysterically funny
person. Did you notice how I used "an" there properly?Do you voice the "h" or not?
If you pronounce it "iss-tear-ick-lee" then you'd use "an".
If you pronounce it "hiss-tear-ick-lee" then you'd use "a".
Personally I've never heard the word pronounced with a silent "h". I checked several dictionaries and they all list the word with voiced "h". None of them even list the silent "h" as an alternate pronunciation.
Therefore I would say that the voiced "h" is the correct pronunciation, and "an hysterically funny person" is wrong.
Given the "did you see" comment, it's a good example of a hypercorrection:
A U.S. Navy Aircraft Carrier.
A one time thing.
English is a pain even for non ESL and that's especially true in the South!
Since the article's title is neither linkbait nor misleading, we reverted to it, or rather to a shortened version of it that fits the 80 char limit.
I'm also demoting this subthread as obviously off-topic. It makes little sense for a bikeshed discussion like this to occupy the top slot of the top discussion on HN, but these things get reliably upvoted.
The trouble comes with the upvotes that make the digression into a top subthread. Bikeshedding is a collective creation that no one particularly intends.
It feels to me like there's a simple and elegant solution to this problem waiting to be discovered.
I'd be looking to provide an extra private/internal support email address made available to the largest customers.
> Support via private (email) ticketing system, with guaranteed 8 hour response time, during European office hours (9AM – 6PM, or 3AM – noon EST). Includes resolution of 5 incidents. Does not include DNSSEC support.
although CloudFlare should have probably bought the carrier grade support.
I'm not sure what else PowerDNS could have done in this situation, and I can only imagine their teeth gnashing when they found they'd lost a lighthouse user.
Typically, lighthouse customers almost always also pay for support. Upon reflection, their post-mortem blog post was extremely well handled and written.
A "carrier grade" support contract sounds expensive.
If I'm going to my boss to ask for $x0,000 for a support contract because of a specific bug we're experiencing, I need to know the resolution won't be "WONTFIX"
Also, carrier grade support does indeed sound expensive for a small shop. Might even be deemed expensive for a medium-large shop (since they don't list pricing details). However, for an outfit as large as CloudFlare, I find it hard pressed to believe they couldn't afford Basic Support considering that one of their engineers on this thread stated that DNS is a critical part of CloudFlare's core service offering.
You don't ask for a support contract for a specific bug, but for a general class of support and current + future bugs. As part of the negotiation, your boss and organization can also leverage "lighthouse" status to influence PowerDNS to fix even minor bugs quickly as part of a negotiated Service Level Agreement ("SLA").
EDIT: To give CloudFlare the benefit of the doubt, it is possible that PowerDNS didn't make their support options clear to users before, and only updated their support pages after this post-mortem.
By all means put an asterisk saying that the prices are guides only, or have large letters saying discounts may apply, or whatever, if you must. But not doing anything doesn't just aggravate your customers - it often means we go elsewhere or just use mailing lists for support.
For all the huffing and puffing I see people do about professionalism, very few of the "professional" mailing lists I'm subscribed to have adults that can say "I'm A at B, and am looking for a contact at C to email me off-list to discuss D."
I think we got to the point where we just said "there are N features that we want" and decided to build our own. The result is that we've been able to do all sorts of stuff that's CloudFlare-specific: bear in mind that we handle two different sorts of DNS queries: external ones (that might or might not serve a CloudFlare IP that is proxying for a customer site) and internal ones (that need access to public, non-CloudFlare IPs). We also recently introduced a CNAME-flattening feature: http://blog.cloudflare.com/introducing-cname-flattening-rfc-...
There are other things we want to do to be fast in two ways: fast to respond (http://www.solvedns.com/dns-comparison/2014/04) and fast to update (which means integration with our replicated, global settings databases).
We also have all sorts of special things that we do to deal with attacks on our DNS infrastructure that include the DNS server talking directly to NICs for filtering and packet handling. We're also generating bpf bytecode directly for specific filtering activities.
Bottom line was that DNS is a core technology for CloudFlare; it made sense to totally own it. I don't think this says anything bad about PowerDNS. Note: HTTP serving is a core technology. If nginx weren't so configurable (especially through OpenResty) we could easily have been having this discussion about a web server. But equally agentzh (who wrote OpenResty) works for us.
Source (was linked in article): http://blog.cloudflare.com/the-story-of-a-little-dns-easter-...
I had no idea what I was missing as from now on, I'll never again have to deal with forgotten serials or manually adding slave zones to the secondary DNS server.
Everybody still running with bind, thinking it's "good enough", ask yourself how many times you were inconvenienced by either of the two issues and decide whether it might be worth looking at alternatives.
The fact that PowerDNS apparently also is really performant doesn't matter for my mostly minimal use case, but it's good to know that if something great happens, I won't have to migrate again (not that migration is hard - powerdns comes with a helper tool to read your bind config and produce SQL for your zone files)
If you are a "large and/or interesting" company, it would behoove you to disclose your identity in a safe and responsible way to projects (open source or otherwise) who represent any important part of your business. The trick is to do so in such a manner so as to derive the many benefits (increased responsiveness) without the many liabilities (spam, sales calls, etc).
By remaining mum about who you are, the signal to noise ratio of your requests do not stand out from the crowd. This is unfortunate for both sides. It does, however, reinforce the notion that companies and projects should aspire to provide excellent support to all users. You never know when you have a "secret shopper" in your midst.
Vendors also need to remember, your users change companies, a user could be at some small unknown company on day, and the next be in control of a large budget at a high profile company. Your reputation follows that user, and what they want to see is you're making a good product. If the smallest company is reporting an issue that would affect your customers, I'd want to see that you're addressing the issue with that company so I don't have to at my large/interesting company.
Would you want to dine somewhere that got panned by a restaurant critic and then wrote an essay saying, "Well, we'd have been a lot more careful with his food if we'd known he was an important critic!"?
As a small end user of open source products, this is exactly what I expect and have assumed the case to be, even if I've never seen it spelt out like they did. I don't have any delusions that an open source product should cater to me.
I don't see how it's tone deaf, or more specifically I don't see how their statement could upset anyone other than those who expect open source to be a magical wellspring of doing their job for them.
Open source is a more like having another team at your company that isn't beholden to yours. You're getting access to the internals of the project and can communicate as if they're colleagues, but don't expect them to work on your project. If working together is mutually beatifical, or you pay, then things change.