Comment by josh_atlassian | Hacker News Reader

Comment by josh_atlassian | Hacker News Reader

Parent Full thread

Just so everyone is aware, this is Atlassian's stance on that language taken from the internal guidance on the ToS. To be clear I'm not defending this stance as I think it is flawed. But I wanted you guys to know what Atlassians are told about it -

------------------------------------------------------

Section 3.3: Benchmarking Can you explain Atlassian's stance on Benchmarking?

Like many other software companies, Atlassian has this language in its terms to protect users from flawed reviews and benchmarks. By requiring reviewers to obtain Atlassian’s consent before publicly disseminating their results, Atlassian can confirm their methodology (i.e. latest release, type of web browser, etc.) and make sure they deliver verifiable performance results. Atlassian is not trying to prevent any users from internally assessing the performance of our products.

The language related to the public distribution of performance information has been included in our customer agreement since 2012.

Customers can obtain Atlassian's consent by filing a Support ticket. The Support engineer will then need to bring in a PMM for approval of the data/report.

------------------------------------------------------

That explains the stance but is not sufficient to justify it. It gives Atlassian infinite power to stomp on any benchmark that shows poor performance under a claim that it is flawed. It is also irrelevant that it's been in your ToS since 2012: Precedent or longevity do not make consumer-unfriendly restrictions acceptable.

This is implicitly recognized by allowing internal assessment: That assessment would be just as vulnerable to flawed methodology and therefore flawed decision making on products. If you were that concerned over such issues, you could issue further restrictions on performance assessment that limited such activity to be conducted only under Atlassian's close review or using your own mandated methodology. One reason you probably don't do that is because potential buyers would balk at those restrictions and either pass on your product or responsibly engage in due diligence and perform their own assessments regardless.

Further, the resources to do extensive internal assessment may be lacking in many organizations, which means your provision to allow internal testing is meaningless to many customers. As a result, the prohibition against public disclosure thereby deprives them of any way of obtaining objective external analysis.

You could satisfy your concerns by requiring that public disclosure be reviewed by Atlassian prior to publication. You could require an option for Atlassian to comment on the results with embedded notations without restricting publication itself. That would still be heavy handed but at least allow a reasonable amount of independent review of your performance.

Those suggestions in your last paragraph look very reasonable to me. At GitLab we explicitly allow performance testing as our ninth and final stewardship promise https://about.gitlab.com/company/stewardship/#promises But I recognize there is a trade-off and companies can reasonably but the balance at different points.

There is no tradeoff.

DeWitt clauses are corporate censorship and are 100% self-serving.

There is zero benefit to consumers having no benchmarks at all available for entire product categories.

There is an enormous benefit to corporations to be able to silence critics with the threat of bankruptcy via lawsuit.

This is big corporations using the law to bully journalists and citizens, nothing more.

Pretty much, yes. That's why I included an example of how they could allow transparent assessment while still addressing their stated concerns. Because it shows the stated concerns, in having a solution, are just a BS smoke screen. (Not that my proposed scenario is still the best case for consumers, just that there's a way they could allow public benchmarks without sacrificing their concerns)

> Atlassian has this language in its terms to protect users from flawed reviews and benchmarks.

The solution to lies is not to censor, but transparency.

Atlassian has all the resources in the world to answer any external benchmarks done by third party.

If you can hire an army of lawyers, surely its possible to have a full-time engineer running benchmarks.

It's pretty clear in context that they are discouraging detailed performance benchmarks as a proxy for reverse engineering. Not some kind of gag on complaining that "JIRA is too slow".

> protect users from flawed reviews

Perhaps California will address this problem by banning "performance benchmarking platforms" from listing evaluated products without an agreement from the vendor... [1]

[1] https://news.ycombinator.com/item?id=25601814

Reply on news.ycombinator.com