Hacker News Reader

Comment by Jedd | Hacker News Reader

We're using Jira, Confluence, and BitBucket on-prem at our DC now, on some pretty sizeable VM's with good network connectivity back to head office.

Despite that, it feels very slow. Not the occasional 10+ second page load, where you've accidentally clicked something that happens to generate a cascade of cache misses. I can forgive that kind of infrequent performance hiccup.

But rather, regular usage, especially of Confluence and Jira, just feels sluggish when doing bread and butter actions - adding a comment to a ticket or reloading a kanban view, loading a new page or opening the search dialog on Confluence.

Yet, everything I've heard about the responsiveness of the 'cloud' offering is that it drives people to move to the on-prem server version.

View on HN

Atlassian does not provide particularly good guidance on how to get their tools to perform well. The first big issue for any moderately busy system is that you need to dedicate huge gobs of heap memory (think 16G+) to the tool. After that, they suffer from poor schema design - there are tables that are literally missing indexes on commonly queried columns, and some columns that have delimited values embedded in them.

It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem. The incentive would always be for them to try to minimize the amount of memory every instance gets, which does NOT maximize your individual performance. Shared resources will also make performance more variable and "noisy", making it more difficult to narrow down specific causes. It's hard for me to see how "Java" and "cloud" go together.

> It's hard for me to see how "Java" and "cloud" go together

AirBnb, LinkedIn, Netflix, Twitter, Foursquare, Sony, Shopify etc.

Most of the top tier websites use something on the JVM e.g. Java, Scala.

> It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem.

I always wondered that, too! Like, you'd think that if you have devs building a product and a team running a hosted instance of the product in the same company, that they'd have a very good feedback loop and make it easy to run the thing and to do it well. Although, I wonder if the answer is that it's not the same product - like with bitbucket, where the thing they run in the cloud isn't the same thing as they sell for on-prem. Or even a lighter version (BB, AFAIK, is literally 2 separate codebases), where they cut features in order to make the cloud version work.

> It's hard for me to see how "Java" and "cloud" go together.

Nah, it's not Java, it's the specific product. I've worked at a SaaS shop that did all Java, and while the pathological cases are comically bad, it's actually fine most of the time IME.

They don’t.

They forked the code base around 5 years ago now. Cloud/on prem.

Cloud is or was multi tenanted, you didn’t get your own isolated server, you got routed to some compute that would then pull data at request time based on your tenant id.

With a hard fork of two seperate product streams it was inevitable at some point maintaining feature parity between products would get to much.

Java and cloud go together quite nicely. Backend services, a JVM that has now be tuned for many many years by lots of clever people. If you have a long lived service it works really well. It works better than your node js app running a single instance on a multi core server with the event loop getting blocked.

Makes sense. SaaS and desktop have subtle different needs that cause widely different problems to solve.

The amount of effort I would fathom needed to maintain a single code base that's architected to do both would be exhausting.

I'm disappointed they don't just come out and say it, i.e. play open cards and help your customers understand

> It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem.

The homedir for our on-prem Confluence instance is >120GB and the MySQL dump is 26GB uncompressed (2GB gzipped).

Not sure how that import would/will go.

If Atlassian gets paid by month to store it, it seems ok. Most images seem to be in AWS anyway, so they probably don’t back them up. The DB can get big but is that a problem if they are paid to maintain it?

>It's hard for me to see how "Java" and "cloud" go together.

Netflix.

> It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem

Yes, which is why the atlasaian cloud is no longer a collection of individual customer instances. More details: https://youtu.be/0N4KknY_zdU

> It's hard for me to see how "Java" and "cloud" go together.

Why? From its release, Java has been used for SaaS. SAP runs its cloud services using Java and this includes subsidiaries Ariba, Concur, and SuccessFactors.

> It's hard for me to see how "Java" and "cloud" go together.

Given the number of companies known to use Java in the cloud very successfully, this is a “Citation Needed” moment.

I've no recent experience of the on-prem version, but nowadays I have to use the cloud version and it's painfully slow. One common problem I encounter is that after a page loads I might want to search for something in the search box. I'll click it and start typing ... then at some point the rest of the page has loaded including the stupid key bindings which do various random actions on different keystrokes. I don't know why they have this, it's infuriating.

Oh and they keep updating the UI which involves shuffling things around and every new version is slower than the last. If you're running on-prem you at least get to decide when to update and notify everyone accordingly. We'd just find out one day that everything has been moved around and learn the new locations of different features.

Sorry I'm really just generally ranting about Jira now but honestly I find it so infuriating to use. Confluence as well, how did they manage to make wikis suck so bad?

Agreed. Hijacking ctrl-f in my browser (chrome) is user-hostile, given searching on-page is a reasonable expectation for any web app.

I use Confluence regularly over a high latency (600ms) link - and this has the surprise / delayed page load you describe, with the newer search features refusing to action your search request until it's loaded & rendered a large overlay on the right-hand side of the screen.

These two attributes make are hugely frustrating, given looking for information is the key function of a wiki.

Yep:) At a previous job, we also did Jira/Confluence/BitBucket self-hosted, running on what should have been nice hardware, and spent months chasing performance issues, including a charming issue that was so bad that it broke jira for the entire company every... Monday, I think? When you have a well-used standard operating procedure for restarting Jira just to unfreeze it, you have a problem. We were pretty sure it was somebody running a weekly report with some monstrous query, but 1. not even Atlassian support could seem to tell us how to find such an event, and 2. I don't really care how bad the query is - if you can run a query and take down the system for the whole company, then the software is bad. And honestly, even "normal" performance was poor - I seem to recall that it at least acted like the web UI was the slow part; running a local CLI client against the API was faster.

Well they certainly noticed companies running their own Jira hurt the image of Atlassian, and stopped offering the service altogether. It would be a knee-jerk reaction but possible.

The performance on the cloud version is awful. When I previously self hosted it was a miserable management experience, but it definitely felt snappier than the current cloud version does.

Jira is probably the slowest piece of software I use in my job. I can't believe there's an interstitial loading screen with a status bar for cloning an issue.

I don’t share the same experiences with slowness (maybe we do something right on the instances we manage). I do however experience slowness in cloud. I assume they increase cloud resources in some kinds of steps that make them appear extremely sluggish.

Sorry to hear it's been a frustrating experience. I'm a PM for Confluence Cloud and we're always trying to make it better. Would you be willing to share more specifics, such as: - Pages with content X are the slowest - Trying to do A/B/C is annoyingly slow - etc ?

You should be able to experience the slowness for yourself by spending a few minutes using your product. Then try something like notion or pretty much any other web app and notice how your blood pressure drops and your stress levels decrease.

It is not a particular area it is the whole thing. Every interaction makes me want to gouge my own eyes out. It's a glorified text area with a menu, why should every click cause dozens of requests that take more than 10 seconds to complete?

I can only conclude Atlassian engineering is utterly disfunctional for this to have gone on so long unimproved.

Hi ratherbefuddled,

We definitely recognize we have a lot of work to do on performance in general, and we're definitely trying. "fixing everything" is definitely part of our plans, but we'd also like to "fix whatever users find the most frustrating".

If any specific items stand out as the most frustrating please do let us know.

I think you are taking entirely the wrong approach here.

Pressing refresh on the front page causes TWO HUNDRED AND SIXTEEN requests for about 7MB of resources.

The only meaningful content on this page is a list of 20 or so links.

You don't need customer feedback here, you need some basic engineering principles and a tiny bit of pride.

Hi ratherbefuddled,

Thank you for the insight. We are definitely working on generalized efforts to make 'most/all/everything' faster (including speeding up the app overhead), but those types of projects unsurprisingly take a little longer to finish.

On the other hand, specific user feedback might be 'the most annoying thing to me is that pages with Links load slowly, I need the links to load earlier because of X', or 'I need inline comments to load faster than page comments, because of reason Y'

-> this type of feedback could give us something we can fix quickly and alleviate user frustration sooner. Hopefully the intention makes sense.

Of course, in general, we're also always looking for any feedback that can help us improve Confluence!

Your response seems to show the problem. Trying to add little quick fixes might sound reasonable from a business perspective, but it's not going to help on the tech side. If anything it's going to make matters worse.

Hi pintxo,

Thanks for the consideration - we are working on big fixes as well, but can't share information about those until we are closer to shipping. In the mean time we're open to suggestions and feedback of all types.

Where is the data hosted? Somehow it feels like the devs are located close to the data and don't notice the lag. And for me in central Europe, some requests must travel across the world and back.

Hi sheeeep86,

My original post did not mean to give the impression we think Confluence is fast - we do recognize that we have a lot of room for improvement.

The reason we're asking for feedback on specific areas that are annoying is that while we can work on making 'everything' faster, knowing which specific items may be of the most concern may allow us to focus fix those items first.

If you do have any feedback around such specific items we would be happy if you're willing to share it.

Surely you have metrics?

Hi vasco,

We have a lot of metrics, and we recognize that Confluence is not as fast or as good an experience as it should be.

However nothing beats user feedback - hearing specific feedback from users may provide us with specific areas that we can focus our work, on top of the work we're already doing.

If you have any feedback like that, we're hopeful you're willing to share it with us.

I agree with you on user feedback being valuable, but not to figure out what's slow. You should be getting that information accurately through metrics and know what to prioritise without having to speak to anyone. Reserve questions you can't get answers for otherwise for user interviews.

If your avarage is always 10 seconds, 7 is fast!

While we had very similar experiences - only used JIRA but the performance was borderline-unusable - I've heard reputable sources say that configuration plays a tremendous role in that. I'm inclined to believe we just hadn't configured it "correctly," but the fact that you can destroy performance by casual misconfiguration is itself a pretty big issue.

I've heard the same about cloud performance, which seems to be an argument against it being simply configuration, but who really knows?

When I used JIRA Cloud about a year ago I measured 30s+ load times on every single page. For a nearly-empty instance with no customizations, just their scrum template.

Yes, it's like that out of the box. The next gen stuff offers no improvement whatsoever.

My experience with Jira makes me think that extensions are a big portion of the problem here.

Confluence just sucks though.

What about Confluence sucks specifically? I share your view but how could it be better?

1. Performance. It is incredibly slow without customization.

2. Editing is a clunky process. Not only is the UX subpar, it never loads all the possible widgets you can insert, making dynamic content a pain.

3. Awful search. It always manages to find internal process documents from 2014 and not the service pages I actually want.

4. Tough to reorganize content. Moving a page around is just hard enough that people tend not to do it, resulting in really poorly organized content.

Honestly, I’d be happier using notion or just GitHub wikis. But the main benefit of putting our docs at wiki.$COMPANY.com is that I can guarantee discoverability for other teams, so we’re kinda stuck with it.

Reply on news.ycombinator.com