Scrumdog – a program to download Jira Issues to a local database | Hacker News Reader

Scrumdog – a program to download Jira Issues to a local database

whoek.com

149 pointsbosveld61 comments

Readers may also enjoy Steampipe[1], an open source [2] tool that uses Postgres FDW's for live SQL queries against Jira, Confluence, GitHub, GitLab, AWS, Slack and many more tools [3].

1 - https://steampipe.io 2 - https://github.com/turbot/steampipe 3 - https://hub.steampipe.io

There is also git-bug [1], an offline bug tracker fully embedded in git which can bridge to Github, Gitlab, Jira

[1] https://github.com/MichaelMure/git-bug

Steampipe sounds interesting (and will be looking into scrumdog too).

Is Steampipe a community + services model, or just open source + community?

The Steampipe open source project and community are managed by Turbot [1]. We also have Steampipe Cloud [2] in preview if you'd prefer a hosted version focused on teams.

1 - https://turbot.com 2 - https://cloud.steampipe.io

I built a very similar tool for my company to download a set of DataDog logs into SQLite. Products like Jira and DataDog have their own query engines for sure, but you really can't beat the capabilities of true SQL for deeper analysis.

link?

Wish I could! It's locked away in a private company repo.

Absolutely love this. I have a whole bunch of similar tools for importing stuff from different services into SQLite but I never got up the courage to take on Jira!

Here's my github-to-sqlite tool which imports GitHub Issues https://datasette.io/tools/github-to-sqlite - being able to query them in a local SQLite database is super-handy. I run a demo instance of the resulting database here: https://github-to-sqlite.dogsheep.net/

Heads up:

> Error 500 cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)

Thanks for letting me know - fixed that here: https://github.com/dogsheep/github-to-sqlite/issues/74

I had used https://github.com/go-jira/jira in my previous job. custom-commands is a very useful feature. I had built a set of commands to filter out issues for different use cases. The command line approach was way faster than waiting for the jira page to load and click the correct set of filters.

Just noting that you can use an EL(T) tool like Airbyte (OSS) for this as well! The integrations are already built - Jira -> mongo, Jira -> Postgres, etc.

https://docs.airbyte.com/integrations/sources/jira/

Yup it makes a lot of sense to use the already built tools for this.

If you're forced to use Jira, and setting up a local replica database is surplus to your interest / requirements, I can recommend the Jira Assistant chrome extension.

The licence is a bit wonky (source-available, basically) but the feature set fills a few gaps with the regrettable stock UI.

https://jiraassistant.com/ or https://chrome.google.com/webstore/detail/jira-assistant-wor...

You might also try https://www.beekeeperstudio.io/ or https://dbgate.org/ for working with databases (SQLite and others). I really like the UIs of each and they're both open source- supported by donations if you find either useful.

Missed naming opportunity: should have been “scrumbag”.

How about Jira-is-dog-slow-and-the-interface-is-garbage-so-just-give-me-the-data? Not catchy enough?

Presumably if you built a front end for the database you’d then have a not shit issue tracker.

if he sold it for high enough, could be a Scrumdog millionaire

Agreed, but i also think the -dog portion of the name likely refers to *fetching* of data...you know, like a dog fetches...well, i woud guess anyway. ;-)

"slumdog"

They are definitely hoping to make millions of dollars off this tool, and then become a...

I liked the approach to slowly narrowing down the language choice based on needs. A good example of finding the right tool for the job.

I wish there was a program to enter hours into Jira (and related tasks of course) and upload them.

We have to use Jira for logging hours at work now because they think that makes us agile. Needless to say the only agile related thing we do is logging hours in Jira so that's a complete joke. Our project management department is full of complete idiots.

But Jira is such a dog to work with. Especially if you only enter stuff for the sake of it. It won't accept "7h33m", no it has to be "7h 33m". For every change you need to refresh all pages before they're picked up. It's ridiculously slow.

A frontend where I could just dump my made-up hours that would then take its time to upload that shit to Jira would really help me to be less stressed.

Just to get ideas, what custom reports are users looking for that can not be built by Jira.

I've been wanting to suck in Jira tickets to my own database so that I can full-text index them along with code, docs, customer requests, Slack, etc. Then when you're looking for information, you only have to go to one place.

(My ulterior motive is to write a blog post that's like "how I used the product I make at work to index all information at the company". I can usually just do a linear scan across the backlog to find the issue I want, but hey, when marketing and software combine, I can't complain.)

If there's some service I can buy that does this, though, I'd rather use that.

I think this is more of a way to automate pulling large amounts of data from Jira rather than doing it directly from a dashboard. Also could be utilized as a backup repository.

In my case, I find it hard to query related issues of different types. It's kind of hard to explain unless you see how we use Jira, but I basically want to do joins. You can do some interesting things with the Structure plugin, or issueFunction() from the Adaptavist ScriptRunner plugin. The other thing I've done is to pull a few REST queries into a single-page app., but that takes more work.

You might also be interested in the excellent Bugwarrior [1] which syncs issues from a wide variety of issue trackers (JIRA is amongst them) [2] to your Taskwarrior [3] database.

[1] https://bugwarrior.readthedocs.io/en/latest/

[2] https://bugwarrior.readthedocs.io/en/latest/services.html

[3] https://taskwarrior.org/

This is what I need but for Mac! Any idea of alternatives? I have multiple Jira instances I want to combine -- literally can't find an app that does this.

Author here - Yes that is exactly what it does. I'll compile it for the Mac later today and load it on the Scrumdog.app website. Thanks for your comments.

Wow thank you so much!!

> Any alternatives?

You could use Steampipe w/Jira plugin: https://hub.steampipe.io/plugins/turbot/jira

My manager a few jobs back did something like this, but just using Microsoft Excel with Power Query. No need to even install sqlite. Normally, that probably isn't much of an obstacle, but this was in a classified environment where it was nearly impossible to get the local IT admins to install additional software on your workstation. Every corporate Microsoft box is going to already have the Office suite.

I'm curious about this. I would say I'm fairly versed in Excel but haven't dug too deep into Power Query / the data model. If you wouldn't mind, how is this done?

Looks like it's documented on the atlassian website: https://community.atlassian.com/t5/Jira-Core-Server-question...

I didn't get back to this in a timely manner but I wanted to thank you.

Is possible to create single file applications in C#

https://docs.microsoft.com/en-us/dotnet/core/deploying/singl...

While it is possible, it is certainly less than stellar compared to the other languages that were reviewed. Granted my experience was 1 - 2 yrs ago with .Net Core 5. I recall having to deal with setting different temp directories each time I wanted to run a another instance of a 'single binary' because the first thing it did was unzip the bundled runtime into the system wide TEMP directory. It certainly was a single file app but behaved more like an installer.

Whoever came up with that obviously didn't understand the why of single .exe applications.

Maybe someone can build a functioning UI for jira with this because Jira is terrible.

Jira's querying and reporting facilities leave a lot to be desired. I have some scripts to pull the data down into a sqlite database so I can query it much better.

If program/project managers having coding love/passion, a lots of innovation could come around Jira

FYI - This type of workflow automation and connecting to Saas product APIs and moving data around can also be easily built using a no-code RPA tool like PixieBrix[0].

[0] https://www.pixiebrix.com/

Looks like a cool project!

I like the name. Scrumdawg would have been better though.

Atlassian may want to provide SQL interface to Jira data. JQL is limited in many ways.

This is my favorite limitation: https://jira.atlassian.com/browse/JRACLOUD-21372

Wow. Going on 12 years to get exact text matches working; that sounds crazy.

There's an sql database backing Jira, but it looks rather ugly. I guess that's why they translate it.

I think it's more about access control. They don't do any access control at the db level afaik.

No explanation necessary. Anyone that has followed Atlassian in the news knows that downloading Jira Issues is an excellent idea!

Are you suggesting they’re going out of business?

Until the current crop of managers retire, Atlassian will still have business. Some don't know anything other than Jira, so will recommend it on corporate purchases.

Probably a testament to performance and reliability of the hosted offering (we all remember the multi day outage not long ago).

Are you referring to the April outage?

Technically 14 days is multi day, but when it can be measured in weeks, you might as well.

> Technically 14 days is multi day, but when it can be measured in weeks, you might as well.

What does that mean? Two days is multi day. Even 25 hours is multi day.

I believe they’re saying that we might as well say “multi-week” rather than “multi-day”, given that it was 14 days!

Multi day seems to imply a time period of 2-5 days, not 14…

There were zero issues with performance/reliability that caused the outage, they just deleted hundreds of customers data, and had to manually restore the data due to lack of automated restore scripts.

Reliability is measured from a customer's perspective, not from a geeky vendor-oriented perspective of a backend service. When the vendor deletes data, the vendor's reliability quotient decreases, because that data is no longer available to the customer.

Once again we learn the hardware that accessibility is not the same thing as reliability.

Apparently they've never made a profit...

Why are you hosting compiled binaries on GitHub and no source code...

Reply on news.ycombinator.com