Hacker News Reader

> You could argue that I'm using the wrong tool for the job.

Without comparing Excel vs LibreOffice vs etc, I've found the category of spreadsheet software to be immensely helpful for analyzing, aggregating, visualizing and reducing small datasets.

You do have to end up falling back to python scripts using pandas or whatever if you need to run jobs that need to take data from one DB and put it into another DB or something like that.

But if my output is basically a reduced set of tables, series or graphs for presentation, spreadsheets are an immensely useful tool.

What I've specifically done is have one sheet of the spreadsheet represent the "raw, unreduced data", and write up formulae for aggregation in other sheets, based on the raw data and intermediate aggregations. Starts becoming less ideal when you have 10 or more sheets, but for simple cases, it's very much underrated by us engineering folks.

I always found that excel can be as simple and as complex as you need it to be. You can use it to write your shopping list (ok, not ideal), or a small budget, and you can ramp it up for really complex simulations. Once you know vlookup, indirect/match, sumif/countif and very few other formulas (for example text manipulation to clean and harmonise data inputs), you can do really interesting stuff even with datasets of thousands of lines. I found that often in excel the most difficult stuff is not the back-end (data analysis and simulation) but the front end (i.e. giving the user a clear interface for inputing data and conditions in the model, and visualising outputs, for example with buttons, constrained inputs, conditional formatting etc.).

Why not ideal for shopping list? When we go skiing with a couple of friends, we setup a long shopping list in excel, there each row has the name, category, amount and amount unit.

Before heading into the store we have a Pivot table sum it all up, group everything by category sorted by the order those categories are encountered when walking though the store.

This way it takes 2-3 people less than an hour to shop for a 4 day trip for 10-20 people. One holds the list, and then 1 or two runners go fetch stuff.

It’s been perfected over the years:)

Hats off for this. I was thinking more about as a substitute of a simple paper list (or if on mobile, just a list on whatever default note app is you have in iOS or Android)

>What I've specifically done is have one sheet of the spreadsheet represent the "raw, unreduced data", and write up formulae for aggregation in other sheets, based on the raw data and intermediate aggregations

This is exactly what I do. When it works, it is just way too fast for ad-hoc reporting to consider doing anything else. Pretty much all of my first draft reporting is built this way, so i can cut through the back-and-forth of management change requests before going through the trouble of setting up a full thing.

It is amazing how much you can accomplish with appending a couple columns to your raw data for categorization and then pivoting.

when im in a rush and need another layer (pivottable of a pivot table), I even set up formulas to reference a produced pivottable and then pivot on the formulas. You can extend the formulas far beyond expected use-cases and filter out the irrelevant rows later

> LibreOffice absolutely chokes when I try to do any filtering or calculations.

File a bug. https://bugs.documentfoundation.org/

It's easy to complain in an HN comment, but a clear description of the test case would certainly help with getting the issue resolved.

I wish this comment was banned from HN. Every time someone says that some free software is in any way flawed there's always the completely useless reply "well did you file a bug?" or even worse "it's open source - why didn't you fix it?"

I'd hope it's obvious why these comments are bad, but I guess not so:

* Not everyone has time to fix a bug or even file one.

* The ability to fix or file bugs doesn't mean that the bugs don't exist!

* The ability to fix or file bugs doesn't mean that one shouldn't talk about the bugs.

* The possibility of fixing a bug doesn't help if you don't have the time or ability to fix it and actually want to use the software.

* Not everyone is able to fix all bugs

* It's not nearly as easy for people outside a project to fix bugs as it is for people familiar with the code.

* Everyone already knows it is possible to file and fix bugs in open source software so you're not adding anything to the conversation.

Specially because you usually have to jump through a lot of hoops to file a bug. I remember some time ago trying to file a bug for some major open source software. It was so painful that I gave up before finishing.

Without context, it was likely painful as they need that information to have any chance of fixing the bug.

I don’t mind collecting useful information, it’s the having to create accounts on some hard-to-navigate project-owned bug tracking thing that kills it for me.

There's a word for people who expect to get things for free without giving anything back to the authors or community that made it possible.

Could you point me at the paid for equivalent for excel on Linux? If he donated even a thousand dollars he wouldn't get the product he can get for almost free on windows.

Doesn't Excel run under Wine? That might actually be the better option.

(LibreOffice user here on Windows, donated in the past, but I find LibreOffice Calc really unpolished. It puzzles me because NeoOffice is really quite good on the Mac. I should probably just switch to MS Office.)

Certain versions yep :) . Although I'm not sure it's flawless. Crossover office is best if you're going down that route, or just put windows in a VM and run it there if it's crucial to your work. That's what I do.

There's also a phrase for people who give things away for free and expect something in return.

This just turns Foss into the inferior but free option, and I don't think that was the intention.

And since you mentioned you'd pay for Excel, please contribute a one-time life-time $50-$100 bounty to help fix that performance issue.

Would this hypothetical fix include a performance suite, and a regression testing system, to ensure this task stays fast forever?

IME, systems that don't focus on performance as a primary metric tend to slow down over time as they accumulate features. Someone who is willing to buy Excel ($130) may not be willing to pay a $100 bounty every time some other programmer makes LibreOffice slower again.

I googled for LibreOffice performance regressions and found several, going back many years, and they tend to include a one-line fix. For all its faults, I'm sure there's a team at Microsoft that knows every time Excel performance regresses.

Good point. How can one prevent this though, for any free/open source software? Introducing issues (intentional or careless) to keep some funding going.

Why is that? Nobody ready to improve LibreOffice Calc? Calc is not good enough to start improving on?

Do we need a list of missing features? I would also add:

* Copy paste from Writer does not work

* Summing a column (not a range) isn't possible

* Type detection is not as good

* Auto-update doesn't work at all

Copy and paste on linux when using multiple apps and remote vms/vnc/rdp etc, is a pain. Depending on what app you use, its looking for which clipboard buffer and what commands to copy/paste. Mixing Os's and apps, is a clusterfuck on a multiheaded desktop with excel in a windows rdp session and libre office in another, then throw google sheet in because work decided to switch to google office at 1/2 the price of office 365 per user.

Sigh.

I've just pasted some data from Writer into Calc.

Could you elaborate on your column summation problem please. =sum(E:E) works as does the summation button.

Don't understand your third point but if it is text vs numbers and dates etc then I've had plenty of fun with this in Excel as well over the decades.

Auto-update - please elaborate where it fails

I've just pasted some data from Writer into Calc.

Try to paste a table from Writer into Calc so that the structure is converted to rows and columns (like Word and Excel does).

Could you elaborate on your column summation problem please. =sum(E:E) works as does the summation button.

~I get Err:522 when I run =SUM(E:E)~

It works!

Don't understand your third point but if it is text vs numbers and dates etc then I've had plenty of fun with this in Excel as well over the decades.

Me too. It's annoying in Excel, but worse in Calc.

Auto-update - please elaborate where it fails

The built-in auto-updater doesn't so much fail, as hasn't ever worked at all (Mac, but I think Windows as well). iirc I get a %%PERCENT%% placeholder sometimes, and the rest of the time I have to download the update from the website instead. Same on several machines.

I can paste a table into Calc OK. It was a simple effort with a,b,c,d in row 1 and 1,2,3,4 in row 2. The numbers came in as numbers OK. Paste as unformatted text is rubbish (single column of data) and I think I might file a bug for that.

Sorry, I get you now about auto updater - software updater not [Google: Excel auto-updater]. That's not really a thing on Linux because when you update the system, the whole thing updates - apps and all.

The pace of development in LO is absolutely amazing but there are some horrendous bugs with near immortality status. If you can put up with them then use it. No software is perfect but LO is moving at increasing speed towards the light.

Isn't that the job of brew or whatever package manager you're using on apple?

I absolutely hate type detection in Excel. It's at least comparable to type coercion in Javascript, maybe even more capricious. I know the type thank you very much.

I have been using https://openrefine.org/ for interacting with large csv files, and it has been amazing. It supports the normal filtering / transformation and other import / export actions. Plus it's open source.

I run LO on this laptop under Linux. I've just opened the member's table which has 74,853 records in it. It's a 54.8MiB FoxPro database and took about 10 seconds to import. I've put an auto filter on the whole thing and a search for France took blink. I created a pivot table in seconds showing counts of sex by citizenship (roughly 10% of UK climbers are women.)

This is a fairly beefy machine but over two years old so nothing particularly special nowadays.

>You could argue that I'm using the wrong tool for the job.

For ad hoc filtering, calculations, merging, analyzing tabular datasets a dedicated tool such as our EasyMorph (https://easymorph.com) would do a better job than Excel in the vast majority of cases. It can handle millions of rows, keeps data in memory, has a wide range of available transforms and built-in charting. We target heavy Excel users and it works quite well. Disclaimer: I work for EasyMorph.

I think both Excel and Libre Office have their place. Excel has the power and refinement for professional users such as accounting, actuarial, and the commenter's data analysis.

Libre Office is for home use: tax prep, budgeting, graphing science projects. That's what I use it for, and it's perfect and free. That might be unfair to all the hard work that's been put into it, but if you don't push its limits, it works great.

So I guess I am arguing it's the wrong tool for data heavy jobs.

Always felt it was a shame that scheme in a grid was abandoned (and last I tried getting it up and running I gave up;I believe the only viable option is to change the gui library - and that didn't seem trivial at the time. I suppose these days one might rustle up an ancient version of Slackware in a docker container...).

http://siag.nu/siag/

Anyone know of something similar, but more up to date?

> Excel is a piece of software that I would absolutely not hesitate to pay for... but I'm unfortunately stuck with LibreOffice.

But... according to the rest of your post, you _did_ hesitate to pay for it and are finding that LibreOffice doesn't fit your needs. So why not buy Excel?

From the parent:

> At home I run Linux... where there is no Excel

You can run it in a VM, but at that point you're already settling for a sub-optimal experience, so why not try a alternative that's free and packaged for your OS?

I had the same issue with LibreOffice choking with big datasets so I've started using SoftMaker PlanMaker for loading anything with more than a few hundred records. It's works just fine with documents with tens of thousands of rows and it allows me to do the - admittedly very simple - calculations and charting I need.

The whole SoftMaker Office suite is pretty nice. Not sure where it's on feature parity with Microsoft Office, but the price isn't terrible (something like $20 a year for the paid version, or free if you don't care about the paid features) and it does a great job of reading Office files.

have you tried gnumeric? I prefer gnumeric to open office's spreadsheet when it comes to open source.

gnumeric is great for simple tabulated data as far as simple charts etc, but the advanced features that spreadsheet lovers use aren't available.

Have you tried WPS Office? It kind of works for the things I do, but I am not an advanced Excel user.

I would be very curious about your experience as you certainly tax the software much more than I do.

  LibreOffice is just so painful

You were running it on a different OS and different hardware. LO vs Excel was not the only variable.

* You could argue that I'm using the wrong tool for the job.*

You are. You could try this reporting tool instead: http://pebblereports.com it has an Office like interface, works really well with databases and has performant crosstabs.

They said they're using LibreOffice because they're running Linux. That tool you're suggesting is Windows-only.

If they were using Windows, they could save a couple hundred bucks and just use Excel.