Automate Excel with Python: From manual grind to one-click workflow | Hacker News Reader

Automate Excel with Python: From manual grind to one-click workflow

nostarch.com

55 pointsteleforce29 comments

These sort of things always seem to assume a fairly relaxed software environment.

In practice I’ve found the big corporates try hard to keep their excel files with financial data and their Python environments with pip & all those associated risks far apart. That’s if pip works at all & isn’t caught by a firewall

Most corporates, last I knew, didn't use Python outside of IT. Devs outside of IT would be using VBA.

However in Financial companies, Python and Excel have always been used together by devs and also quants.

And they tend to use Anaconda, and also like all their other package managers, they would host an in-house package repository and block the public one. That way only approved packages are used, and they only update packages as needed.

Many though have a policy of minimising Excel and rolling out formal platforms whether in-house or off the shelf, as Excel is regarded as a ongoing risk of in-accuracy as full editable at all times, lack of git/version control and so on.

>Many though have a policy of minimising Excel and rolling out formal platforms whether in-house or off the shelf, as Excel is regarded as a ongoing risk of in-accuracy as full editable at all times, lack of git/version control and so on.

Can confirm this practice in place at fortune 50 financial institutions. One in particular calls it "End-User Computing System Risk", meaning the end user created a business-critical "system" (i.e., a complex Excel file or Access database) on their own computer.

This. The amount of begging and pleading I had to do to get basic python access was insane ( long story, but boss did not believe in it ). In some organizations, things change slowly and even if I could work around some of those restrictions, doing so does not really do me any favors.

This is just an advertisement for a book. And a terrible advertisement. No sample chapters to evaluate the book.

Yes it’s an add for a book but it does provide a sample chapter:

https://nostarch.com/download/samples/automate-excel-with-py...

Yeah, a $50 book (physical) or $40 ebook

With normal inflation, a $30 O'Reilly book in the year 2000 would be about $58 today.

We all have to start somewhere. It does not invalidate your point that it is an advert, but I can't categorically state it is terrible.

Looking at the content, if you are familiar at all with Python and basic programming, this provides very little new. I sometimes have to stuff massive Excel-abominations with 50k+ rows and rip data I need out of them with Pandas, but it only requires reading Pandas documentation (which is very good) a bit.

But perhaps this might be good if you know no programming and want to make your life easier.

Yup... I wrote this book for folks who love Excel but no programming experience.

Been working on a dashboard that takes in a bunch of different public data sources in different formats and pandas has truly been a godsend for this.

I've got csv, txt, xlsx in all different shapes and sizes and with just a few settings I can go through them quite easily and very fast as well.

Depending on the size and the transformations you are doing, polars is worth checking out. Syntax is a bit different from pandas but the performance is really nice.

My biggest bottleneck right now is upload speed but I'll keep it in mind thanks.

I am guessing from the TOC that they are using pandas. My team quickly ran into memory spikes when multiple programs using pandas would run. We have since migrated to using ibis with a duckdb backend to smooth out the memory spikes.

Yeah... I wondered about the title "Automate Excel with Pandas" but Excel users would have no idea what I was talking about. Those who never leave the safe harbor of Microsoft will have heard distant rumblings about Python but not specific modules. Forgive me for sort of equating "Python" with "Pandas" but if it gets more folks using Python then I hope the ends justify the means.

in a world of ai agents, python is just an implementation detail that you don't need to know, between you and your data/business task

You are quite possibly right but I can think of several reasons why total non-programmer might want to learn some basic coding. First, you learn the exact discipline and kind of thinking that improves what you ask AI to do. Second, if you are using AI to create logic in a corporation, you will need to defend the script when internal audit comes a-knocking. Finally, you will need to know a little code to debug or deal with edge cases. It's like Google can help you translate and survive a one-week visit to Paris but you really need to know some actual French if you moved there for a year.

Yesterday I asked an AI to generate a report as a CSV. But then I wanted it split into multiple sheets, and to add some formulas, so I asked it to create an XLSX, and it did (with Python). I'm imagining Microsoft embedding an AI agent and Python interpreter in their tools... no more need for a software dev, excel expert, or technical book author

I prefer DuckDB

How do you automate Excel with a RDBMS?

DuckDB isn't an rdbms, bit the answer is the same: ODBC or OLAP. There's a "data connection wizard".

Terrible title. Nothing to do with automating excel. From what I can tell it seems to be about ingesting spreadsheets into panda (and incredibly narrow use of Excel) and working outside of Excel.

its just a really badly written ad

Titles are incredibly hard to get right... but the spirit is about automating what you do with Excel. To the absolute beginner, writing a Python script to ingest a spreadsheet into a dataframe, doing all the manual copy/paste blah blah blah, and the saving back out to an Excel file DOES qualify as automation!

What are best practices for using Excel as a front end for python tooling? I’ve got a use case where the business users are maintaining a complex spreadsheet and we need to hook some genuine optimization into it. It’s all fine and good if you assume the people will use the template perfectly, but hahahahahaha

There’s probably some ideal blend of locking regions, in-excel validation, in-python validation, and clean separation of human inputs and machine outputs. Has anyone figured out what that is already?

Many negative comments here stem from readers applying some specific corporate contexts and dismissing the book's use cases. That misses the point. This book isn't an advanced automation manual for 2026. It’s an excellent book for beginners who want to learn how to automate some tedious work.

Yes, actual programmers don't need this book. But their non-programming roommates or relatives might!

The word "tedious" totally jumped out at me! The vast majority of corporate desk jockeys (like myself) use Excel as part of a tedious daily workflow. Python is the perfect de-tedious-ification tool. And easy to learn to boot.

"Stop wasting your time on Excel and waste even more coding an Excel parser in Python!"

Reply on news.ycombinator.com