Learning to code kept me sane when I was a diplomat

The article shows an interesting illustration of the disconnect between programmers and normal computer users. Like the author says at the beginning, any experienced developer knows that automating interaction with a GUI is a crazy idea. It's a painful, fragile way to move data around. But to the author and anyone who's done data entry, it's an improvement to their workday that demonstrates the power of programming.

So much programming instruction seems designed to teach people who already understand programming how to solve problems that nonprogrammers wouldn't even understand. A tutorial that started with automating tasks like this might be more effective than all the ones that start by demonstrating how to do basic arithmetic in $LANGUAGE and go on to a detailed explanation of $LANGUAGE's object system.

This is a really great post. For me, the recurring theme is knowing what's possible.

From a class he took in high school, Vik knew that some sort of automation was possible. That spurred some reading and exploration, which led to some success.

Success led to thinking about other opportunities for automation and more exploration of what's possible, which in turn led to more success.

To me, it absolutely doesn't matter that things could have been done in other ways. Nothing to cringe about here. A developer can only use approaches that he/she knows or suspects are possible.

In the day-to-day work of a developer, knowing what's possible is perhaps the most important thing. Anything else can be googled.

This is a fantastic article and illustrates a HUGE problem in government systems, where data gets aggregated, but not associated.

Oof. Had a hilariously similar experience doing data entry at my first internship, which was at a financial services firm.

I knew nothing about programming, but I had taken an intro stats class that had a few assignments in R. A super simple R script did half of my day's work in seconds (plus some debugging because of course I did next to no error handling).

In retrospect, I've realized that if I knew a thing of two about regular expressions at the time, I would have been able to automate my whole summer internship.

So it goes.

I haven't seen much discussion of a more ideal way to solve this problem (to say: as a one-off like the author was doing, not reusable software), so I thought I'd just think about how I'd do something similar.

Luckily there are libraries that are great at interacting with a web page or multiple pages at the same time without needing to actually have a browser instance open. There's selenium and its various scripting environments, which are meant for browser testing but work just fine for many automation tasks: http://www.seleniumhq.org/

And, newer, are headless technologies like phantom.js: http://phantomjs.org/ and Dalek.js: http://dalekjs.com/

This is SO much easier than using Javascript to find purple links on a page, and so much more reliable.

He could have looked up each link on DB1, then used the data to look up the DB2 information.

In any case, what he did saved him hours and hours of time and started him on a lifelong interest in programming.

Also, obligatory relevant XKCD's: http://xkcd.com/1319/ & http://xkcd.com/1205/

This is golden. I've wondered more often than once myself what would it be like to have a "normal" office job.

For instance, take some city department or some big firm where things generally flow at a slow pace and require a lot of time spent on work items, doing possibly repeated tasks each week. What would a programmer do? Exactly as described in the article:

- learn what kind of tasks the job consists of

- identify repeated actions and patterns in work

- think of a way to automate the easiest parts, still requiring human to finalise the task

- gradually learning more and automating the harder and harder parts and building tools bottom-up based on earlier blocks

- finally ending up with a software mechanisation puppet that I can instruct and program to do my work

- gradually spend first some of my time doing the automation and as it would slowly take over I could devote more and more of my time automating the rest, and when all is done, I would feel the job is "completed" and get bored.

This is something that feels so natural to a programmer: I have dozens of little tools to aid in doing my programming job at work. The goal of a programmer is to program anything that can be automated and only work himself on the hard tasks that can't.

It's intriguing to imagine how you could apply those skills outside programming jobs.

Love it. I started my career in International Development and one of my team's first projects was figuring out how many miles our organization's 1000+ staff flew in a year (why is a separate story). The travel agent could only provide records in PDF. 1000's of them.

The intern I shared an office with was assigned to go through each record, type the origin and destination airport codes into an online flight distance calculator, and enter the results in a spreadsheet. After two days of watching this I volunteered to take over and spent a few nights learning enough Java to read/write CSV and make the HTTP request. Saved days of pain for the intern and was a satisfying accomplishment as a novice.

Brings up old memories reading this. I remember my first job about 20 years ago at NEC as a data entry operator. It was 8 hours each day of taking a bunch of mal-formatted CSV files from some reports the previous day and importing it into excel and drawing graphs. I was just getting into learning C at the time and ended up writing an importer (parsing and formatting in C and ODBC to write back to excel along with excel macros to draw graphs). Very hacky and ugly but got the job done. It got the 8 hour job done in about 30 minutes - most of which was detecting patterns in the input and adding rules for those, which was fun in itself.

Before you knock on "writing code that clicks things" remember that on Windows clicking things is the primary if not solitary interface to most of the system. Even the administrative interfaces are secondary to the visual interface. A lot of programs don't offer an API available to outside processes.

The most important thing about any program is correctness: does it work and do what it is supposed to do. The only thing cringe-worthy about having to script GUI interactions is that fact that the operating system and environment requires it.

Why has no one mentioned Autohotkey? (http://www.autohotkey.com)

It was practically made for this use case, and has an active community behind it. It even has its own script editor (SciTE4AutoHotkey) and was recommended as a Kevin Kelly-approved Cool Tool (http://kk.org/cooltools/archives/4147).

My go-to program for my desk job.

this reminds me of when i started to code - find the first solution to the problem that comes to mind and hack it until it works, even if was something as abstruse as manipulating the UI. of course the outcomes were incredibly fragile and broke at the first possible moment, but in the end they still did the job they were supposed to do most of the time.

today i couldn't do that anymore; i've grown too uptight to try any of those inefficient "solutions". i guess i'd rather give up and admit defeat if there were no APIs or passable hacks (i'd say using phantom.js would be a passable hack) i could work with than solve the problems this way.

i'm usually unable to follow the advice given by the xkcd automatisation timetable described here: http://xkcd.com/1205/ "is it worth the time?" - if a dirty hack is destined to run more often than once, i just have to make it passably elegant, readable and efficient. i'm sure nine out of ten times the effort is pretty much wasted.

i guess i've just grown old.

> I decided to write a program that clicked on things.

I cringed a little. I'm glad he's learned enough to know now that this is not the right approach.

The biggest problem is not the lack of code and automation.

The biggest problem is that apparently Guyana allows the US embassy to search their criminal records system:

> Search a different database with a separate program (“DB2”) for criminal and other history

So either Guyana allows the US wholesale access to its criminal records system or the US systems gather this data more or less illegally from somewhere.

edit: downvotes? seriously?