Frustration project: Automate data entry into PeopleSoft with Selenium

THANK YOU for giving us Selenium. When I first ran my script and saw the browser pop up start "running on it's own," I was pretty wowed.

Selenium is amazing and saved my sanity when I was fresh out of college in a boring office job with terrible software. Thank you for it!

Thank you for starting selenium. It has helped me in so many ways over the years.

> Selenium came out as a side-project for another internal system we needed.

What kind of system? I had thought that selenium was created to TDD web development front-end.

The 4 ways to spend money[1], in this case, spending someone else's money on someone else. Likely a procurement person with no IT knowledge, who never has to deal with the painful solution, but got wined and dined by the salesman.

[1] https://investorjunkie.com/economics/milton-friedman-ways-sp...

No surprise that it is backed by Oracle. The sales people at Oracle are really good at getting technology inept people/businesses into multi year contracts.

Once they started citing the list of features in their hundred slide presentation, the administrator stood no chance.

All these enterprise systems are crap. Trying using SAP. It's not better.

Believe it or not, Peoplesoft was way better than what it replaced at my school.

There's a whole class of solutions beyond Selenium that automates not just web apps but all apps. It's going under the name of "Robotic Process Automation" and people really like it because they're not dependent on their software vendor to get basic integrations done.

Yes, multi-threads of it...that would be scary!

If you click on the "initiator" column in the network tab it will jump to the code that created the request. Can also see stack traces, set breakpoints, etc.

Other comment about initiators is probably more useful for your use-case but there are also "Event Listener Breakpoints" in the sources tab in chrome dev tools. You can use mouse->click to figure out what most buttons are doing (you may have to step through a couple onclick handlers to get to the one you want).

RPA (Robotic Process Automation) is a big category of tools that can be summarised as ‘automating boring stuff’. You often see it used for repetitive data entry, e.g.

1. Load excel sheet from X:\...

2. Login to Y

3. For each row navigate to ... then enter data

4. Submit, if there’s a validation error add it to and excel workbook

5. Email exception report to ...

Booking concert tickets seems to be a popular one: https://github.com/thomasms/glastoselenium

The fact that you have to update Chromedriver with every Chrome release now doesn't make things easier.

> "automating boring stuff using Selenium"

With an Appendix for the sites that block using Selenium.

When you start using xpath that's 20 parents deep and every line of code scrolls off the page you quickly realize how important it is for your devs to to write good accessible and unique element IDs.

Ah yes, the genetric algorithm (GA). Well, in short, scheduling personnel is a very visible, policital, and important job. It affects people's lives, and has to be done farily and correctly. I did my best (by hand) for a few years (scheduling 50ish professors into 200 class sections), and I knew I was doing a bad job, but had no choice. It would take 2 weeks to do, because I could only mentally handle working on such a job for so much in a given day before I had to set it aside. So I came up with a preference form, that each could fill out dealing with things like preferred time of the day to teach, daily schedule compactness, preferred class levels, etc. These preferences could also be weighted (0..1) by importance to each person. So the GA starts out with a long, two-part inital random sequence that consists IDs to all professors followed by IDs to all possible classes. Using crossover and mutation (the usual GA steps), the order of the professor IDs vs the order of the classes (prof 0 gets class, prof 1 gets class 1, etc.) are shuffled by the GA. (There are a lot of strange constraints to this you'd only see in academia, related to pay, and contractual teaching requirements.) A fitness is calculated for each professor's latest schedule, by computing a weighted sum of their preferences, based on the assigned classes pulled from the long GA sequence. It's incredible how well it works. It is done in under an hour, people seem happy, everyone's schedule fitness is minimzed, and each's schedule fitness is explained to them (i.e. you got +3 points because you wanted an 8am class, and got one). Everyone can also see everyone else's fitness so it's a "share the pain" kind of resulting schedule. At times when we go in to tweak things by hand, it is clear how tightly the GA optimized things, as any human intervention we try usually and obviously makes things worse. Bottom line, the GA generates the CSV, which Selenium types into PS. After using the GA, I'm convinced that neither humans (or spreadsheets) should be invovled in scheduling at all. I also think GAs are also pretty incredible.

Do you have any more information about this trace in HTTP requests? My understanding was that it was the _browser_ doing the requests; Selenium was just driving the browser.

I've managed to automate SharePoint data uploads and downloads using the SOAP API.