Hacker News Reader

Comment by ndiddy | Hacker News Reader

The approach the author describes is batch processing, not time sharing. The whole point of time sharing was to allow users to work with the computer interactively rather than having to submit a job and wait for results.

View on HN

Wouldn't Google Colab then we closer to both "time shared" and "supercomputer"? Jupiter

notebooks are certainly interactive and Colab's purpose is to allow often low-powered clients to offload ML training/inference to a GNU-enabled server, so kind-of like time-shared systems of the old?

This reminds me of my dad talking about how amazing it was when time-sharing became a thing at UWaterloo. He'd talk about how batch-processing was such a pain when you'd discover some tiny error and have to get back in line, and how with time-sharing you could show up at 3am and have an entire PDP to yourself.

My PhD advisor stayed up late at night (in the '60s) to get exclusive access to an IBM mainframe that was normally used at UC for payroll. When I joined the lab the convention was to go find a machine that was only lightly loaded (manually; there was no batch queue, just telnet to a cluster of machines) and start your multi-month simulation and hope nobody else popped onto the machine to steal half your processor.

Batch processing and time sharing, there are pros and cons. Time aloning, though, there are no cons.

It needs JCL cards for proper batch processing.

Speaking of mainframes... I ran across this video on the latest workflow for COBOL / CICS development on your IBM mainframe. You have the old 3270 emulation way yes, but then we move into 2010 technology: a custom version of Eclipse IDE that submits the jobs for you, then we move into modern times with Visual Studio Code integration (using something called Zowe CLI):

https://www.youtube.com/watch?v=_CYUYnKim7U

http://www.winestockwebdesign.com/Essays/Eternal_Mainframe.h...

Yes. In the courses I give on MLOps, we use Github Actions to run on a schedule:

1. feature pipelines

2. batch inference pipelines

I needed to setup GitHub actions for several open source repos for automated CI/CD, using it for ML pipelines seems awful.

I would love to hear your suggestion of an alternative free platform that can be used to run python programs on a schedule with.

Reply on news.ycombinator.com