Hacker News Reader

Comment by sillysaurusx | Hacker News Reader

A bunch of you seem interested in taking up the challenge: having to write smarter software, wistful for the old days when it was forced on you.

Turns out, you can! There's a big domain that needs your help, today: deep learning.

A lot of the software is currently crude. For example, to train a StyleGAN model, the official way to do it is to encode each photo as uncompressed(!) RGB, resulting in a 10-20x size explosion.

There's plenty of room at the top, and never moreso in AI software. Consider it! Every one of you can pivot to deep learning, if you want to. There's really nothing special or magical in terms of knowledge that you need to study. A lot of it is just "Get these bits from point A to point B efficiently."

There's also room for beautiful tools. It reminds me a lot of Javascript in 2008. I'm sure that will sound repugnant for a majority of devs. But for a certain type of dev, you'll hear that ringing noise of opportunity knocking.

View on HN

> There's a big domain that needs your help, today:

Enterprise software!

But we’re more likely to see true General Artificial Intelligence before we see a 2x increased in enterprise software performance.

Robotic process automation (RPA) can really help boost the performance of enterprise software, and it is just starting to take off.

What is the difference between RPA and macros? edit: It seems fragile in the face of any GUI changes.

My understanding is that they’re more autonomous, more extensible, and generally result in more widespread gains in efficiency than standard macros.

https://www.uipath.com/blog/whats-the-difference-between-rob...

RPA is something like "macros on steroids". It tries to be something like shell scripting for script-hostile GUI applications, but at least the one product I've gotten to play with is still way more clunky than proper shell scripting.

In response to your edit: I can see how GUI changes would be a problem for RPAs, but that seems solvable. For minor GUI changes, an RPA can theoretically identify where salient features have remained and just re-train on where they were subjected to change. If that approach fails, you can "show" the computer the new GUI and manually identify what changed. If that fails, you can restart training from scratch with the new GUI. That should take less time than training on the old GUI if the new GUI is any good. Maybe that's the best/cleanest approach in most cases and just becomes annoying with complex processes that need a lot of training data.

Keep in mind, though, that RPAs often link together different pieces of software, and if the GUI changes for one of them you'd only have to retrain on that piece. I wouldn't be surprised if enterprise software vendors start optimizing their programs for RPAs so that they don't have to rely on hacky GUI monitoring as much.

The end game is that large, frequently run RPAs will serve as a flag for the organization to develop, find, or outsource an end-to-end program that executes the same process without an RPA. RPAs, then, will always be the scout at the frontier of automatable office work, finding who can be freed from rote drudgery next and serving as a bridge to the best programmatic solution.

Is it like Automator for enterprise applications?

http://macosxautomation.com/automator/

Only in the sense that it automates stuff. RPA is differentiated by its use of machine learning to "observe" your workflow and automatically create automations, even where it's hard to create a standard macro.

I’ve got a fairly comprehensive AutoHotKey script going on at work to macro the fck out of repetitive tasks in the CAD / CAM setup I use.

Very heartening to hear. You think some of us tired enterprise/product devs that come from non-traditional (read: didn't study CS) background can make it? Any recommended starting resources? I've heard supposedly good things about fast.ai as a "guided tour".

Absolutely. I didn't study CS.

What you need is determination. There's no substitute for this.

If you have that, there are all kinds of resources. Here are a few...

Resource 1: a community. We've set up a discord server for AI dev. It has 360 users. At any given time, around ~50 people are online, of which ~10 are skilled devs. Come join! https://discordapp.com/invite/x52Xz3y

Resource 2: Find something fun for you, and pursue that. I like generative AI, so for me that's been GPT-2 and StyleGAN. Gwern has some lovely tutorial-type articles on both.

GPT-2: https://www.gwern.net/GPT-2

StyleGAN: https://www.gwern.net/Faces

Peter Baylies' StyleGAN tutorial notebook is a hands-on resource. This was actually how I started, nearly a year ago.

github: https://github.com/pbaylies/stylegan-encoder

notebook: https://colab.research.google.com/drive/179SPYbBC8pKDxVRjZep...

(The original notebook was broken; this is a copy I've updated with some fixes.)

Lastly, follow a bunch of AI people on twitter. Here are a few to get you started: https://twitter.com/i/lists/1160386581850730496/members

The reason to follow them is, whenever you see something that seems interesting or fun, tweet at them and say so! Ask questions. Ask how to get started. Everyone is shockingly nice and helpful. My theory is, the software is so crude and often hard to use, that we all like to celebrate together whenever one of us gets it working, and we're happy to share that knowledge however we can. (Twitter is a bit chaotic right now due to world events, but I imagine it might return to normal within a couple weeks.)

And yes, you're right about fast.ai and other courses. You can go that route if you like it. I found it more exciting to dive into the deep end, though, and try to tinker with stuff.

>> Absolutely. I didn't study CS.

But, have you achieved something singificant in the context of deep learning, as per your earlier comment? There's no information about that in your profile and a cursory glance at ddg and google results for "shawn presser" doesn't turn up anything very relevant.

So, I have to ask: without having studied CS, what contributions have you made in deep learning that are widely recognised?

I hope you agree this is a reasonable question to ask, and that you are not offended by it. Otherwise, I apologise because it's not my intention to offend you.

Certainly! Some highlights:

- A Newsweek article https://www.newsweek.com/openai-text-generator-gpt-2-video-g...

- GPT-2 chess https://www.theregister.com/2020/01/10/gpt2_chess/

- ... which DeepMind referenced: https://twitter.com/theshawwn/status/1226916484938530819

- GPT-2 music https://soundcloud.com/theshawwn/sets/ai-generated-videogame...

- Swarm training (WIP) https://www.docdroid.net/faDq8Bu/swarm-training-v01a.pdf

To be honest, and again without having an intention to be harsh, but those are not what I'd call "widely recognised contributions to deep learning". They're mainly articles in the lay press and a honourable mention in a DeepMind blog post. They certainly sound like contributions to Shawn Presser's reputation, but "contributions to deep learning"?

To clarify, what I was hoping to see is, at best, an article published at a reputable venue for AI research, a conference or a journal, or at a minimum an arxiv article that at least looks like it was meant to be submitted to a conference or journal. And at worst, a software tool that can be used in deep learning research. But it seems to me that your achievements are mainly having fun with and in one case finding an interesting use for tools that are already available.

Again, I'm not trying to be harsh, neither do I want to say that all this is not worth the trouble. But it should not be held up as an example of what people can achieve without studying CS. Because, I think you'll agree, they are kind of underwhelming when compared to what people routinely achieve who have studied CS.

> But it should not be held up as an example of what people can achieve without studying CS

You really need to get out more!

Excuse me, but is there a reason why you chose to reply to my comment with an insult? And six days after I originally posted my comment?

I _super_ appreciate this, thank you for sharing. I've had to deal with a lot of gate-keeping in my career, so its great to interact with people who are so open to invite people in. We could use more with such a positive attitude.

Definitely try fast.ai, I also have no formal CS background, am bad at math, etc... and with a little hard work I was able to keep up and do things I never thought possible. You will truly be blown away after even just a month of the class as to what you understand and can build. It is not a domain solely for geniuses, roll up your sleeves and you can be great.

Thanks a lot. What was the most fun you've had with it so far, if I may ask?

Admittedly, I took the first iteration like 4 years ago, and I've heard it's much improved since then, but the fun part is they get you building real things right away! You build an image classifier in the first lesson.

With other ML resources I find they get bogged down trying to explain the concepts/details, and I would usually lose interest before getting to the implementation of things.

They've also added a "theory" portion after the practical, which I think is a great way to solidify the concepts scaffolded in building the thing initially. Thank you for your input.

May I ask, why are you interested in deep learning? What are you trying to achieve? If you don't have a traditional background in CS, might you not be better served by working to acquire such a background first, before thinking of making any contributions to more advanced sub-fields of CS?

For instance, I'm sure that self-studying some fundamental subjects in CS, like formal languages and automata and complexity theory, will give you important tools to tackle any CS-related problem, be it programming a more efficient deep learning implementation or really, anything else you like.

Spending the same effort to get into deep learning instead, will most likely leave you with only superficial knowledge that will not transfer to other subjects.

I think its safe to say I'm looking to see if this field is something I'd be interested in pursuing deeper, and get a feel for the types of problems that might or might not put bread on the table should I go further.

I have no idea if I would or would not be better served by a traditional CS background in this case, so part of this is to find out if that is or is not the case.

If I understand correctly, your main interest is in finding ways to up your game when it comes to placing yourself professionally?

In that case it's very difficult to go wrong with acquiring a traditional CS background. There is really nothing you can do with computers that will not benefit from a CS background, including anything that may have to do with deep learning etc. On the other hand, learing about deep learning will only help you when it comes to working with deep learning.

So, regardless of whether getting into deep learning may help "put bread on the table", getting a well-rounded CS education, certainly will.

The highest-performance deep learning framework is most likely Theano which still has to be ported to current Python 3.x. There's quite a bit of low-hanging work to be found via fixing things like that.

Highest performance by what metric? I thought Theano has been dead for a couple of years, at this point.

Reply on news.ycombinator.com