Hacker News Reader

Comment by bigmit37 | Hacker News Reader

Is C/C++ still worth learning if o want to create some models from scratch (new layers or different paradigms)

I hear that C++ is a nightmare to work with and was wondering if Rust,Julia, or even Swift would be worth learning instead.

I know Python but deep learning frameworks seem to be written in C++, so to come up with new layers I need to understand C++, which I was told has lot of peculiarities that takes time to pick up. Compiler isn’t also very user friendly (what I’ve read)

View on HN

C++ is not as tricky as people make it out to be. There is a lot of elitism among programmers, and a lot of people seem to claim it’s hard solely to make themselves look smarter for being able to write it.

If you know the basics of programming and have the persistence to. RTFM (Read The Fucking Manual), C++ will not give you any trouble. In fact, you might actually start to enjoy it more than the other languages you used in the past.

All that said, if you are focusing on machine learning rather than programming, then you should look into Python and R. A great resource is “an introduction to data science with R” by David Langer: https://m.youtube.com/watch?v=32o0DnuRjfg

If this is the case I would actually love to play around with C++ as a lot of software that Python wraps around is written is in it and it gives me chance to look a little deeper into the source code.

Julia is a blast to do research on this stuff in, if you want to go beyond the basics like TensorFlow and PyTorch allows. The 2020's is going to be the decade of mixing numerical PDEs with machine learning IMO, and Julia already has a lot of features along these lines that are missing from "traditional ML" libraries.

Interesting. I was going to go through their yearly conference talks to get an sense of Julia’s capabilities. JuliaCon2019 etc on youtube. Is that the best way?

Possibly. On this topic (machine learning, differentiable programming, GPU and parallel computing) I'd recommend the following videos:

https://youtu.be/FGfx8CQHdQA

https://youtu.be/OcUXjk7DFvU

https://arxiv.org/abs/1907.07587

https://youtu.be/7Yq1UyncDNc

https://youtu.be/_E2zEzNEy-8

https://youtu.be/6ntJ_al4oXA

https://youtu.be/HfiRnfKxI64

You can actually implement most new layers or experimental ideas using frameworks like pytorch or tensorflow. They support fairly low-level primitives which are much more flexible than keras or pytorch sequential models. That said C/C++ is still very useful for implementing high performance systems.

Ah. I haven’t played around with Pytorch custom layers enough so I am going to give it a try. I was initially trying to do it in keras but Keras was just using tensorflow layers for most operations so I couldn’t tweak the original tensorflow layers through keras easily.

The concept of "layers" is not in fact enforced by pytorch or tensorflow at all. This tutorial is a really nice overview of the levels of abstraction available in pytorch https://pytorch.org/tutorials/beginner/nn_tutorial.html

No point unless you have an interest in numerical linear algebra. The people who write the foundational Fortran/C/C++ libraries are experts in numerical analysis which is another rabbit hole.

If you want to write your own for fun, then there are some great algebra libraries in C++ you can use or you can use bindings for PyTorch or TF.

Yeah I don’t want to write my own libraries but create new layers from the existing numerical algebra layers.

I was originally trying to create a new type of convolution layer in Keras and asked in their official google board, stackoverflow etc , after being stuck for a while but the answers I got weren’t solving the problem.

I haven’t tried creating custom layers in Pytorch yet though so maybe it’s possible to do so with Pytorch and can just learn C++ for other purposes.

The Rust SDK for Tensorflow is worth a look.

Reply on news.ycombinator.com