Hacker News Reader

raibosomeon Ask HN: What's your quarantine side project?

I have 3 pet projects.

(1) Bython, a basic Python interpreter written in C. I just want to have a taste of what it's like programming out a language so I won't go too far. It is dynamically typed and has automatic memory management. https://github.com/remykarem/bython. PRs welcome!

(2) minishell. This is a simple shell with only 2 commands: either you (i) hit enter to view 10 files of a folder at a time, or you (ii) enter a filename and view nbytes. I built this because a co-worker wanted to view a folder with 20+GB of files but couldn't do it with an `ls`. With this, I hope that we can casually explore a folder without having to print everything. https://github.com/remykarem/minishell.

(3) Scrollable Python documentation, a hack from the scrollable interface found in https://allennlp.org/tutorials. Use case is for people who are explaining Python code. https://github.com/remykarem/scrollable-python-documentation.

My command of C isn't that great so if you're interested to collaborate, I'm happy to be your apprentice :)

raibosomeon Deno 1.0

Nice, I deno about this.

raibosomeon Ask HN: How do you manage self-study?

You need a way to organise and prioritise these. Use a Trello board.

Because you're self-studying, you're sort of going to a self-study university taking different modules from different faculties. This Trello board spells out your self-study university curriculum and you're in charge of it.

Here is a 5-step process to build this curriculum.

Firstly, create say 3 lists on the Trello board: ML, CS and Math. Each list represents a 'faculty'.

Then, for every list, create Trello cards where each card is a 'module'. For example, you would create 'Data structures & algorithms' in the CS list and 'Decision trees' under ML.

The next step is to figure out for each module if it's something you either (i) wish to know or (ii) must know. You can use Trello labels or even use the Trello separators for this.

This following step requires a bit of work and it's the fun part, only because it's self-study. For each module, list down (you can list things in a card) the resources you have for that module. For this there are various resources you can get from the comments, search engines, and your peers. Consider the different modes of instruction: books, e-books, videos, lecture notes, slides, articles, blog posts, online learning platforms and so on. Choose what's best for you. If you can't decide just pick something first and find another time to source for another material.

Lastly, prioritise the modules. This can be done by easily dragging the modules which you want to do first on top of the list (having considered what you wish to know and what you must know). Set, say, top 3 modules for each list then you'd like to do for the next 2 weeks.

This is the high-level curriculum planning. If you plan on a micro-level planning like what modules to do for this week or for today, that I leave it to you.

raibosomeon Ask HN: How can I pick a side project and stick with it?

I do exactly this!

raibosomeon A Gentle Introduction to Bayes’ Theorem for Machine Learning

Shameless plug but if anybody wants to look at Naive Bayes classifiers for continuous and categorical dat, I wrote a library here:

https://github.com/remykarem/mixed-naive-bayes

raibosomeon Stop Using `rm` in Bash. Use `del`

Okay I think this is more elegant than my workaround. But it doesn’t work for directories, does it?

raibosomeon Stop Using `rm` in Bash. Use `del`

Thanks for pointing out! Now everyone’s Bashing me in their comments.

raibosomeon Stop Using `rm` in Bash. Use `del`

As pointed out, the alias should be

  function del() {
    mv “$@“ ~/.Trash
  }

instead. Thanks @floatingatoll!

raibosomeon Convert Python source code (.py) to Jupyter notebook (.ipynb) with p2j

Say no more to copying code line-by-line to Jupyter notebook. This conversion just needs your code to be paragraphed, like how codes should be. Comments are converted to markdowns. Codes are converted to cell codes. It's as easy as running pip install p2j and run p2j your_code.py

Pull requests are welcome!

raibosomeon Attention in Deep Learning

See how attention is used for neural machine translation. One of the examples is Google’s NMT which powers Google Translate.

raibosomeon Animated RNN, LSTM and GRU

Do you mean the attention module?

raibosomeon Animated RNN, LSTM and GRU

I created GIFs for us to understand these recurrent neural networks. Do let me know if you have any feedback!

raibosomeon Step-By-Step Tutorial on Linear Regression with Stochastic Gradient Descent

This is a supplementary material to the backpropagation demo found on http://raiboso.me/backpropagation-demo/.

Do let me know if you have any feedback!

raibosomeon Interactive Backpropagation Demo for Simple Linear Regression

I did think about doing that but it seems a little unfeasible.

You might want to check out my repo https://github.com/raibosome/raibosome.github.io/tree/master... It's a little messy oops.

raibosomeon Interactive Backpropagation Demo for Simple Linear Regression

Explore how (stochastic) gradient descent works on a simple linear regression. I built this demo to help me better understand backpropagation, by keeping an eye on the values of the weights as they get updated.

I've also added step-by-step remarks and graph plots to the values of the weights and the loss function.

Things you can play around with: optimiser, learning rate, variable initialiser, loss function, batch size, no. of epochs

JavaScript libraries used: Dagre-D3 (GraphViz + d3), MathJax, ApexCharts, jQuery

Any comments to this demo are welcome!

raibosomeon Gradient Descent Optimisation Algorithms

Thank you for pointing these out! I have made the necessary edits to the citations for (1) and (2) and republished the article.

For (1), the paper by Sutskever et al., 2013 (http://proceedings.mlr.press/v28/sutskever13.pdf) attributed the classical momentum to Polyak, 1964 (https://www.researchgate.net/publication/243648538_Some_meth...). A Distil article on momentum (https://distill.pub/2017/momentum/) also cited Polyak's paper and also included a much earlier publication in 1959 by Ruttishauser (https://doi.org/10.1007/978-3-0348-7224-9_2), but I will just make reference to Polyak's.

raibosomeon Gradient Descent Optimisation Algorithms

Sweet! Another intuition for gradient descent:

Regularly updating your parameters using an educated guess. This educated guess is the gradient value.

raibosomeon Gradient Descent Optimisation Algorithms

If I may, I had also built a simple demo of linear regression using gradient descent before writing this post. https://raiboso.me/backpropagation-demo/

This demo allows you to choose between four optimisers, and lets you track the values of your variables during training.

Compare your runs with different optimisers using the graph at the bottom of the page.

raibosomeon Gradient Descent Optimisation Algorithms

This is great! I'd love to do something similar in JavaScript.

raibosomeon Gradient Descent Optimisation Algorithms

At the end of this post you will get a cheat sheet of the 10 common gradient descent optimisation algorithms.

Using more readable notations, I will walk you through how the vanilla stochastic gradient descent slowly evolved into the popular Adam optimiser and others. I also came out with an ‘evolutionary map’ of the optimisers to visualise this.

The motivation for writing this post is that there is a lack of simple-to-read equations for parameter update and a compiled list of these optimisers.

Hopefully this benefits the community.