pinouchon

Yes I am. Without more detail about the case it's hard to say what was the issue. Binance has what they call "machine learning limits": https://www.binance.com/en/support/articles/360004492232.

What lowers my concerns is that such issues might come once the bot is profitable, and by then I might have found other ways to raise capital (I'm doing a trading bot as a way to raise capital to create an AI research lab). Being able to setup a profitable crypto trader is a good thing to add on a resume or for personal branding even if I shut it down at some point. I'll be in a very different position by this point so it's a bit premature to be concerned about that now, although it's still somewhat of a concern.

On the positive side, I hope this scares the competition away.

I spent the last year working fulltime on a system similar to the one described here. I trade the top ~20 cryptos on binance. I use deep learning models (combination of temporal, causal convnets and RNNs) with heavy data augmentation. I built my own tooling for data collection, training, backtesting and live deployment. Having a data engineer background coming into this was hugely helpful: most of my time was spent manipulating data in some way (and not playing around with the models). One of the most demanding parts was estimating spread/slippage costs and including it into the loss function.

Most of what the author talked about, I learned the hard way.

I'm now at the point where I ran some tests (trading small amounts) live on binance and the results are positive: I do manage to make small profits, but more importantly, the recorded live trades reflect very closely the backtest trades (for a given period). I'm currently scaling up my model and adding better monitoring / reporting / CI.

I'd be happy to chat with anyone having done similar projects or willing to exchange ideas.

I have been thinking about this recently. That applies to physical products too: the highest quality products are usually not the ones with the best marketing. What we need is a SEO-resistant search engine (for tutorials, products & services). Maybe a way to approach this would be to have the engine function more like an independent amazon-review engine where it's really hard to put fake reviews and have a "quality index" that's community based. I think that the successor to google will look like this: a search engine that cuts through SEO and marketing, and shows you the actual good stuff.

As Karpaty said:

1. Become one with the data.

The first step to training a neural net is to not touch any neural net code at all and instead begin by thoroughly inspecting your data. This step is critical. I like to spend copious amount of time (measured in units of hours) scanning through thousands of examples, understanding their distribution and looking for patterns. Luckily, your brain is pretty good at this. One time I discovered that the data contained duplicate examples. Another time I found corrupted images / labels. I look for data imbalances and biases. I will typically also pay attention to my own process for classifying the data, which hints at the kinds of architectures we’ll eventually explore. As an example - are very local features enough or do we need global context? How much variation is there and what form does it take? What variation is spurious and could be preprocessed out? Does spatial position matter or do we want to average pool it out? How much does detail matter and how far could we afford to downsample the images? How noisy are the labels?

source: http://karpathy.github.io/2019/04/25/recipe/

They shouldn't have accepted to get bought by Activision. The only way they make good games now is by releasing old titles (sc remastered, classic, reforged)... And now this

Also, get high quality sleep. I'm still not sold on video games and coffee/tea. I'll have to look at it more. If you some research that shows the beneficial effects of coffee/tea and video games, please share

My take is that it's your job as a developer / software engineer to find the the best jobs. Most jobs may be boring, but we are lucky to be in demand as developers so it's relatively easier to find fulfilling jobs (compared to other sectors). For example, I left my semi-boring job as an web+data engineer to create an ML-based crypto trader. Is it easy? No. And I happy with that choice? Hell yeah

Also, most of my relatives / family don't quite understand or approve of that choice, and that's fine. Sometimes you have to be a little daring

I'm in the same boat: my standards for myself are way lower than when I work for other people.

I think self-discipline doesn't cut it. What you need is commitment devices: put yourself in a situation where it's harder/more costly to not do the work rather than do it. Have the environment do the motivation work for you, "tie yourself to the mast"

For example: time tracking, deadline for a public speech, deadline for a demo day, going to a dedicated room where only work is allowed, working with other people (even if they don't work on the same stuff as you), developing a good routine/habits. Keeping a log of progress/failures is good too.

It's hard stuff

Talking about facebook creating the "social media" genre and taking over (at 18:58):

> It makes you wonder what else is coming [...] The impulse is "there won't be something like that ever again". But of course there will be in about 3 years probably. Whatever it is, it's probably being built now. You just can't tell what it is. And you know what I can't tell either.

This has been recorded in 2009, the year bitcoin has been released as open source software. So PG was pretty spot on that great stuff was being built as they speak.

What do they mean by: "Microsoft will become our exclusive cloud provider"?

Being forced to use Azure for all your ML workloads seems a stupid constraint. For example, you might be comfortable with tensorflow/TPU and changing frameworks/tooling might be costly.

Go down that rabbit hole. https://www.reddit.com/r/TheRedPill/ It will give you a toolkit for "debugging" relationships.

I can relate to that. For me the lesson is: We are often confused about what we want, and society/evolution makes us want stupid things. It takes conscious effort to really author our desires

I'm guessing your employer or your gf walked away with a good chunk of what you had.

It was pretty pre-detemined (at least in the matches vs Mana. The best pros can react very fast to unusual strategies (cheese), or change their strategy if the situation calls for it (eg: terran all-ins). I agree with the sentiment of disappointment of having to make a "league" with specialised AIs. It seems to be lacking a high level reasoning and executive module, and the league acts as a crutch. The league idea still super impressive and clever, but human players do not go around in gangs where only one of them takes a seat for each match.

- A Personal Viewpoint on Probabilistic Programming https://www.youtube.com/watch?v=TFXcVlKqPlM by Dan Roy (I think this is the best intro to PP there is)

- Probabilistic Models of Cognition https://probmods.org/ by Noah D. Goodman, Joshua B. Tenenbaum & contributors

- An Introduction to Probabilistic Programming https://arxiv.org/abs/1809.10756 By Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, Frank Wood

Sounds like a good fit for probabilistic programming, much like Stuart Russell did here https://www.youtube.com/watch?v=GYQrNfSmQ0M&feature=youtu.be.... His method could find locations of nuclear tests better than the existing UN system at the time by a lot. His model uses bayesian inference in a stochastic/symbolic/simplified/statistic model of physics (how shockwaves propagate on the surface of the earth).

You could do the same here: assuming the relevant laws (kepler? newton law of gravity?), and a prior distribution on the location/mass of your 9th planet, given what we observe for the other planets, what's the posterior distribution on the mass/location of the 9th planet.

The statistical model is likely to be small (the Russell statistical model for Nukes fits on one slide). The issue is how to do inference efficiently. Fortunately, probabilistic systems have come a long way and can do these kind of inferences.

Same journey for me. I went from using AZERTY without touch typing to QGMLWY with touch typing (http://mkweb.bcgsc.ca/carpalx/?full_optimization). I practised with https://thetypingcat.com/ and zty.pe. It took ~3 months@30m/day to get to 60wpm, then I made the switch and it took about a week to get comfortable using it everyday.

Some takeways:

- the keyboard layout doesn't matter that much, what matters most is touch typing vs not touch typing

- I still cannot touch type with azerty. The touch typing skill does not transfer to the other layout surprisingly. On mobile I use qwerty, but I don't type that much on mobile.

- On OSx, I use this horror of a script: https://github.com/pinouchon/keybest/blob/master/rewrite_c/a.... It works fine, except that some password inputs use the mapping, some don't, which is slightly annoying. On ubuntu, I use XKB. I made it work through days of digging the documentation, XKB is the most confusing thing ever.

- The QGMLWY mapping preserves ctrl-x, c, v, z so cutting and pasting is not an issue

- I re-learned ctrl+letter and cmd+letter hotkeys. I like them better with the new layout.

- I created a custom layer called 'gamma' for coding with the modifier on the 'm' key: with the following layers: "top: #[({<>})]^ middle: ;=:"'/_-! bottom: ~?$\&+*.,". For coding this is invaluable. For example, to type underscore, my fingers don't move, it's just simultaneous pinky+index while staying in home-row position. On traditional layouts or not-so traditional layouts (qwerty, dvorak, colemak), modifiers are all over the place and don't take into account special character frequency.

- I also have a 'alpha' layer for movement that I do use. I moved the shift and tab keys. I also tried moving backspace and enter keys but ended up keeping their original positions.

- One added benefit with messing so much with the keyboard is that when I need a new hotkey or a special mapping for a game (where the default bindings are stupid), I can jump to my script and create a special mapping for that game.

- One big downside is that whenever I switch OS, I have to redo my keyboard setup otherwise I cannot use the keyboard

- Like others have mentioned, a neural interface would be better

You forgot the most powerful of them all: Ctrl+Z followed by kill -9 pid

I chose to work on AI (AGI research / probabilistic programming / deep learning). Choosing wasn't hard, but getting to work on it was: How do you make a living if you first need to educate about the topic for a long time? Answer: you don't, research is about endlessly educating yourself.

If you want more details, ping me at pinouchon at gmail com.

Thanks for the feedback. I think the next step for me is to actually try it out live and see how it performs. If the results are positive, I will have an easier time selling my idea

I think your argument would be better if you don't just say "the product would be better with more people working on it", but also say how you plan to recruit them, what you need to do that, and how velocity would increase. If you can show that you have thought it out and you can manage the product side of things, it shows that you are serious about growing the product team, not just randomly throwing capricious (and costly) demands.

Regarding the possibility of quitting, I think it's better to warn the CEO early about that, than making a decision and suddenly tell him that you are leaving. If you were the CEO, how would you want your coder/product guy to behave? Then behave like that.

Is the product deserving more engineer attention (I suspect it does, but you should still carefully consider the question because that could be how the CEO thinks). And if so, do you think you have done everything you can to convince the CEO, and allow more resources towards the product ?

I would try to speak openly to the CEO about the fact that you are considering quitting.

Also, what about equity/options/vesting?

My guess would be that evolutionary speaking, being able to make useful tools provided a significant advantage (hunting, fire, writing...) So hacking around, and liking hacking around was selected for.

Same thing, took me a while to understand that facid was indeed facility_id.

In my previous work, we used this to test candidates when their job involved writing SQL. Worked great.

I think folding the update rule inside the MCTS loop (in alphaGO Zero) is genius.

Hi. Thanks for the great article.

Can you elaborate on what you mean by "directions in activation space" ? If I understand it right:

You take a few neurons in a layer, and you follow some linear combination of their weights; you are then walking along a random direction. You take a single neuron and walk along its weights; you walk along this neuron's direction. Is this correct?

Also another comment: Szegedy et al [9] (https://arxiv.org/pdf/1312.6199.pdf) has the following abstract:

> First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains the semantic information in the high layers of neural networks.

This is one of the predictions of the following paper, even though I think it came later: Opening the Black Box of Deep Neural Networks via Information (https://arxiv.org/abs/1703.00810). Here: https://youtu.be/FSfN2K3tnJU?t=1h19m23s, Tishby says: "One of the consequences of this theory is that single neurons in a layer don't tell us much."

Also, If you extend this idea to training examples, you get Mixup (https://arxiv.org/pdf/1710.09412.pdf)

"When Jeff Dean sends you a code review, it's because he thinks there's something in it you should learn."

They are similar like java and javascript are similar

This is not a simple explanation, but one I quite like (by Dan Roy) https://www.youtube.com/watch?v=TFXcVlKqPlM