Dzugaru

ChatGPT actually showed me how much I'm frustrated with Google nowadays. It gave me a glimpse of what real question answering can be - and it's not opening 10+ tabs and endlessly reformulating the query in hopes something relevant and not just popular will come out.

I really want a tool like that - just ask a question and get a simple straight answer. For now its Google's "here is a bunch of websites, search it yourself" or ChatGPT "sure, here is the answer, but maybe I hallucinated the whole thing, lol"

Does it really matter? If it eventually gets basic logic right and can do research/correction of it's own mistakes - it'll be insanely valuable tool even if it doesn't have consciousness as we do.

I don't usually copy blocks of code they produce verbatim, it's either line-by-line work (CoPilot) when I check every line - the same way I check my own lines - or I just take rough ideas (ChatGPT). The main good thing is - it makes me think more about what I'm doing, like a second mind (granted not a very bright one, but with a vast knowledge database).

I think the AI will actually leave us with the exiting part instead. I'm already seeing this with CoPilot and ChatGPT.

There is one reason is that the protocol of storage is open. Doesn't mean the thing can't die obviously, but there is some hope that someone will continue to store the data - there is some incentive (crypto value, however low or unstable that can be) to do that.

Also, nothing stops you from storing the critical data in S3 too. I wouldn't bet what will last longer as a permanent storage.

It is fed using a fascinating mechanism called "cross-attention" that originated in the NN architecture called Transformer - which was used to achieve state-of-the-art translations. It uses something like associative memory, where a NN inside Stable Diffusion, that generates image (UNet, working in latent space), at almost each step "asks" the whole encoded prompt to provide data at various positions using query vector Q that is matched against key vectors K and value vectors V [0].

How Stable Diffusion works [1] as a whole is not really hard to comprehend at a high level - but you'll need some prereqs - probability theory underlying this is explained in Variational Autoencoders [2], then Diffusion Models [3] sort of made a really cool "deep variational" autoencoder that uses small noise-denoise steps, but largely the same math (variational inference), but they were unwieldy because operated in pixel space, after that Latent Diffusion Models [4] democratized the thing by vastly reducing the amount of computation needed - operating in latent space (btw that's why the images in this HN post look so cool - the denoising is not in the pixel space!).

[0] https://jalammar.github.io/illustrated-transformer/

[1] https://huggingface.co/blog/stable_diffusion

[2] https://arxiv.org/abs/1906.02691

[3] https://arxiv.org/abs/2006.11239

[4] https://arxiv.org/abs/2112.10752

Implementing certain things in PyTorch can be fun. I've recently started to make an economic game simulator (where you have players that can buy and sell items, do fighting etc.) and began to outline the design in the usual C#, but then, suddenly, thought - why not make the whole thing run on GPU and support millions of players with tens of millions sim steps per sec? This should make experiment cycles really fast - and that's what simulators are for!

If you don't want to program CUDA kernels yourself, what's the easiest tool for the job? PyTorch, of course. Mutable by default tensors, lots of library operations and generally a pleasure to use.

The programming is really interesting - you start to think in batches - every operation is a parallel operation, it's not 1 player/item doing something, it's a whole lot of them at once - and that's really fast.

I've eventually did run into some problems - for example, ragged/sparse tensors support is not designed for this at all, and since I have variable number of items per player - this is a problem. Also, producing something like logs inside tensors is challenging - reallocation is very expensive and must be done in batches too. There are some other interesting things like launching several operations / CUDA kernels at once.

Location: Russia

Remote: No

Willing to relocate: Yes (very much)

Technologies: Deep Learning, Reinforcement Learning for gamedev, computer vision and robotics (PyTorch), Python, Unity3D/C#

Résumé/CV: On demand

Email: dzugaru@gmail.com

I'm an ML engineer/researcher and C#/Unity3D programmer with 10 yoe looking for job relocation opportunities. I mostly build custom ML/DL systems for various applications like product and business analytics, computer vision, etc. and do research like self-trained Reinforcement Learning agents for games. I'm open for work as ML/DL generalist or in game development. Please, message me if interested.

Observable? [0]

[0] https://observablehq.com/

Honestly, the thing kinda breaks if you have several sets of rules aka Bitcoin clones. No, ideally, the thing works if you have > 50% computation resources of the planet consumed (and of course, controlled) by miners using one set of rules. This way you cannot have competing chains at all, and the correctness of the set of rules is kinda obvious - they're correct if they make sense to me (paying arbitrary rewards to oneself does not make sense, obviously) - no need to ask anyone - I see an enourmous chain with enourmous burned computation and I see the code.

In this sense there could only be Bitcoin. PoS cannot come close to that at all.

P.S. To clarify a bit more - I don't think the most-difficult chain can be with broken nonsensical rules, not in the long term.

> Ethereum will use dramatically less carbon to be more secure

Why is that? PoS security model is fundamentally different from PoW - for instance - you cannot determine which of the two chains is "correct" anymore, you have to consult "a friend" [0]. That's not the case in Bitcoin, where the correct chain is simply the longest.

All PoS algos are very complex and have a ton of seemingly bandaid solutions for various problems that arise when you can create alternate histories freely (even if you use VDFs I think), and there is a disturbing lack of critique on that. So, why is it seen as "more secure"?

[0] https://yanmaani.github.io/proof-of-stake-is-a-scam-and-the-...

Some percentage? I haven't used polars, but I ditched pandas after observing x20 speedup by just rewriting a piece of code to plain csv.reader. Pandas is unacceptably slow for medium data, period.

Me and my friends have a funny habit calling these Spellbooks and pretending to be magicians. This leads to all sorts of hilarious interactions.

I'm fascinated by problems, that are NP complete, but still can have a ton of optimizations that can reduce the time needed by many orders of magnitude (millions or more), like SAT solving. Or approximate solutions / heuristics like approx. traveling salesman. Seeing huge improvements on real-life problems, even if you know the problem is asymptotically exponential feels like you've fought against impossible odds and won.

> Also, you can learn a lot about how senior someone is by watching how fast they can parse out search results and SO answers.

This is true, but you don't need to customize anything for that. VSCode text search gets you 99% there.

> I have never met a good craftsman that doesn't customize their tools.

To the contrary, personally I even take some pride I can work on any PC you can possibly give me immediately. This has tons of upsides - from helping someone walking over to them, to knowing tools in their default configurations inside out and explaining keybindings or quirks that are guaranteed to work (and writing better docs).

Programming really isn't about my personality, I want to strip any of that and produce the least amount of very default, simple code anyone can work with. And you don't need to type fast or use scripts to produce that small amount of code.

Largely same goes for debugging, I better create a good logging and monitoring system than spend time tweaking my tools, because that'll immensely benefit the person who maintains the code after I'm gone. Nobody will install my personal tweaks to catch bugs. Granted I mostly work with very good tooling already (C#).

> "do you have any tools you find value in customizing?"

Anecdata: don't think I have any? My best tools for years were pencil and paper, and I usually get things done, very fast.

Yeah, I've read more carefully and it seems they're doing full scan.

So you do a full scan and it's ~50 CPU cycles per row (48 CPUs at 4 GHZ), correct? This is possible I guess? And in this case Clickhouse is wrong.

"while QuestDB utilizes its full indexing strategy to read just a tiny fraction of the actual data"

Can you please elaborate on this?

Well, I don't know how QuestDB works, and I couldn't find anything in the original benchmark, but probably they already have some sort of (geo)index in place? It's really strange to search geo-data by scanning the whole surface of the Earth. The point that Clickhouse outperforms this by just sorting on one axis (and even not using any fancy 2D indices) is reasonable.

This is of course already done (to some extent) in aviation. Human error is indeed a start of investigation - there is extensive psychological profiling for pilots, why they did what they did given the circumstances, how much sleep did they get (often - not enough) and what function did they optimize (e.g. their salary bonus). One more thing is we have a co-pilot that can mitigate errors of the captain. Sadly, we don't (yet) for driving (and timescale of decision making is much smaller typically).

Honestly, a 10x engineer can do most of the list, but better. It's common misconception that if a person excels at something (s)he must lack in some other capacity. This is not true - there are extraordinary people that are insanely good across the board. They solve problems with unnatural speed and cunning, can explain complex things to 6 year old, really nice and social and love all programming languages (the last one maybe not actually).

My mom was a programmer (in the 90s) and they never thought this is the way to make a living. But they saw my interest in programming and went along with it, taking me to classes and buying a PC. It turned out spectacularly good for me, obviously.

Why don't we just remove gamification from it and be done with this? I wonder why anonymous forums (imageboards, etc.) aren't even mentioned, ever.

A couple of ELI5 followup questions for this cool ELI5 video:

1) Why would you create a new provider, why not just use [insert biggest provider name here]? In other words, whats' stopping this thing from centralization? It's not like you need to build a rocket to move to the other planet, right?

2) How do you so easily move videos from one provider to another? The amount of data is enormous, you can't just copy it everywhere?

I bought some crypto for the first time recently. Because Mastercard and Visa don't work for me anymore - I can't pay for services outside my country of residence, and maybe there is some obscure way to do it not touching crypto, but I chose to get some Bitcoin because that was relatively easy.

Sure, there is a gradient of complexity here, first you can just do your stuff in kernel in a single thread, then you'll probably want to load programs and execute them in userspace [0], then you'll want multitasking and task scheduling, memory isolation and process syncronization etc. You can call any of those steps an OS of course, its just I personally find earlier steps not that interesting.

Another one great resource (that actually inspired me to tinker with all this awesome stuff) is this book [1]

[0] https://wiki.osdev.org/Getting_to_Ring_3

[1] https://pages.cs.wisc.edu/~remzi/OSTEP/

Unfortunately, this (really nice) PDF isn't complete.

For me an Operating System means processes in userland, sadly it comes short of that.

I'm now pushing my own x86 32bit OS in Rust using [0][1]:

There is definitely a shortage of information on this stuff (for example writing a USB driver is really hard and I couldn't find easily available tutorials - you have to resort to raw *HCI and USB specs to get things done).

[0] https://wiki.osdev.org/Expanded_Main_Page

[1] https://os.phil-opp.com/

This was just delisted from HN because... why?

I tried to do that a year ago, but the problem was my PC froze (full lock up, even mouse pointer is frozen) regularly. I didn't know how to even start troubleshooting a thing like that so I left to Win10 again (was Linux Mint btw).