calebh

There's a tool now that can do matrix calculus: http://www.matrixcalculus.org/

This comment on Reddit by a microbiome researcher says that these tests are useless: https://www.reddit.com/r/Microbiome/comments/84ax5i/is_there...

Imo gradient descent is not going to get us to general AI. I'm a programming languages theory person, and I also know a fair bit about machine learning. The current problem with neural networks is that they suck at processing variable length data, and they have problems with remembering the past. Programs can be represented by trees, graphs, or text. Deep learning isn't great at dealing with graphs or trees, and it's not that good at text either.

The other issue is that deep learning works great for recognizing common patterns, but it sucks when faced with novel situations. So I don't think that we're going to have programs that program anytime soon. The first applications of AI to programming will probably be with programming assistants or with AI guided proof solvers. Programmer productivity will improve, but we're not going to see everyone losing their jobs.

Isn't this just the Maybe monad without any benefits of abstraction?

You're completely ignoring the time dilation effects of relativity. From the perspective of a fast moving spaceship, the distance between stars contracts, and from the perspective of an outside observer, the time experienced in the spaceship slows down. This is why I believe that interstellar travel will be feasible, although creating any sort of empire will be impossible. Here's a quote from the Wikipedia article on time dilation:

"Theoretically, time dilation would make it possible for passengers in a fast-moving vehicle to advance further into the future in a short period of their own time. For sufficiently high speeds, the effect is dramatic.[2] For example, one year of travel might correspond to ten years on Earth. Indeed, a constant 1 g acceleration would permit humans to travel through the entire known Universe in one human lifetime.[12] Space travelers could then return to Earth billions of years in the future. A scenario based on this idea was presented in the novel Planet of the Apes by Pierre Boulle, and the Orion Project has been an attempt toward this idea."

https://en.wikipedia.org/wiki/Time_dilation#Velocity_time_di...

Ah, thanks for the tip. When I'm reading research papers I often find clusters of related papers with the same keywords. Often finding the separate clusters is a difficult challenge, even if you use Google Scholar and follow all the citations.

I've recently become interested in permutation invariant neural networks. There has been very little work in this area - just PointNet and a few derivatives.

Anyway, I think that neural networks are now entering the trough of disillusionment as people begin to discover the limitations. Maybe in the future, somebody will come up with a new machine learning architecture that has better generalization. I'm not expecting gradient descent to give us general AI.

I recently purchased an electric bike to commute to work, and I ended up returning it. Electric bikes go very fast, which means that cars often have difficulty passing, and drivers are not quite sure how to treat you. I ended up returning the bike because I didn't want to end up as a splatter on the side of the road. If the US had protected bike lanes, it might have worked out differently.

Is there a way to cryptographically sign a video to make sure that it's tamper free? I'm thinking about some sort of encryption that happens directly on the camera while a video is being recorded.

I'd pay money for a Netflix-like service for newspapers. I know it's probably not as good for the newspaper business as the old subscription model, but it's pretty much the only viable model that I can see working.

There's a lot of confusion around understanding monads, and I remember having trouble understanding them. Here's the two things that helped me the most: understanding kinds, and staring at the type signature for >>= for a long time. I'll attempt to explain both of these here:

Just as expressions have a type signature, a type expression has a kind signature. In its most basic form, a kind tells us how we can construct a type. We represent kinds by using asterisks * and kind functions ->. The asterisk is pronounced as "type". The easiest way to understand kinds is by looking at a bunch of examples of types and type constructors. Monomorphic types such as Int and Bool have kind * . Type constructors are handled differently. An example of a type constructor is [] (list), which has kind * -> * . So list is a type constructor that takes in a type (which we represent with an asterisk), and returns another type. Therefore [Int] has kind * , since we applied the type Int to the list type constructor [], resulting in the type [Int]. Types constructors can also in some situations be partially applied, just like value constructors. Kinds are right associative, so the kind * -> * -> * is the same as * -> ( * -> * ). I have a table of different kinds on my blog here: http://www.calebh.io/Type-Inference-by-Solving-Constraints/

Now that we understand kinds, we are now ready to understand monads. In Haskell, type classes are used to overload functions in a disciplined way. One such function is >>=, which is defined in the Monad type class. When we want to make a new Monad for a different type, we overload the >>= function. Since the >>= function is the most important function in a Monad definition, here is its signature:

(>>=) :: forall a b. m a -> (a -> m b) -> m b

How do we interpret this signature? Well we can see that the >>= function takes in two arguments, one of type "m a" and another of type "a -> m b". Remember the kinds from earlier? In this case, "m" is a type constructor of kind * -> * . So "m" could be the list type constructor, the Maybe type constructor, or really any other type constructor that has this kind. It can even be a type constructor that we define ourselves. So what can an instance of the >>= function do with the first parameter? Well it can do anything that it wants, as long as it follows some laws, which I'll talk about later. However notice that bind also takes in a second parameter of type "a -> m b", which is a function that takes in a value of type "a" and returns a value of type "m b". So the >>= function might end up calling this "callback function" and using its result. It could even call this function multiple times if it wanted to. The point is that >>= can do anything as long as it adheres to the type signature and follows the monad laws.

The monad laws are not as relevant when learning how to use monads, but I will cover them anyway. When you write your own overloaded instance of the Monad type class, you have to make sure that your overloaded functions follows these laws:

Left identity: return a >>= f ≡ f a

Right identity: m >>= return ≡ m

Associativity: (m >>= f) >>= g ≡ m >>= (\x -> f x >>= g)

You can think of these laws as analogous to the laws for operations on numbers such as commutativity and associativity.

Ah I was thinking that they would throttle you to ridiculously slow speeds if you went over the limit. It looks like the throttling doesn't kick in until 15 GB, which is pretty good.

Now that we have super fast mobile internet, can we please stop charging absurd data costs? I have Google Fi, and they still charge $10/GB. I could consume that much data within a couple minutes on a cellular network. Sometimes I worry about accidentally using up all my bandwidth when I press the wrong button on an app.

You might want to look into the new web authentication API.

Yes, a password manager is just an encrypted database for your passwords. 1Password synchronizes all of your passwords across devices and makes sure everything is secure. You only need to remember a single "master password", which is never sent outside of your local device. In the event that you lose or forget your master password, the password vault is completely unrecoverable.

1Password can also store other information besides passwords such as credit cards, software license numbers, passport numbers, etc. There is also a secure notes feature for storing arbitrary text.

The other password manager that I tried before 1Password is Lastpass. I ended up choosing 1Password since I think it's better designed and overall feels slicker. The /r/lastpass subreddit is littered with complaints about broken updates and bugs...

I highly recommend using a password manager. I finally bit the bullet and started using 1Password a few weeks ago, and I haven't looked back since. It's just so much better than having to remember a thousand different passwords.

Besides securely managing passwords, you can also use a password manager to secure your digital legacy. 1Password has a feature where you can print out "emergency kit" sheets that has the information required to access your password vault. I printed out two of these sheets and gave them to trusted family members in sealed envelopes. In the event that I become incapacitated, they will be able to access my accounts.

What happens when the num_digits field overflows?

To call a function declared by a type class, you can use dictionary passing or monomorphization. GHC uses monomorphization as much as possible since it has zero overhead. Dictionaries are required only in very specific situations.

I interviewed at a company called Chip Scan which is a startup that aims to detect backdoors in chip designs. I didn't end up accepting, but it did sound like an interesting job.

Hindley-Milner monomorphizes the type of a function inside the body of the function, so if you make a recursive call on a different type the type checker will complain. There are many subtle "gotchas" with type inference that are often not encountered by most programmers. Unless you've implemented a type inference engine yourself, you probably don't know about these sort of restrictions.

Here is an example of the type error (I used the datatype from the Wikipedia article on polymorphic recursion): https://repl.it/@CalebHelbling/PolymorphicRecursionError

Type inference for systems with subtyping is a different problem. There has been progress on this issue, most notably Stephen Dolan's thesis on "Algebraic Subtyping":

https://www.cl.cam.ac.uk/~sd601/thesis.pdf

I've looked at this page and some other documents since I wanted to implement type classes in Juniper (a FRP language for the Arduino). It turns out that the dictionary passing is only necessary if you want to support polymorphic recursion. In all other cases, the call to a function that is part of a type class can be monomorphized.

Hindley-Milner has problems with inferring types in the presence of polymorphic recursion, and a user provided type annotation is usually necessary. Polymorphic recursion does allow some cool things such as arbitrarily nested lists. This is a feature that users from a dynamically typed language might miss.

For those who don't know Starcraft: microing refers to the small scale/precise maneuvering of units during a battle, as well as activating special unit abilities at certain times. In the lower level leagues microing is not as important as having a good overall large-scale "macro" strategy. The microing really starts coming into play at the higher levels where it can give the edge needed to win a battle.

I am not very surprised that machine learning has been able to successfully execute micromanagement. In my opinion the macro decisions are more interesting, since they typically require higher level reasoning (and comprehension of what the opponent is doing). We have yet to see an AI system that can successfully execute macro strategy when playing against a human opponent.

That's really neat! What do you use for training data?

Yup, I'm wondering how many more days Allo has to live. I don't even bother with the proprietary Google messenger apps anymore because I know they'll be killed within a couple years or less.

I use a pretty large set of editors. PyCharm + CLion + Sublime + VS Code at work, and Visual Studio 2017 + VS Code at home (I primarily work on gamedev on the side in Unity).

I just got a new laptop with a 4k screen, and was very disappointed to discover that most coding editors do not support the high DPI screens. VS Code is one of the few that does, so that is what I'm using now. The text in Visual Studio 2017 actually looks worse on the 4k monitor since Windows is forced to use some sort of fuzzy scaling.

I've had problems with malicious ads redirecting me lately. I began to suspect that my phone was infected, but it turns out that this is now a common occurrence:

https://www.wired.com/story/pop-up-mobile-ads-surge-as-sites...

I don't know why the ad platforms even allow custom JavaScript...

I've often thought that this sort of thing would be great for user interface design. "Recursive drawing" in combination with a CAD style constraint system would be really cool to see. My preferred name for such a system would be "inductive constraint layout".

I've used ENet for my games which is pretty similar to this project. Unfortunately ENet does not support IPv6, and the pull requests on the ENet repository appear to be ignored. The ENet author refuses to accept any changes that break the ABI, which is highly unfortunate.

http://enet.bespin.org/

I highly recommend GameMaker for teaching programming (this is actually how I started programming). The drag and drop interface is easy to understand, while also being powerful enough to use on real projects. There are even numerous examples of successful indie games that have used GameMaker. It's pretty great for rapid prototyping in game jams as well.