mreid

Don't sell yourself short. I only started running when I turned 38. I ran my first marathon in 3:53 when I was 42, and my second one in 3:41 when I was 44. I'm now nearly 48 and am planning on doing another one later this year and am training for a sub 3:30 time.

It makes me glad to see that Aleph (and ILP more generally) are still being used. It's a really powerful and interpretable way of constructing code from examples.

I made a few contributions to Aleph as part of my PhD on doing transfer learning in ILP and really enjoyed working in Prolog.

https://mark.reid.name/bits/pubs/unsw07.pdf

It might be easiest to give a sense of what "unprovable but true" means by way of an imagined example.

Goldbach's conjecture is that "every even number bigger than 2 is the sum of exactly two prime numbers", so 4 = 2 + 2, 6 = 3 + 3, 8 = 5 + 3, etc.

For this statement to be *true* it just means that every even number there must exist two primes that add to that number. This is a statement about infinitely many integers.

A *proof* of Goldbach's conjecture consists of a finite number of formal reasoning steps that start with some axioms and end up at the statement of the result.

To this day, it seems as though Goldbach's conjecture is true. It holds for every number we've been able to test. However, no one has proved it is true or false yet (or proved that it is unprovable).

The proof of Gödel's result's involves very carefully formalizing what statements and proofs mean so that they can be encoded as statements about arithmetic. He then shows there is a statement with encoding G that says "The statement with encoding G cannot be proved" – if it is true, then it cannot be proved.

It's kind of confusing at first, but there is an easier way to get an intuition why there might be true statements that cannot be proved. Think of each statement about the natural numbers as a subset where each number in the subset makes the statement true. There are uncountably many subsets of the natural numbers (by Cantor's diagonalization argument). Proofs are finite chains of finite statements so there are only countably infinitely many of these. Therefore there must be subsets/statements that are true that do not have a matching proof.

The approach that Gödel's proof takes is not too different to the above argument – it is essentially a diagonalization argument – the complexity is in making the encoding of statements are numbers very precise.

I learnt the strategy of "satisficing" from Herbert Simons' "Sciences of the Artificial" book. It suggests not bothering to split hairs amongst alternatives once options have met some threshold for acceptance.

https://en.wikipedia.org/wiki/Satisficing

I'm not the parent, but I think they are referring to William Sethares' book "Tuning, Timbre, Spectrum, Scale": https://www.springer.com/gp/book/9781852337971

It's a fascinating take on how those concepts relate to each other. You can find some of the papers that were synthesized into that book on his web page: https://sethares.engr.wisc.edu/pubs.html

Definitely looks like synesthesia. FTFY

I never thought of it as onomatopoeic before but, if I string the sound out – pooooooour – making the “o” noisy and its pitch rise slowly it does remind me of the sound of a cup being filled from a tap.

Ok, I still think it’s an unusual example but you’ve convinced me.

I retract my earlier comment and replace it with a “today I learned” for me. :)

Another thing to add to your learnings for the day is the difference between “metaphor” and “onomatopoeia”. The latter is only for words that are related to a particular sound they are describing, e.g., “quack”, “bang”, “sizzle”, “whoosh”.

“Pour over” is the sense you describe would be a metaphor.

A recommendation for IoBC should come with a strong warning. It is a extremely short piece but, word for word, one of the most hard-hitting and distressing things I’ve read.

It’s a testament to DFW’s talents as a writer to get so much impact out of so few words but it’s an incredibly tough piece of writing to recommend. Especially if you have young children.

I don’t know about puns being the lowest form of humour, but the bun is the lowest form of wheat.

According to this article Service Workers is on by default in the latest Safari Tech Preview: https://9to5mac.com/2017/12/20/safari-preview-46-service-wor...

I saw this at the end of the article, was hit by a huge wage of nostalgia, and had to comment:

> Update 9.10.2015: exec.library base address is stored in > memory address $4. The library base address itself is not $4.

I learnt 68k assembler for the Amiga in high-school and made _exactly_ this mistake. After many "guru mediations" (the flashing red and black box) and much head-scratching I finally realized the problem.

It was a big breakthrough in my early days as a programmer and taught me a solid lesson about indirection and pointers.

Ditto. That was the first language I was taught at UNSW (in 1993). It was quite a change from the BASIC, assembler, and C I had taught myself in high-school.

It's been a while since I looked at it, but I think Computational REST (CREST) tried to do something like that: http://www.erenkrantz.com/CREST/

I haven't used Hugo but my entire site is generated with Hakyll: http://mark.reid.name/ . I've found it works well for a mixture of blogging and more static content.

Jennifer Egan did a New Yorker story called "Black Box"[1] that was intentionally composed as sub-140 character chunks. It was released on Twitter tweet-by-tweet and collected in a New Yorker science fiction special last year.

She is an impressive and experimental writer (check out her "Visit from the Goon Squad") who does a great job with this form. The thought-sized chunks give the story a lot of space and cleverly invites the reader to imagine a lot of context.

[1]: http://en.wikipedia.org/wiki/Black_Box_(Jennifer_Egan_story)

So you are willing to discredit everyone's claim to the importance of their trips based on examples of people who falsely claim that their's was important?

I don't think anyone here is arguing the position that LSD will have a profound impact on anyone who takes it, just that it can give people a different way of looking at the world and sometimes that leads to novel and important insights.

Replace "LSD trip" with "university degree" in your example. Sure, there are many people who take a physics course and feel as though they have had profound insights but actually haven't. However, you can't use those examples to argue that everyone who claims their physics course has inspired their life and work is lying.

Is there evidence to support the claim that Van Gogh believed cutting his ear off was important to his life and work? If not, then this is a very different example to that of Jobs' claim and is an equally poor way of refuting a position.

Sloth, apathy, and procrastination.

Firstly, I agree that machine learning is effectively parameter search but the name an artefact of history and we both appear to understand what it means so I don't see how this adds to the argument.

Secondly, no, John didn't say learning theory is "useful only in a crude way", he said, "learning theory is most useful in it’s crudest forms". Big difference. And besides, he says right at the beginning of the post that he believes "learning theory has much more substantial applications [than generating papers]".

To be convincing, theory needs to be precise – if you are not careful about what you are talking about it is easy to believe things that are not true. However, what I think John is saying is that value of theory comes from afterwards abstracting away the precision and understanding the main message (i.e., the "crude form") of a theoretical result. In general, I don't think it is possible to get to a convincing "main message" or "crude form" without someone having grappled with the details.

No matter how many experiments you run, you will only ever show that an algorithm works well or not in a typically small finite number of cases. What theory does is look those cases and ask something like, "It seems that every time X is true of my problem, an algorithm with property Y works really well. I wonder if that is always true?" This type of question gets carefully formalised and then (hopefully) answered. The process of formalisation (i.e., defining things carefully) can yield new ways of thinking about things (e.g., over fitting and bias-variance trade-offs), and having an answer to the general question means that you can be assured that future uses of your algorithm will behave as expected.

You seem to have an unshakeable opinion that mathematics/theory/"pencil-pushing" is inherently a waste of time. That's a real shame. Why do you believe the pursuit of precision, insight, and proof are somehow inferior to running experiments? I find both to be valuable and the interplay between the two extremely rewarding.

I'll ask him whether he considers himself mostly a pencil pushing fraud when I see him in December. I'll also see what he thinks of the claim that VW's usefulness is "not due to any theory". I'll think you'll be surprised given he's written a blog post on this topic titled "Use of Learning Theory" http://hunch.net/?p=496 at his blog which, by the way, is called "Machine Learning (Theory)".

It's easy to dismiss what you don't understand but you should consider the possibility that it is significantly more difficult develop algorithms like those in VW and "bang out" implementations of them that are fast and correct.

I see that you use VW for machine learning. You should ask the lead developer (John Langford) what he thinks of the efficacy of mathematics for machine learning given most of the algorithms it uses were derived from theoretical considerations (e.g., SGD, LDA, reductions, etc.)

Would you describe him as a "pencil pushing fraud" too?

There is no argument that fundamental research (including, but not limited to pure mathematics) is a high-risk investment with a small chance of a large pay-off. But, like most good, high-risk investments the potential pay-offs can be massive and, importantly, not always well-understood in advance.

Would we have GPS without relativity which in turn depends on non-Euclidean geometry, initially considered a curiosity? What about public key encryption, built upon number theory, proudly described as "useless" by the pure mathematician G.W. Hardy? There are so many examples like these.

Be careful of committing the fallacy in assuming that because a clear line of ideas can be traced back from the present day that finding that line was as easy as recounting it.

Out of interest, what is your position of the spending on sport, the arts, and other cultural endeavours?

Yes, that's more or less how it was done. The explanation is here: http://www.spinellis.gr/blog/20090805/index.html

Here's a self-referential URL that doesn't use any URL forwarding/shortening service:

https://twitter.com/#!/selfrefer/status/3128391843

Right, because "experience in algebra, established reputation in field, and familiarity with communication of math" can be acquired by anyone with only a modicum of smarts.

I think you are underestimating just how intricate and challenging understanding a proof can be. Especially one of this length and by someone of this standing.

I wrote a very similar blog post a few years ago when I first encountered the same fact:

http://mark.reid.name/iem/warning-high-dimensions.html

Thanks to the comments, I learned this is a common warning, used in at least three textbooks.

I think you meant to say "pedantic nerdery".

I agree, and had no problem understanding his intended meaning but I was irked by the unnecessary misuse of technical notation to attempt to get across an idea that would have been perfectly easy to write without it (e.g., "10 times" and "1000 times").

(Pedant warning)

I think you may be abusing Big-O notation a little here since, technically O(10) is equivalent to O(1000)—and both are O(1).