JHonaker

A geometric algebra is another name for a (more well-known?) algebraic structure called a Clifford algebra. If you know what an Exterior/Grassmann algebra are, then it's that with a metric structure.

If you don't know what any of those words mean, then it's a way to represent n-dimensional geometric objects and operations on them all within an algebraically closed environment. Moreover, (at least in the specific algebras I'm familiar with) you can view a lot of transformations/operations as reflections or compositions of reflections of one object with respect to another.

It's quite beautiful really. Transformations compose nicely and produce new (uniquely represented [1]) objects. You can work in a way that mirrors your geometric intuition instead of getting lost in the mire of representational details.

[1] One of the real sticking points of trying to embed everything in a vector space is that a lot of the representations overlap, and you have to operate on them specially. Planes are represented as normal vectors, and so are regular vectors.

I agree with you! It's not really that bad. The two biggest problems for me are the lack of namespaces/modules like you said, and the default dynamic scope (with the ability to enable lexical scoping).

They're both basically just ergonomic issues at this point, with prefixing names and always setting lexical scoping being cultural norms.

My favorite thing he’s doing right now is suing GitHub/Microsoft for blatantly violating almost all the licenses on their hosted code for GPT.

https://githubcopilotinvestigation.com/

I’m also an enormous fan of his book, Beautiful Racket. It’s what set me off on the path of being a self-identified Scheme programmer.

Preach.

There are way too many people that conflate MSE or other abstract technical measurements of model performance like they actually represent the impact any model has on a problem. Even if we could somehow perfectly predict an actual realization instead of a conditional expectation that still forgets to ask the question of why we predicted that. Are we exploiting systemic biases, like historically racist policies? Almost definitely (unless we've consciously tried to adjust for them, and still we've probably done that incorrectly). I've become much less interested in models that basically just interpolate (very well I might add), and more in frameworks that attempt to answer why we see particular patterns.

I don’t think anyone actually thinks it’s anything other than a joke. Hell, many Emacs users use a very complete Vim emulation package called Evil. In fact, the two most popular “pre-configured setups” are designed around Vim keys.

Yea, I fully anticipate that they'll be fixed eventually. Having toyed around with Julia since it's 0.3 days it's amazing to see how far the language has come so quickly. I only bring that stuff up because I think it's better for people to realize that there are some pain points in Julia. If you come to the language with a more realistic viewpoint of it, then I think you're more likely to stay.

The one that gets me most often is the boxing of captured values. I come from Scheme/Racket and I'm just so used to exploiting closures. I also struggle with "over-typing" function declarations. I'm getting better about that though, now that I have a better handle on what exactly the dispatch mechanism is doing.

I really like Juila. I just came back to writing a lot of Julia (after I realized I was starting down the path of recreating the enormous effort of the Julia devs by adding specialized multiple dispatch to Scheme). It feels incredibly flexible yet performant in a way that most languages fail to realize.

However, it can be incredibly frustrating sometimes. This article is a great example of the kind of wizardry you have to do to figure out why something isn't as fast as you thought it would be. There seem to be a lot of, "If you don't do it this way, it'll still work, but it will slow as hell," practices that are in the community's collective knowledge, but are big stumbling blocks in the beginning. (making sure you have concrete types in struct fields, the Val type and dispatch vs an if statement, how to write "type stable" code, etc.)

I had a lot of incorrect ideas and assumptions about the type system and the specialization mechanisms until I watched the Julia internals talk on YouTube from like 2014. This is probably due to it still being relatively young, and it has gotten better over time.

There are a lot of things to like about Julia that make this bearable though! It's really a great language to work in.

I completely disagree. There is no substitute for playing actual opponents. Playing against people is completely different than playing against the CPU.

You need to lose your fear of losing as quickly as possible to enjoy fighting games. (Also you won’t lose as much as you think.)

Tekken in particular you really just need a couple of core strings and learn to block.

I credit Scribble with the amazing depth and coverage of the Racket documentation. It is truly some of the best written documentation I’ve come across. Everything in the core language has at least two sources: one in the Guide with a casual, high-level explanation of the topic with examples, and one in the Reference that documents everything in excruciating detail.

I didn’t realize how much I appreciated it until I dove back in after learning Nix.

Not really a place for discussion, but https://okmij.org/ftp is hands down the best place on the internet if you're into PLT.

I figured there would be. It just provides another point that it's not really understanding, and it's not really "extracting the pattern".

I really wish half the effort on generation would be spent on leveraging them for guided exploration. It wouldn't matter if you were provided slightly incorrect suggestions for "likely relevant functions," but even likely correct source code doesn't cut it.

Or even snippet prototyping. Like don't use the generated text directly, provide an interface that I can use to transform the generated code into a snippet that I can save.

I want to preface this by saying I'm not trying to hate on the author. Good on them for applying a new technology to try to solve an annoying part of their work. I hope they continue to be creative and fearless in their approach to problem solving. The comments that follow are born more out of my increasing frustration with the ML and ML-adjacent communities that seem to have failed to learn anything about the dangers of hype and over-promising from the 1980s (and really just human history in general).

While it's an undeniable achievement that these kinds of models can generate syntactically correct code or even the correct solution some of the time. I find it hard to get excited about them for anything but boilerplate (and even they I think they pose the greatest risk, lulling one into a false sense of security until you get bitten).

If we take formal methods as the way to ensure the code we've written provably does what we say it does, we had better damn well make sure that what we've said it does is what we want it to do. In other words, I feel like this is offloading the most important part of the problem, and the one that can't be checked automatically onto a very iffy situation.

We should really stop to consider what these kinds of models are. Pattern recognizers, not thinking machines. See the author's quote about this theorem:

    Let G be a Lie group. Then the tangent space at the identity is a Lie algebra.

which is translated into the theorem:

    theorem tangent_space_at_identity_is_lie_algebra (G : Type*) [group G]
      [topological_space G] [topological_group G] [charted_space G] [smooth_manifold G] :
        lie_algebra (tangent_space_at_identity G) :=

Their quote

    For some completely unclear reason, the app decides to consider a Lie algebra over \mathbb{R} instead of my explicit instruction to use a field k, and that’s even before addressing the fact that tangent_space_at_identity is not a mathlib definition. I tried a few different responses to the app’s initial attempt, and couldn’t get it to do anything helpful.

They try to fix this by "prompt engineering":

    You must specify a field for your Lie algebra as in lie_algebra k M

which leads to the change:

    lie_algebra R (tangent_space_at_identity G) :=

This is exactly the kind of boilerplate we'd want to automate with this. The specification of a Lie group seems onerous here, but it's also exactly the kind of mistake that makes using this kind of mistake that makes relying on these systems so scary. Here it's a syntax pattern, and we have to end up proving the correctness, so maybe we wouldn't notice the mistake here. It's almost certainly going to be missed often in (choose your favorite dynamically typed interpreted language that let's you find all this fun stuff out at runtime). There is no understanding of what we do and do not know. What we've proved or have yet to prove be damned.

I guess my point hinges completely on what you consider "knowing" to be. It's kind of funny. Nobody would ever say that a compiler knows how to transform one language to another. The personification of ML really irks me. I try very hard to avoid it, but it's easy to slip.

I don't want to say we should throw the baby out with the bathwater. I do think that there are uses for these things. The recognition of the pattern for the syntax to specify the characteristics of a Lie group would make a great first pass at a snippet template. (This is ignoring the question of why isn't Lie_group a definition that just pulls in the necessary requirements on G rather than needing to specify them each time. If one of those is not really a requirement for Lie groups, forgive me, my exposure to Lie theory is very limited.)

I would be much more interested in a tool that allows me to identify patterns in the code with a summary printout or something like the likelihood of a given pattern in the neighborhood of my cursor. The generation part is just not good enough to trust IMO.

Also, I really hope we see some sort of backlash for MS just deciding to just ignore everyone's' licenses.

Well I did it in Scheme, but there's no reason it wouldn't have worked in Julia. If you're interested in this, the relatively recent Software Design for Flexibility by Chris Hanson and GJS [1] has a pretty good couple of chapters on creating a generic procedure system (and a lot of other really cool things). Edit: In fact, it has an example doing just this example: symbolic alternative evaluation. Where they get the closed form solution from a simple differential equation solver without changing any code in the solver.

Now that I think about it, this idea is pretty much how automatic differentiation works in Julia. (Really everything in Julia.) You can look at the libraries Flux.jl and Zygote.jl. There are probably more now, I've been out of the Julia ecosystem for about a year or so.

[1] https://mitpress.mit.edu/9780262045490/

I mean, the systems described in the TAotMOP, and CLOS, are early implementations of multiple dispatch, which is the core paradigm of the Julia language.

Julia doesn't go quite as far as implementing a MOP. It doesn't allow for the arbitrary modification of the semantics of evaluation in the language itself, but I'll bet it'd be pretty easy to write a MOP on top of the base language.

I think the ideas in the MOP are exactly those that you've identified in the first sentence. You can draw lines that allow you compare the object implementations of particular programming languages. This isn't so useful when you're just working in a single language, but in something like CL or Scheme (or Racket more specifically) where a major design strategy is to design a DSL that suits your problem, having a MOP really lets you tailor that DSL to the problem in ways you might have a hard time doing otherwise.

For example, I once wrote a library for a computing in particular area of mathematics, but I also wanted to see some symbolic equations. I just changed the basic evaluation semantics of my numeric operators to operate on symbols as well as numbers and to run some simplification logic and boom. I had a rough view of the equations that were actually being used to compute that I could use to debug or tidy up and write down.

This is really cool, but I the red font coloring is very hard for me to read. I have to highlight the text to be able to clearly read it.

I was too frustrated by the insistence on libre purity by the main Guix channels, even if I think it is superior software, so I switched to Nix (and actually went fully NixOS after a month or so).

I had R package compilation issues on 4.2 as well, but I also had them on my Windows work machine. I'll try to test things out and see if I can figure out if it's still an issue.

It's rare someone follows up on a "I'll post this soon." Thank you for the excellent list! Quality books about writing are hard to come by; books about non-fiction are even scarcer. I'm looking forward to reading through your recommendations.

This… shouldn’t happen. The whole point of Nix and Guix is that you can have multiple versions of the same thing in different dependency chains. It’s designed to avoid precisely this issue. If a package relies on something it should be built before hand, and if and only if it’s identical (well content addressable vs inputs is a whole other thing) it will share the dependency.

Would you mind sharing exactly what the R packages you were having problems with were?

I wonder if you were relying on globally installed versions of things instead of an R installation that had the packages wrapped into its environment. I’m more familiar with Nix, but you’ll typically see people do something like add this to a local project build input or development shell:

    r-lang.withPackages with rPackages; [ r-ggplot2 r-data-table ];

Rather than

    nix-env -iA r-4.2 r-ggplot2 r-data-table

Or the Guix equivalent

    guix install r …

Globally installing things can lead to situations where you think it should work but if you really think about how the store dependencies work, they don’t.

I second the request. I think communication is a drastically underappreciated skill even though we have all suffered due to poor communication professionally (undocumented libraries, poorly specified requirements, an inability to generate buy-in for our ideas).

I'll throw in a few recommendations I have from some recent research at work:

https://eugeneyan.com/writing/writing-docs-why-what-how/

https://eugeneyan.com/writing/ml-design-docs/

https://reqexperts.com/wp-content/uploads/2015/07/writing_go...

https://eugeneyan.com/writing/what-i-do-before-a-data-scienc...

I went in expecting someone to have discovered Mosh and setting up a self-hosted VPN (a road I'm about to embark on).

What I got was way cooler. I really love reading after-action reports like this.

It depends. For books that have exercises, no. I just do the exercises. For books without exercises, yes. Usually I try some of the key proofs, or to write some code that uses it.

Usually with math/stats books I’m reading it for a specific purpose, so I start using it right away. For instance, I had to do some differential geometry and Lie algebras recently. I did problems in the book until I understood the material and then wrote some code that used it.

For the PL theory stuff, I mostly just tried to write a simple Scheme-like language interpreter that added worked with the thing or tried to use a new language that leveraged the ideas. When I was reading about compilers I basically just started adding compiler stages to a simple compiler. When I was reading about dependent types I played around with Coq, Agda, and Lean.

Emacs’s undo system is quite frankly amazing. Not only can you recover any state through an arbitrary number of undo and redo operations, but you can also localize the undo/redo operations to a selected region of the buffer.

It’s the number one feature I miss when I’m using another editor.

I read a lot. I basically always have at least one fiction book and one technical book (usually math, statistics, or programming) that I'm reading at any given time.

I think I'm an outlier on the other end. This year I've read 3 or 4 books on programming language theory, one specifically about Scheme, and The Rust Book. I've picked up and read a a little bit to a lot from a lot of math textbooks.

This is all probably stems from graduate school training and my ADHD... It also helps to have PDFs or online versions and a baby that had some serious aversion to sleep, so the only thing you can do is sit in the dark rocking her and read...

I didn't think I'd ever see anyone list the BMV as a good example of anything. I just recently learned that we can now renew our driver's license online though, so maybe it is getting better.

I don't think using frequentist methods makes anyone a eugenicist by transitivity, but I do think it's important to understand the motivations behind the development of notoriously confusing, misapplied methods.

If you consider the context, I think the author's position that the confusing aspects of it are very likely because of the desire to (falsely) profess that we have removed all aspects of judgemental decisions from a particular procedure which was driven by the desire to convince people that it wasn't their opinion, but fact, that they were genetically superior beings.

I agree that it combines things to try to form a more rhetorically convincing argument, but to me, most of them are convincing on their own.

I think it's also more representative of the author's view that he believes that it was precisely the extreme conviction that white, European society was morally, culturally, and genetically superior that led to the development of methods that were devoid of a preference for position (and thus completely objective) that led to many of the confusing methods in use today.

I also think there's a useful distinction to be made between the terms "genetics" and "eugenics". I don't think anyone would argue that there aren't meaningful genetic differences between different groups of humans. It's the degree of extremity and the sheer horror of the conclusions drawn and methods advocated for by eugenicists that makes the distinction. Like yes, an increased probability of sickle cell anemia is a direct result of a population's adaption to an environment rife with Malaria, but that doesn't justify (I'm vaguely paraphrasing Pearson or Galton from memory here) the genocide and replacement of those people with those more capable of exploiting the natural resources of their environment (meaning white Victorian aristocrats, or at least slaves "employed" by white, Victorian aristocrats).

It's interesting to see conversations around human genomic differences rise up again with things like CRISPR, and I guess that would fall under "eugenics" but it's clearly not the same thing.

I also take issue with the idea that "it's not the science that's bad" in even tacit defense of early 1900s eugenicists. It's very telling to read the descriptions of the studies and conclusions that Galton, Pearson, and Fisher performed. It's telling to watch them discount clear counter-examples or re-define the desirability of traits, to watch them call an aristocrat industrious and a Jewish migrant greedy, to watch them all but spell out exactly what their conclusions are, develop methods "devoid of opinion" and of "pure objectivity through data," and then unabashedly call for the justified forced sterilization, mass genocide, and the denigration of non-white, non-aristocratic, non-male, non-Europeans as deterministically inferior.

Sorry for the rant. I wasn't aware of the horrific history of statistics when I decided to take it up, and now I take it as a person mission to fight against the "let the data speak for themselves" attitude. I want to make sure people know what "objective in-difference" with implicit assumptions can justify.

Maybe more to the point of your points :D :

1. Yea, not much more to say about that.

2. We do agree based on our conversation elsewhere in these threads.

3. It is bad because of both its immoral position and its faulty scientific work. The justification is equivalent to that of any of the latest unrepeatable studies on power posing and finite will power.

4. Again, it was intentionally misleading (as much as someone wholeheartedly convinced of something is misleading), and doesn't stand up to even contemporary criticisms (IMO). I do think we have objective standards, but I don't think those objective standards are reflected or implied by NHST.

Nor do I think a modern person using NHST is somehow a eugenicist by transitivity. It's more that my position is NHST has infected the scientific community precisely because of its self-reported objectivity. I think it's my field's (statistics) personal failure to bear. I think we stayed content too long as the dispensers of "objective methods" without really doing a good job of stressing the nuances and frailties associated with any particular method.

Yea, it's difficult, especially because the right way to interpret them is both confusing and unhelpful.

Why do I care about the properties of my test if it doesn't say anything about the what I should know about the thing I'm measuring in this specific case. Covering the true parameter 95% of the time is useless if I can't make that leap to saying this one contains it 95% of the time.

It's so confusing, and I say this as someone who has devoted all of their adult life to statistics.

I mean the discussion of the reasons and motivations for the development of many classical techniques actually does lend credence to the argument that they are misleading (by design).

That said, I find the technical arguments against NHST to be just as necessary (and I also find them just as convincing).

I'm curious what your actual objections are to this?

I read this article when it originally came out, and I think it's mostly a historical discussion. But, I can summarize the "leap of logic" that significance testing makes for you.

If T(X) is a statistic designed to measure a quantity A, then significance testing are statements that say

P(T(X) is close to A | A) = P

In other words, the statistic T is close to the true value of A (which is fixed) with probability P. So far, I think this is pretty straightforward, but the tricky part comes when you apply the idea that "closeness" is symmetric. In the physical world, if A is close to B then B is close to A, but that isn't necessarily a valid thing to infer based on a significance test.

The thing ostensibly implied by a hypothesis test is that the these two are reasonable to exchange:

P(T(X) is close to A | A) = P (A is close to T(X) | T(X))

The interpretation of the right side is what you really want. The probability that the thing I want to know about is close to my statistic given my statistic. Unfortunately, this is not true in general. It's this misunderstanding that's at the heart of "logical fallacy" of frequentist hypothesis testing. No matter how careful we statisticians are about p-values, empirical evidence has shown that this leap is what people make time and time again (including us).

This is to say nothing of the absurdity of interpreting probability as a long run frequency for events that happen once or relying on results that formulate things that way when you're doing iterative analysis and development (as all actual inferential processes are).

I'm a statistician, and I have to say, I agree with the fundamental premise of the author:

The problems with null hypothesis significant testing (NHST) are a direct result of two things:

1. the false air of objectivity it provides

2. the lack of consideration of the relative likelihood of the data under any alternative

I also agree that Pearson, Fisher, and Neyman knew what they were doing when they appealed to these methods' "objectivity" to support their terrible views.

I also find it just absolutely hilarious that Fisher's adamant opposition to Bayesian inference was the (in his opinion) absurdity of the uniform prior distribution, and yet one of his most famous contributions (Maximum likelihood estimation) is essentially the same result you'd get from the Bayesian estimate of the most likely parameter with a uniform prior.

If anyone is interested in the history of this stuff along with a discussion of the philosophical meaning of probability from the same author, his book Bernoulli's Fallacy is a pretty good read. My only qualm with it is the lack of much exposure on how to actually do Bayesian inference or a discussion on what it's like to do it in a modern setting. There's a brief few pages at the very end that mention things like Markov Chain Monte Carlo and the fact that most modern Bayesian inference is necessarily numerical approximations to the posterior, not the analytical solution (but that's not necessarily a bad thing nor is it an escape-able problem).

If you want a book that makes many of the same philosophical arguments about probability but also backs it up with math and descriptions on how to do Bayesian inference, Edwin Jaynes's Probability Theory: The Logic of Science is for you. It's quite literally a modern cult classic. People that read this book tend to go out and evangelize for it.

My final comment is that there is a place for frequentist methods, even if you're a modern Bayesian. The theory of Markov Chain Monte Carlo (one of the key methods in making modern Bayesian inference work) is exactly the domain of frequentist probability: Long runs of repeated things. Many of the diagnostic tools for assessing whether the inferences of MCMC results can be trusted are hinged on results from the frequentist interpretation and are steeped in their strange-ness. For instance, the lack of an issue is taken lack of evidence of an issue rather than evidence of no issue. The interconnected-ness of the two is unfortunately missing from the author's work (that I have seen).