Math Foundations from Scratch | Hacker News Reader

Math Foundations from Scratch

learnaifromscratch.github.io

213 pointspaulpauper128 comments

This site bills itself as follows:

> This is a draft workshop to build from scratch the basic background needed to try introductory books and courses in AI and how to optimize their deployments.

I'm not sure learning the things on that page – Peano axioms, basic real analysis, etc. – really helps with deploying AI. I know plenty of talented graduate students with papers at NeurIPS, ICML, and another top conferences who don't know a lick of set theory, so certainly it's not necessary, even for highly academic work.

Based on my experience and what we know about the psychology of learning, I'd recommend finding a motivating ML project or application (e.g. point the phone at a sudoku and it overlays the solution), and then working backwards to figure out what you need to know to implement it. Probably it will not be the Peano axioms.

Maybe type theory is good to know! I would like to see cutting edge NN libraries for Coq or Lean or Optimization ecosystem like Scipy or Scikit for Agda.

You develop your algorithm and you prove a bunch of theorems about it at the same time, incredibly difficult but totally wild!

There's some scattered projects like this https://github.com/OUPL/MLCert

There are two schools of machine learning education: Math first like Andrew Ng and code first like Jeremy Howard. That both of these leaders have found academic and professional success shows you should approach machine learning in whatever way comes more naturally to you. Later on, you can approach it from the other angle to improve your understanding.

To clarify, I'm not arguing that math isn't important. You need your linear algebra, your probability theory, and so on. That's non-negotiable. But set theory? Type theory (as someone below suggests)? I would avoid unless you're separately interested in mathematics for its own sake.

I’m fairly certain they’re not suggesting you need to know type theory for ML, but that theorem provers should have more ML libraries. My wife does some ML as a postdoc at Stanford and I doubt she’d even know what type theory is.

I agree, however I'm not sure of how far you can get in probability theory without set theory. For instance, I recall my stats prof defining a lot of sets in class, e.g. sigma algebras.

Another point of clarification: I'm all for telling people what sets are. I'm less warm toward discussing things like ZFC, esoteric cardinality issues, and certain paradoxes (and large cardinals, forcing, etc.), which is typically what mathematicians mean when they say "set theory."

Right. What most people need to learn is "algebra of sets," essentially.

Its not immediately practical... but spending a lot time learning a science like math in depth makes learning various applications very easy later.

I think sometimes educators have to let people be motivated by some particular application to teach them something more important. Perhaps that is what the author is up to?

How do you even do serious linear algebra or probability without set theory?

I have a PhD in physics and studied graduate level mathematics. Naive set theory is sufficient for basically everything except a few instances, where you have to bite the bullet and invoke the axiom of choice. But this happens rarely and typically is only relevant for instances where there are more constructive but less general approaches.

For the most part sets are used as a notational device, you almost never use any axioms about sets explicitly. If you asked most professional mathematicians to give you the ZFC axioms (or whatever) they would need to look them up.

yes but most of the math-based approaches to ML start from real (multivariate) analysis, or from probability theory. i haven't heard anyone claim that Peano axioms are important. as much as I will defend the notion that applied mathematics is extremely important for ML research, peano axioms are unnecessary and will just cause confusion

While I would agree you don't really need to know or worry much about mathematical foundationalism to grok machine learning, anyone writing graduate level papers is going to understand some set theory since probability theory relies upon it. Even just the high school treatment tends to start out with intuitive visualizations of univariate discrete probability like defining a probability as the cardinality of an event space divided by the cardinality of a sample space, defining conjunction and disjunction as set intersection or set union of event spaces, defining mutual exclusivity as disjoint event spaces, and so on.

That said, there is probably no reason for anyone to dig too deep into theory without understanding why it matters first. And I don't think it ever matters if you're just trying to solve a problem with ML. Even if you want to overlay a solution on an image of a Sudoku puzzle, you don't need to understand why a particular algorithm works in order to be able to use it. Theory only matters if you're either trying to formally prove something works or possibly to guide you down more productive research paths in developing new algorithms, though honestly I'm not even sure that's really true any more. Just gaining a ton of experience with various possible techniques might generate intuition that is as useful or even more useful than actually understanding anything.

I almost hate saying that, as I have a deep personal love of math theory and think it is worth studying to me at least for its own sake, but this is sort of in the vein of whoever invented the curveball almost certainly didn't know much about aerodynamics and didn't need to. Discovering something does work may happen decades or even centuries before anyone figures out why it works.

Thesee are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning".

From a cursory look the linked page covers a broad range of subjects necessary to understand the last 65 years of AI research, not just the 9 years since the deep learning boom that most people are familiar with.

That you know many graduate students with published papers in major conferences in machine learning who don't have a background in AI is ...unfortunate? I guess?

Which topics in AI require knowing the Peano axioms?

Maybe if you are just experimentally gluing black boxes together and trying to make observations about them

>ah these parameters/network layout did this, here's a guess as to why

Or just mindlessly gluing stuff to solve a particular problem

>we took x model and trained it to recognise cats

then yes, you probably do not need much mathematical knowledge. However if you are researching fundamentals, neural tangent kernel, neural odes etc, then you will probably be pretty decent at mathematics. You will also probably have been exposed to Peano at some stage in your life, even if you don't remember it. Its just good background mathematical character development.

I would say even if you are more of an applied person, being exposed to the fundamentals probably helps shape your thinking. Yes you can still be a good productive programmer, but maybe you will have something missing. The person with a broader background may be quicker at solving a new problem, they have a bit more to draw on.

I meant to but was not explicit enough. My question to the parent was focusing specifically on their statement

"These are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning"."

While I agree that some amount of mathematical maturity is useful, I think it is a far reach to claim one has to know the Peano axioms in particular, at least in their set-theoretic form. I assume you don't mean knowing and using them implicitly, since you seem to be arguing for explicit mathematical training.

At the risk of being overly presumptuous, the opinion you give sounds like that of someone who does research in such an area. That's what it sounds like to me, at least, because I once argued similarly while I was working from a more mathematical side about the value of understanding things rigorously. And I would issue statements like "well, sure if you just want to put things into PyTorch, _sure_, but that's not _real_ math."

Unfortunately for us (or at least me, because why did I spend so much time learning this stuff?), many top ML or AI researchers have only a vague understanding of fundamentals, and I vehemently disagree that you need rigorous exposure to mathematics to contribute to ML or AI.

My experience is quite the opposite. Lots of people in mathematics / academia are asleep at the wheel. Well, they were at the wheel fifty years ago, and some departments have caught up, but overall mathematical rigor is not what made the field go forward. (D. Donoho's "50 years of data science" seems relevant).

If you are adamant that Peano axioms in particular are necessary, why doesn't your argument that they are necessary extend to things like the axiom of choice? You know what they say: "the axiom of choice is clearly true, the well-ordering principle is clearly false, and who knows about Zorn's lemma?" Or, if the introduction to Folland's Real Analysis is to be believed, if your research project depends on the validity of the axiom of choice -- choose another research project.

All this to say your post makes me recognize a sentiment I once had: mathematical rigor is important. That depends. I realize that more often than not I made that argument to convince myself such knowledge is relevant -- and that creates a dangerous blind spot (at least for an academic who wants to be useful to industry).

>> Unfortunately for us (or at least me, because why did I spend so much time learning this stuff?), many top ML or AI researchers have only a vague understanding of fundamentals, and I vehemently disagree that you need rigorous exposure to mathematics to contribute to ML or AI.

But that depends on what you mean by "contribute". Machine learning research has turned into a circus with clowns and trained donkeys and the majority of the "contributions" suffer heavily from what John McCarthy called the "look ma, no hands disease" of AI:

Much work in AI has the ``look ma, no hands'' disease. Someone programs a computer to do something no computer has done before and writes a paper pointing out that the computer did it. The paper is not directed to the identification and study of intellectual mechanisms and often contains no coherent account of how the program works at all[1].

Yes, anyone can contribute to that kind of thing without any understanding of what they're doing. Which is exactly what's happening. You say that "many top ML or AI researchers have only a vague understanding of fundamentals" matter-of-factly but while it certainly is fact, it's a fact that should ring every single alarm bell.

The progress we saw in deep learning at the start of the decade certainly didn't come from researchers with "only a vague understanding of fundamentals"! People like Hinton, LeCun, Schmidhuber and Bengio have a deep background not only in computer science and AI but in other disciplines also (Hinton was trained in psychology, if memory serves). Why should we expect any progress to come from people with a "vague understanding" of the fundamentals of their very field of knowledge? In what historical circumstance was knowlege enriched by ignorance?

______________

[1] http://www-formal.stanford.edu/jmc/reviews/lighthill/lighthi...

Learning Peano arithmetic from examples, for instance learning to count over the natural numbers, is a classic machine learning task. Modern machine learning systems can only solve this task approximately, that is they can learn to count over a certain interval but not, say, to arbitary values. Same goes for other arithmetic. This, despite the progress in apparently more complex tasks such as classifying objects in images etc. Of course, if you don't know what Peano arithmetic is, you can't imagine why it may be a useful test of learning ability.

But I think your question is more along the lines of "why do I need Peano arithmetic to build an image classifier?" (or any machine learning application). To be clear, you don't. But without knowledge of some fundamental concepts in mathematics and the theory of computation, you won't understand the power and limitations of any system you may be capable of buidling. As a lay person, that's by the by. As a machine learning researcher, that'd be disastrous.

Unfortunately, as the OP points out, such lack of deep background in AI research is common today, and probably responsible for the absolute quagmire in which machine learning research finds itself.

Why exactly would you study math from 65 years ago that isn't relevant today as determined by the community itself?

> math from 65 years ago

Calculus and linear algebra are a tad older than that.

> isn't relevant today as determined by the community itself?

Which community? NeurIPS/ICML/ICLR/et al.? That's only a subset of the AI community.

Also, I'm not really sure that "pure ML/DL" is even a particularly in-demand field of research expertise right now; we've been absolutely saturated with supply for at least junior phd students in that subfield for close to an academic generation by now... whereas there is now a genuine drought of folks who speak "both languages" in AI. I think these days you really need a second field in order to not be in a horrendously bleak over-saturated labor market. E.g., systems and ML? God, please yes. "Symbolic AI" tradition and ML/DL? Yup, lots going on there right now. But pure NeurIPS et al. style work? You've gotta the best in a very, very, very, very crowded room.

That's right. Starting today as a PhD student in deep learning is career suicide, even if it may look like this is the thing to do. The number of papers put on arxiv each month must number in the thousands and the top machine learning conferences are so awfully crowded it's impossible to get a paper through.

From my point of view and much like you say, the interesting, groundbreaking work has moved outside strict deep learning research. I mean, I sure would think so, but here's the website of the International Joint Conference on Learning and Reasoning, that brings together a bunch of disparate neurosymbolic and symbolic machine learning communities for the first time:

http://lr2020.iit.demokritos.gr/

This is an active field of research with plenty of space for new entrants and full of intersting problems to solve and virgin territory to be the first to explore. I'm hoping we'll soon see an influx of eager and knowledgeable new graduates disappointed with the state of machine learning research and willing to do the real hard work that needs to be done for progress to begin again.

>Starting today as a PhD student in deep learning is career suicide

lol maybe if you want to go into academia but, speaking from experience, FAANG is paying handsomely for this skillset.

It's the same thing in FAANGs. The competition for the over-inflated salaries they offer is such that it's a matter of chance whether someone gets hired or not. You might as well invest your money at the blackjack table in your local casino.

Edit: as a personal recommendation, try not to start your comments with "lol". It makes the comment sound shallow and detracts from your point.

Having gone through the process and coaching a friend though it, it's as much of a crapshoot as studying for any standardized test (which is to say not at all). The things on the test are standard grad school ML stuff with a dash of systems engineering. You know what's not on this test though: peano axioms or set theory lol.

No, of course, but who said they were?

Also, I'm getting the feeling all the lols are meant to underline something clever and pointed, but I fail to see it. Can you clarify?

The problem with these online resources are that you end up bookmarking them, but you are most likely never going to come back read them. Apologies, if I am generalising.

Learning something is not only reading the course content, that is one part of it (of course). But developing a robust and simple system where you can CRUD your mental model. If you got your note-taking thing figured out, technically you can learn and should be able to learn large amounts of material with sustained and relatively lesser effort. Once again, apologies for going on a tangent.

If anyone is looking to teach themselves CS along with CS Math, then we[1] are creating self-paced computer science courses. Our course content will be available as free online e-books [2] as well as their corresponding (paid) interactive versions and will be started getting released mid-November. We have two free courses as of now.

Although it doesn't start from scratch and we assume that you have got atleast highschool mathematics part covered.

[1]: https://primerlabs.io [2]: https://primerlabs.io/books

I actually find i'm the opposite (everyone is different of course - to each their own). I can learn from written resources fairly well once i get in the flow.

Interactive content or content where i have to do exercises a lot break my flow, because i am constantly switching gears.

Same here. For me the best flow comes from a simple information dense book, like any famous tome from Springer Graduate Texts in Mathematics.

In relation to this, IMHO the best way to learn the foundations of mathematics is to teach yourself either:

* Linear algebra and real analysis (if your end goal is ML or statistics)

* Logic and type theory (if your end goal is formal methods or PLT)

For the first option, a typical sequence (like the one taught at Harvard Math 55) is Halmos (or Axler) plus Rudin (and a good companion like Gelbaum & Olmsted). A more practical bottom-up approach is to use a good matrix analysis book such as Meyer, plus a more practical calculus (vs analysis) textbook like Hubbard & Hubbard.

For the second option, a good theoretical logic book (Ben Ari and Schoening are my favorites) paired along with Girard. A more practical option would use Huth & Ryan as a logic volume along with Pierce's TAPL.

Could you provide the full titles/authors of these books?

Full titles, authors and ISBNs should be available via Goodreads:

* Halmos: https://www.goodreads.com/book/show/692030

* Axler: https://www.goodreads.com/book/show/309768

* Rudin: https://www.goodreads.com/book/show/292079

* Gelbaum & Olmsted: https://www.goodreads.com/book/show/818070

* Meyer: https://www.goodreads.com/book/show/1002449

* Hubbard (5th edition available): https://www.goodreads.com/book/show/631871

* Ben Ari: https://www.goodreads.com/book/show/115121

* Schoening: https://www.goodreads.com/book/show/4993214

* Girard: https://www.goodreads.com/book/show/3400118

* Huth & Ryan: https://www.goodreads.com/book/show/151847

* Pierce: https://www.goodreads.com/book/show/112252

Amazing. Thank you.

I have been gravitating towards this. Feel like I only got very shallow learning from my CS courses. Bought physical books now to grok the subject better (for now theory of computation, algorithms etc).

Feel like especially my math foundation is lacking, so thanks a lot for these recommendations!

What's the onboarding page[1] using to determine the user's first name? It knew my first name the first time I opened it.

[1] https://app.primerlabs.io/onboarding

Google Sign In i'd wager

There is another approach[0] centered on a few key textbooks. The goal is to prepare you for fully grokking Deep Learning[1] and Elements of Statistical Learning.[2]

[0] https://www.dropbox.com/s/mffzmuo9fvs5j6m/Study_Guide.pdf

[1] https://www.deeplearningbook.org/

[2] https://web.stanford.edu/~hastie/ElemStatLearn/

Goodfellow's is not good. Hodgepodge of material with scattered theorems and way too much attention paid to GANs (surprise surprise).

ESL is very good but at this point it's the TAOCP of ML book recommendations (besides have steep prereqs).

This is a way more approachable and yet still through book on DL.

Disagree on Goodfellow, personally I found it the most approachable introduction to neural networks & backprop. To be fair, I think this must have been the pre-GAN edition so maybe it's changed a bit since then.

ESL is a fantastic reference but I wouldn't recommend it as a textbook, it's just too terse.

> ESL is a fantastic reference but I wouldn't recommend it as a textbook, it's just too terse.

I dunno, it's really well written (at least chapters 1-9 are, because they were re-written) and the graphs are to die for.

It definitely has a bunch of theorems and proofs that may not be necessary, but the overall flow isn't lost by skipping the ones above your mathematical maturity level.

To be fair though, I also love TAOCP, and have worked through 1.5 chapters at this point.

The title of this sounded very promising. Something that actually describes math from scratch needs to exist, but this ain't it. It runs off the rails from the very first sentence, where it asks "What is math?" but doesn't actually answer the question except to tell you that it's easy.

I stopped reading when I got to "Numbers are objects you build to act like the concept of a number."

No no no no no. No! Math is the act of inventing sets of rules for manipulating symbols that produce interesting or useful results, or exploring the behavior of sets of rules invented by others. One of the earliest example of such interesting or useful results is rules for manipulating symbols so that the results correspond to the behavior of physical objects (like sheep, or baskets of grain, or plots of land), and in particular, to their quantity so that by manipulating the right symbols according to the right rules allowed you to make reliable predictions about the behavior of and interactions between quantities of sheep and grain and land. The symbols that are manipulated according to these rules are called "numerals" and the quantities that they correspond to are called "numbers".

But you can invent other rules for other symbols that produce other useful and interesting behavior, like "sets" and "vectors" and "manifolds" and "fields" and "elliptic curves." But it all boils down to inventing sets of rules for manipulating symbols.

> One of the earliest example of such interesting or useful results is rules for manipulating symbols so that the results correspond to the behavior of physical objects (like sheep, or baskets of grain, or plots of land)...

Or was it the other way round? First came the objects and then people saw patterns in the transactions and then abstracted it away with symbols?

One of the reasons children find maths difficult is because we tend to describe maths in this reverse order i.e. from symbols to physical objects(the way you described above). But explaining it in the actual order i.e. from physical objects to symbols will make maths accessible to many more kids.

The evolution of math came by way of things like tally marks on sticks and pebbles in pots. The line between symbol and physical system is fuzzy.

I think one of the reasons that people find math difficult is that the emphasis is on the symbol-manipulation rules and not on the reasons that certain sets of symbol-manipulation rules are useful. Kids are taught to count "1 2 3 4..." as if these symbols were handed down by God rather than being totally arbitrary. They should be taught to count "*, **, ***, ****, ..." and then, when they get to "***********" or so, they should be encouraged to invent shorter ways of writing these unwieldly strings.

(Note, BTW, that when you use the "*, **, ***, ****, ..." numeral system, addition and multiplication become a whole lot easier!)

I don't get your objection

"Numbers are objects you build to act like the concept of a number."

So, numbers (i read that as a colloquialism for "numerals") are symbols that represent the abstract idea of a "number".

You can't have (non-trivial) rules for manipulating symbols without symbols, so i don't understand your objection.

The definition as stated is circular. It has the same information content as "A foo is an object that you build to act like the concept of a foo."

Also, the symbols that stand for numbers have a name. They are called numerals.

So: a number is an abstraction of a quantity, which is a number plus a unit. "Seven sheep" is a quantity of sheep. "Seven bushels of grain" is a quantity of grain. "Seven acres of land" is a quantity of land. "Seven" is the abstract property that these things have in common, and the symbol "7" is the numeral that denotes this property.

I don't think its circular, just really informal. The two usages of the word "number" are refering to two different meanings of the word.

If your original objection was a lack of formality, i'd say its a matter of taste, audience and author intent, but ultimately a fair objection. But this seems very far afield from your original complaint.

That "definition" rubbed me the wrong way, too. A bit recursive...

I am quite happy with that "definition", not least because it's obviously not intended to be formal. But anyway, a very standard thing for a mathematics lecturer to say is "This is the only thing it could be", or "you know what this concept is already; we are just making some formal symbols to capture it", or words to that effect. It's pedagogically important to distinguish between genuinely new things that the reader is going to have to put some effort into rearchitecting their worldview around, versus things the reader already knows but might not have seen in this form before. Since everybody in this target audience already knows how to count, the treatment of the natural numbers is surely going to be in the latter category.

It's worse than non-formal, it's circular, and hence vacuous. It has the same information content as "A foo is an object that you build to act like the concept of a foo."

No! But I am not going to re-explain it; I have already said precisely why the information content is different, in the comment to which you replied. I shall merely quote it again.

> It's pedagogically important to distinguish between genuinely new things that the reader is going to have to put some effort into rearchitecting their worldview around, versus things the reader already knows but might not have seen in this form before.

Since it is common knowledge that nobody already knows what a "foo" is, but it is common knowledge that everyone already has a pretty uniform idea of what a "number" means in everyday life (really, how many ways are there you can count from 1 to 10), my assertion applies.

> everyone already has a pretty uniform idea of what a "number" means in everyday life

Well, that's true, but that common notion is wrong. So if that is what you are appealing to then you have different problems. For example, on the common notion of number, there is no number whose square is negative one. (And indeed mathematicians got hung up on that for centuries before they figure out that they were wrong!)

Oh come now, it's not wrong. I shall quote the article.

> # The natural numbers

> Numbers are objects you build to act like the concept of a number. Here we begin to construct the natural number object.

This is obviously referring to the natural numbers. It says so both before and after the statement you're talking about. There is no natural number whose square is negative one.

Let's not lose the plot here. I was critiquing a piece called "Math Foundations from Scratch". It's a little hard to tell whether it was intended to be pedagogical, targeted toward people who didn't already understand the foundations of math, or a sort of review or reference, targeted at those who did. But either way I think it fails, notwithstanding the fact that the specific flaw that I cited, a circular definition of the word "number", appeared in the context of a discussion of natural numbers.

That's my opinion. I've explained at some length why I hold this opinion, but it's still just my opinion.

At best, it's a badly-written sentence. If he meant what you mean, he could have said "Everyone has an intuitive understanding of what 'numbers' are. In mathematics, we make a precise definition for the same concept." And that would have been fine.

But with the sentence the author actually wrote, you're left wondering whether that's what he actually meant. Does the first "number" mean the same thing as the second "number" (in which case it's circular), or are they subtly different (in which case, is the reader sure that they understand the distinction)?

And that reveals the bigger problem. Mathematics has been described as the study of things that have precise definitions. If the writer is this fuzzy in the introduction, can I trust him to be correctly precise?

Based on my reading you dont seem to fundamentally disagree with the author. I think hes just playing with the ambiguity of the word number for some effect.

So: (Some mathematical formalism for) numbers are objects you build to act like the (informal) concept of a number.

No! There is nothing informal about the mathematical concept of a number. And numbers can be defined without resorting to circular definitions that include the word "number" in the definition.

It just so happens that the symbols that denote numbers behave like certain things in the real world when manipulated according to the rules that are customarily associated with the symbols. Those things in the real world have properties that correspond to the informal idea of numbers. But that has absolutely nothing to do with the math (except insofar as it's one of the things that makes math worth doing).

"Number" is an informal concept in mathematics. The real numbers are a specific concept with multiple formalizations.

A model of a number system is an assembly of behaviors, rules, or objects which behave like the numbers you want. I don't see the circularity here, and given that they are tackling an informal concept of number, I don't see any loss of clarity either.

> A number is an informal concept in mathematics.

No, it isn't. You're just wrong about that.

Now, it is true that there are different kinds of numbers in math, but each different kind of number is formally defined. All of those definitions are interrelated, and collectively they define what it means to be a number. There is nothing informal about it.

> All of those definitions are interrelated, and collectively they define what it means to be a number. There is nothing informal about it.

Then you could simply point to the formalism which unites various mathematical activity under some universal model of number. I will reiterate the claim again — "number" is an informal concept among mathematicians and they are fine with that.

I disagree, and I believe you have made a category error: you are mistaking the description of a thing for the thing itself. I believe your error is the same as that of the one who asserts "the natural numbers are the finite transitive sets". In fact, the finite ordinals are only one possible implementation of the natural numbers in set theory, and what the natural numbers really are is "the abstract notion of the smallest structure you can do induction on". Similarly, any given symbolic representation of a thing is an implementation of that thing in mathematics, but it need not be the thing itself.

Perhaps our best (or even only) hope of automatic mathematics verification is to find a symbolic representation of a thing, and there are some areas where we have in some sense "definitely found the right symbols" which cleave very tightly to the thing they represent; and it's probably always true that if you can phrase something in the symbolic language, then you will be more rigorously constrained to say correct things by construction. But when an actual human actually does mathematics, they may very well not be manipulating symbols at all until they come to the point where they must transmit their thoughts to someone else. In fact, in some cases it may even be possible to transmit nontrivial mathematical thoughts without symbols; one possible method is simply "analogy".

To revisit the example at the start, by the way: If you choose the language of category theory, of course, you can formalise "the abstract notion of the smallest structure you can do induction on" into the symbolic representation whose name is "natural numbers object", and thereby claim that once more we are in the world of rules for manipulating symbols. But I do not believe the implied claim that "unless you think of the naturals as their category-theoretic representation, then you are not doing mathematics"!

> I disagree

With what?

> you are mistaking the description of a thing for the thing itself

How?

> what the natural numbers really are is "the abstract notion of the smallest structure you can do induction on".

And who granted you the authority to define what the natural numbers "really are"? The last time I checked, numbers were not actually part of physical reality, they were a kind of concept or idea. People are free to attach whatever labels they like to concepts and ideas. They can even overload terms to mean different things in different contexts. So just because I use the word "number" in some colloquial context to stand for a different idea than you do in some other context doesn't make it wrong.

Ah, it is fun to be belligerent on the Internet. But I shall answer in good faith.

I disagree with… well, what could I possibly be disagreeing with except the primary thesis of the comment to which I was replying, namely that mathematics is solely symbolic reasoning. More precisely, "Math is the act of inventing sets of rules for manipulating symbols that produce interesting or useful results, or exploring the behavior of sets of rules invented by others.".

I believe I have demonstrated both with analogies and specific examples what descriptions you were mistaking for the things themselves. I believe you are blinded by the fact that we communicate about maths using well-defined formalisms (drawings of graphs, logical formulae, definitions, theorems), and are missing the fact that we don't necessarily think in those formalisms; and I assert that thinking about the objects referred to by those formalisms is still performing mathematics.

In fact I believe it is you who is claiming the authority over what the natural numbers "really are", because you assert that one cannot think about the natural numbers without adhering to formalisms! I gave both an informal notion and one possible casting of that informality into a formal system. But my point stands precisely as stated if you simply change "what they really are" to "a more general way of thinking of them, outside the ordinals formalism"; you have certainly done an excellent job of quibbling away the actual points. I did assert clumsily that I had the one true way of thinking about the natural numbers, and I shouldn't have done so; my argument certainly doesn't rely on that assertion.

> I assert that thinking about the objects referred to by those formalisms is still performing mathematics.

OK. So can you give me an example of thinking about something such that that act of thinking isn't "performing mathematics"?

Certainly! I am not performing mathematics when I am reading a fiction book and (unconsciously or consciously) mentally simulating the actions and internal states of the characters. I never said anything like "all thought is mathematics", merely "not all mathematics is symbolic thought".

> I am not performing mathematics when I am reading a fiction book

How do you know? You said that thinking about the objects referred to by mathematical formalisms is performing mathematics. How can you be sure that your fiction book can't be described by some mathematical formalism, but you are simply as yet unaware of it?

Maybe you'll come along in ten years and convince me that I was performing mathematics all along! Maths has expanded to embrace other fields in the past. Archimedes probably would have called work on the nature of truth "philosophy" and very much not "mathematics". I, for one, do not wish to adopt any stance that rules out future progress.

I don't have to wait ten years. Unless you reject the Church-Turing thesis, then on your definition everything is math, which makes your definition vacuous.

I shall quote myself:

> I never said anything like "all thought is mathematics", merely "not all mathematics is symbolic thought".

We're going around in circles. If not all mathematics is symbolic thought, then, unless you're a dualist, there is nothing that can be meaningfully distinguished as "not mathematics" (because Turing completeness) and the whole concept of mathematics becomes vacuous and useless.

Well, you're extrapolating to the claim that since humans can be emulated by machines, which are in the domain of mathematics, every aspect of human thought is mathematics. I deny this extrapolation. Merely writing something down does not make it literature; merely encoding something into a Turing machine does not make it mathematics.

Ah, I see you're a mathematical formalist. That's one philosophy of what math is, but it's not the only one.

What's going on in my mind when I visualize 3-dimensional manifolds and manipulate them? Cutting pieces apart, reattaching them elsewhere, stereographically reprojecting objects embedded in a 3-sphere. I don't see where the symbols are exactly here -- it seems like I'm (at least trying to) manipulate "actual" mental objects, ones my visual cortex are able to help me get glimpses of. Everything I'm doing can be turned into symbolic reasoning, but it's a fairly painful process. It seems closer to the basis for the philosophy of intuitionism, that math derives from mental constructions.

I think it's rather interesting that many creatures (including ourselves) have a natural ability to at a glance tell how many things there are, up to about five objects or so. Maybe it's useful evolutionarily for counting young, or for detecting when a berry has gone missing, etc. I think it's also rather interesting that many creatures have episodic memory. Sometimes it seems to me that the invention of symbolic number derives from these two capabilities: through our experience that things can happen in sequence, we extend our natural ability to count while believing these "numbers" are meaningful. Each number is a story that plays out for what's been accumulated.

The boundary where abstract noodling around with mental images becomes math lies precisely at the point where you can render those images symbolically. That is the thing that distinguishes math from all other forms of human mental endeavor.

So spending say 5 years, noodling around with abstract mental concepts and ending with symbols that turn out to be manipulatable and useful in a mathematical way means, you haven’t done math until year 6 starts?

I’m trying to see the line as clearly drawn as your comment suggests. The exact boundary doesn’t seem as simple as, stops here / starts here.

> So spending say 5 years, noodling around with abstract mental concepts and ending with symbols that turn out to be manipulatable and useful in a mathematical way means, you haven’t done math until year 6 starts?

Yes, that's right. Until you write down the symbols you quite literally haven't done the math.

What a weird definition. Which fields of human endeavour cannot be rendered symbolically?

If you are taking a very strict view of what it means to symbolize something (which i imagine you are if you think other fields cant be), then i wonder: do you consider geometric proofs math? Is euclid's elements with its prose proofs, math?

> Which fields of human endeavour cannot be rendered symbolically?

All of them other than math.

> do you consider geometric proofs math?

Yes, because diagrams are symbols. They happen to be rendered in 2-D but they are symbols nonetheless.

What makes something a symbol is that it is subject to a convention that allows it to fall into one of a finite number of equivalence classes, so that small changes in its physical details don't change its function within the context of the symbol-manipulation activity. Geometric diagrams have this property, so they qualify as symbols.

I don't think i agree with that definition of symbol. I think that's closer to the definition of an abstraction, which i would consider to be different. Regardless, for the sake of argument lets accept it.

Words also have this property. The word "Dog" refers to an equivalence class of a bunch of different physical manifestations which are considered equivalent in the context the word is used.

Lets take a first aid textbook. This usually consists of a bunch of words describing situations, and a bunch of 2-D diagrams. This meets your definition of symbolic as far as i can tell. Is first-aid a sub-discipline of mathematics in your view?

> I don't think i agree with that definition of symbol.

Really? How would you define it?

> The word "Dog" refers to an equivalence class

Yes, and if you wanted to, you could use dogs (actual physical dogs, not the word "dog") as symbols. It would be kind of a silly thing to do, but you could. (For the sake of completeness, because I'm not 100% sure this is what you were referring to, the word "dog" is unambiguously a symbol, one which denotes the equivalence calls of actual physical dogs. The equivalence class of the word dog includes things like "Dog" and "DOG" and "perro" and "hund".)

> Is first-aid a sub-discipline of mathematics in your view?

No, because the business of first aid is not the business of manipulating those symbols, it is the business of manipulating actual physical bodies. First-aid isn't math for the same reason that (say) building computers isn't math. This is not to say that math can't be a useful part of either of those activities, but they aren't equivalent.

> Really? How would you define it?

Well first off, i'm not sure why you're requiring finiteness or what that means in context.

But more to the point, i would define it as something that represents (denotes) something else. The thing it represents may be concrete, it might be abstract (represent a group of all things that some property), it might not exist at all, etc.

I suppose i'm actually a bit confused by your definition, i find it hard to parse. "What makes something a symbol is that it is subject to a convention that allows it to fall into one of a finite number of equivalence classes,": it sounds like what you're saying is that a symbol is defined as being a representative member of one or more equivalence classes, which seems obviously wrong, as symbols typically aren't members of the class of objects they represent. I was previously interpreting your statement as missing the word "denote". But denote is kind of the core of the definition of a symbol. If what you meant by your definition was that a symbol is something that denotes an equivalence class, you could probably drop the equivalence class bit, as all concepts can be thought of as equivalence classes with zero or more members, so it doesn't add anything to the definition.

> No, because the business of first aid is not the business of manipulating those symbols, it is the business of manipulating actual physical bodies.

Sure (although what about the activity of writing first-aid textbooks?), but that's a pretty dramatic shift in goal posts from "precisely at the point where you can render those images symbolically". Being able to "Render" something symbolically is very different from having the end goal of the activity being the manipulation of the symbols.

I agree having symbol manipulation be the chief end, feels closer to a definition of math, but it still feels kind of lacking. Mathematicians don't just arbitrarily manipulate symbols, they choose what to do based on a variety of criteria, such as real world applications, beauty, connectedness to other areas, etc. The ultimate end for actual mathematicians isn't solely symbol manipulation.

On the other side, poets manipulate symbols (words) following certain rules. They do so for a variety of reasons, but usually to evoke emotion or perhaps beauty in the minds of their readers. The final goal is a series of symbols manipulated just the right way. Are they mathematicians?

> symbols typically aren't members of the class of objects they represent

That's right. The essential characteristic of a symbol is that it retains its essential character through certain kinds of changes. It's hard to illustrate via ascii text because all of the instances of a given symbol render in exactly the same way, but if I could hand-draw two letter A's I could make them look slightly different but they would still be indentifiable as A's. In some contexts, an upper and lower case character can be the "same symbol", so for example, "DOG" and "dog" are the "same word" (which is to say, the same symbol, because words are symbols).

The reason the number of equivalence classes has to be finite is because the things doing the manipulating are physical entities (like humans or computers) they are only capable of dealing with a finite number of different kinds of things.

> what about the activity of writing first-aid textbooks?

You posed that as a parenthetical but it's a fair question. Let me reflect it back at you: imagine that AI technology has advanced to the point where it is capable of producing original first-aid textbooks. Is that math?

> Mathematicians don't just arbitrarily manipulate symbols, they choose what to do based on a variety of criteria, such as real world applications, beauty, connectedness to other areas, etc. The ultimate end for actual mathematicians isn't solely symbol manipulation.

Of course. That's why part of my definition was that there had to be interesting or useful results.

> poets manipulate symbols (words) following certain rules

Really? What are those rules?

> Really? What are those rules?

This is too fun so I can't help myself. A stark example here is of haikus, which generally have few requirements by way of grammar, and only require syllabic rules.

To parallel propositional logic, we can define an English haiku as a logic through the following:

The set of symbols are the words of the English language, with punctuation marks and newline symbols.

The language is defined as the free language generated by the symbols.

Transformation (inference) rules consist of the following:

If we have no sentences we may add (infer) an axiom.

If the first sentence consists of five syllables, we may infer a new sentence of free words consisting of seven syllables.

If the second sentence consists of seven syllables, we may infer a new sentence of free words consisting of five syllables.

If the third sentence consists of five syllables, we may infer top.

Otherwise if none of these rules are productive, we may infer bottom (we never infer bottom).

Our axiom set consists of all five syllable sets of English words.

That is the formal system of haikus.

Now you can argue that this system doesn't have interesting or useful results, but it's a bit difficult to argue that purely syntactically. You probably need some notion of semantics, and it becomes quite difficult to explain semantics purely using symbols and rules. In particular it can get a bit thorny to describe what a "result" is purely syntactically, i.e. only as the consequence of symbols and manipulation rules.

> haikus

I must confess I had to think about this one for a while. Yes, haikus adhere to strict rules (as do sonnets). But here is something that adheres to the rules of haiku:

   foo foo foo foo foo
   foo foo foo foo foo foo foo
   foo foo foo foo foo

which I think you would agree is not a haiku, or at least not a very good one (unless you're a Dadaist). There is something essential to a haiku, and to a poem in general, that is not captured by the rules.

But there are situations where the boundary between math and poetry can be very fuzzy indeed.

I think i would say the same about math. You can manipulate logical statements in an infinite number of ways, but only a subset are interesting. What makes an interesting/good poem is just as subjective as what makes an interesting/good theorem.

That's not quite true. Some math is objectively useful in ways that poems are not because the math corresponds to -- and hence allows you to accurately predict -- the behavior of physical systems.

> That's right. The essential characteristic of a symbol is that it retains its essential character through certain kinds of changes. It's hard to illustrate via ascii text because all of the instances of a given symbol render in exactly the same way, but if I could hand-draw two letter A's I could make them look slightly different but they would still be indentifiable as A's. In some contexts, an upper and lower case character can be the "same symbol", so for example, "DOG" and "dog" are the "same word" (which is to say, the same symbol, because words are symbols).

Ok, I see where you're coming from with that. I think I would conceptualize this differently:

¼+¼=½

and

1/4+1/4=1/2

Are both two separate "symbols". These symbols both however denote another symbol, the abstract idea of the equation. This abstract idea is what mathematicians are manipulating. The scratches on a piece of paper are just a symbol that represents the symbol being manipulated. This abstract symbol in turn is a symbol which represents different concrete realizations (e.g. it might represent drinking a quarter cup of water, and then another one, for a total of half a cup. Or whatever).

To summarize, People add "numbers" together. They don't add pieces of writing together, those just represent what they add together. Each numeral written on a piece of paper is unique, but they represent the same number.

I suppose that view might be more philosophical, where its not definitively right or wrong but more a matter of taste.

On this view, you could certainly create an equivalence class of all the symbols that denote the same thing, but its hardly necessary and not core to the definition.

In my view, symbols don't have to be physically realizable, so finiteness is not a requirement.

> You posed that as a parenthetical but it's a fair question. Let me reflect it back at you: imagine that AI technology has advanced to the point where it is capable of producing original first-aid textbooks. Is that math?

I have a different definition of math than you. While i think i disagree with your definition, its not exactly obvious to me what the correct one would be. I think I would go with something along the lines of: Math is the process of discovering new truths and consequences of abstract formal systems using logical deduction. Good math, is math that discovers truths which are interesting from a practical or aesthetic standpoint.

On my definition, I would say no. While writing a book does involve the manipulation of abstract symbols as an end (Which i think makes it math in your definition), it doesn't involve discovering any consequences of a formal system. I don't think whether an AI or a human writes it makes much difference.

> Really? What are those rules?

Sibling post discussed haikus which are an obvious example. Other examples involve meter or rhyming scheme. If nothing else, there's rules about which sets of characters make words in english and what not.

The key thing I'm trying to get at is that poetry involves putting symbols in a specific order following certain rules to get some affect. That seems to match your definition of math afaict. I don't think it is math, because it doesn't capture any truths about abstract systems (Maybe you could argue it captures emotional essence of what it is to be human in a similar way as how math captures the essence of what it is to be a triangle and what not, but that's kind of a tangent).

No, you have this exactly backwards. ¼ and 1/4 are the same symbol, and so is this:

      @
      @
      @
      @
    ____
       @
      @@
     @ @
    @@@@
       @
       @

The whole point of symbols is that they retain their essential properties through a variety of changes in their physical appearance.

Why?

I feel like you're just asserting disagreement without argument here. I think this is primarily a philosophical dispute on the nature of symbols, where both are just different foundations leading to the same concept. But if my take is backwards or wrong, then it should lead to some disconnect between it and what people mean by symbol in practise. So what is it?

> The whole point of symbols is that they retain their essential properties through a variety of changes in their physical appearance.

Whether they keep essential properties through multiple physical forms or whether many different unique symbols all point to the same (non physically realizable) "form". Doesn't seem materially different to me, just different foundations for the same idea.

> Doesn't seem materially different to me

It isn't. It's the same idea expressed with different words.

But what matters about symbols, the thing that makes them useful, is that there is a mapping from a potentially infinite number of physical things to a finite number of Platonic ideals or whatever you want to call them. Whether you want to call "4" and "four" the "same symbol" or "different symbols denoting the same concept" doesn't matter. What matters, the source of the utility, is that there is a finite number of equivalence classes that the physical things can be unambiguously mapped to.

> It isn't. It's the same idea expressed with different words.

But one is "backwards" and one is not?

> is that there is a finite number of equivalence classes that the physical things can be unambiguously mapped to.

Unambigious is quite a claim.

Is 3.0 and 3 the same symbol on your view? Depending on context they might denote the same idea (the number three). Other times maybe 3 is an integer and 3.0 is a float and their types are important.

Sometimes my computer spits out 10 to mean the sixtenth natural number, sometimes the second natural number, sometimes the tenth natural number. Its all the same physical thing, however what other physical things its equivalent to depends on context.

> Unambigious is quite a claim.

No, it's not. The symbol systems used for math are deliberately engineered to make this true.

> Is 3.0 and 3 the same symbol on your view? Depending on context

Exactly. It depends on the context. Sometimes "A" and "a" ArE tHe SaMe SyMbOl, SoMeTiMeS tHeY aReN't. (Sometimes "Y" and "TH" are the same symbol! [1]) Likewise, sometimes "3.0" and "3" are the same symbol, sometimes they aren't. But within a given context, in order to qualify as math, the mapping has to be unambiguous.

[1] https://www.youtube.com/watch?v=PVVTk7yy4kU

This is either quite the claim or you have a non-standard definition of "symbol." Each of those three representations are obviously different, though they all refer to the concept of "one-quarter," and I very much question the one made from @ signs. (And to be annoying, ¼ and 1/4 both have completely different UTF-8 representations, but I'd still associate them with the concept of "one-quarter.")

It seems like you're conflating "signifier" with "signified". I'd say the whole point of symbols is that they play a role in semiosis, where an interpreter (us) receives meaning (the "signified") from decoding a symbol (the "signifier"). My own decoding apparatus gets the same roughly the same meaning from ¼ and 1/4. But the meaning's not a symbol in itself.

What I was obliquely getting at earlier in bringing up intuitionism is that math seems to derive from pre-existing mental processes. Recognition and manipulation of symbols seems to be no more special than other mental processes. Rule-based manipulation seems to appeal to our understanding of causality, for example.

> I very much question the one made from @ signs

There are limits to what can be expressed in an HN comment. If we were standing at a whiteboard I could draw many more different variations on the theme than I can here.

> you have a non-standard definition of "symbol."

A symbol is a physical thing that can be unambiguously mapped into one of a finite set of equivalence classes. ¼ and 1/4 and even "one-quarter" can all be unambiguously mapped (within a particular context, i.e. people speaking English talking about math) onto a single concept which is denoted ¼ or 1/4 or "one-quarter" or "one-quarter".

> It seems like you're conflating "signifier" with "signified".

No, I'm not conflating them. It's just very hard to talk about these things because the only tool we have at our disposal is signifiers. I can't actually show you the thing that is signified by ¼ and 1/4 and "one-quarter", I can only talk about its properties (and I can only do that using signifiers).

> math seems to derive from pre-existing mental processes

That is a very human-centric view of things. It happens to arise in humans from pre-existing mental processes, but that doesn't mean that math necessarily or inherently arises that way. In fact, math arises in nature in myriad ways, many of which are not even mental processes at all.

> Recognition and manipulation of symbols seems to be no more special than other mental processes.

Oh, that is where you are very, very wrong, my friend. Manipulation of symbols is very special. There's a reason it's called the "computer revolution".

> No, I'm not conflating them.

Earlier you said "¼ and 1/4 are the same symbol" but now you're saying that symbols are physical things that unambiguously map onto a single equivalence class called a "concept." If they're the same, you're conflating signifier and signified. If they're not the same, then I'll leave you to your workshopping of your own philosophies of math and semiotics. (It would be more interesting if you'd relate what you're saying to established philosophies. I've mentioned a few, and it's a bit frustrating that they were overlooked.)

Formalism is a legitimate philosophy of math, but I think it's rather absurd to declare that it serves as a definition of math.

I like how Simon Stevin translated mathematics (Latin) into Dutch, wiskunde "the knowledge/art of what is certain." Symbolic calculation is certainly a way to be certain of something.

I think we're dealing the limitations of communication via unicode text. When I say that ¼ and 1/4 are "the same symbol" what I mean is that they are the same symbol with respect to their visual appearance, not with respect to their numerical representations are unicode code points (or, in the case of 1/4, three unicode code points).

And I shouldn't have used the word "concept". That just muddied the waters.

Imagine if we were standing in front of a white board and I wrote 1/4 twice. The physical appearance of those two batches of dry-erase marker ink would be slightly different. They would have slightly different shapes. They would be in different physical locations on the board. They might even have different colors. They might even be individually multicolored. None of those things would matter. Both of them would be the "same symbol" because they would both be members of a single equivalence class (with respect to the conventions of mathematical notation).

> Formalism is a legitimate philosophy of math, but I think it's rather absurd to declare that it serves as a definition of math.

How would you define it then?

I see, so as a low-dimensional topologist I'm not a mathematician, thanks!*

But more seriously, if you haven't read it already you might take a look at Thurston's "On Proof and Progress in Mathematics." He had a stunningly accurate intuition, and putting what was in his mind into a form that other experts could understand and consider to be a proof gave many mathematicians a job for quite a long time. So long as you had him around, you could query him for any amount of detail for why his claims were true, so in that sense he had real proofs.

Symbolic proofs are sort of a lowest-common-denominator, serializing what's in one's mind to paper and reducing things to machine-checkable rules. It's also a rather modern idea that this is what math is.

I think it's very much worth considering math to be the study of what can be made perfectly obvious, which is closer to what Thurston seemed to be doing vs formalism.

* This is sort of a joke. At least I'm able to convince people it's math because in principle it can be written symbolically, and formalism is rather compelling as a foundational philosophy, if a bit nihilistic.

> as a low-dimensional topologist I'm not a mathematician

Huh? How do you figure?

> Symbolic proofs are sort of a lowest-common-denominator

You are reading the word "symbol" too narrowly here. What matters here is not so much the symbols but the existence of rules for manipulating them. Letters are symbols, and so are words, and so proofs written in natural language can be math if those proofs respect a set of precisely definable rules. I don't know much about low-dimensional topology, but I'd be surprised if it didn't adhere to that definition. (And if it turns out not to then I would have no qualms saying that it isn't math.)

"the quantities that they correspond to are called 'numbers'." is rather circular here no? You haven't really defined quantities or numbers except in terms of each other and yet at least one of these seems important enough to be called a mathematical concept that is distinct from a numeral.

Put another way, we clearly can distinguish many different numeral systems as referring to the "same thing." We can encode that "same thing" symbolically via a set of logical symbols to form axioms, but even there the same issue arises. Encoding the natural numbers via an embedding of second-order Peano Axioms in first-order ZFC or via a straight FOL Peano Axioms definition also seems to fall flat, as we still have not been able to capture the idea that "these are all really the same thing" since these are now different sets of symbols and rules. And we haven't even touched the thorny issue of what "encoding" and more generally "mapping" really means if everything is just symbols and rules.

But perhaps you are fine with infinite egress (which is perfectly defensible). The natural numbers have a more distressing problem when it comes to Godel's Incompleteness Theorems. There is presumably only one "true" set of natural numbers in our universe because the natural numbers have physical ramifications. That is for any symbolic, FOL statement of the natural numbers, we can create a corresponding physical machine whose observable behavior is dependent on whether that statement is true or false. And yet by Godel's incompleteness theorem we can never hope to fully capture the rules and symbols that determine these "natural numbers," and yet presumably most people would agree that these natural numbers exist and are a valid object of mathematical study. So how does that fit in? What do we call the subject that studies these objects? What do we even call these objects if not mathematical numbers?

(As an aside I'm perhaps more of a formalist than I let on, but I find the realist side a fun playground to explore in.)

> "the quantities that they correspond to are called 'numbers'." is rather circular here no?

No. "Quantity" is different from "number". A quantity is a number plus a unit. "Two" is a number, not a quantity. "Two sheep" is a quantity. This is definitional progress because I can actually show you two sheep in order to explain to you what that means, whereas I cannot show you "two".

> infinite egress

It is not an infinite regress, though explaining why is much too long for an HN comment. It also makes an interesting little puzzle to figure out where and how the recursion bottoms out, but here's a hint: how do you know what a "sheep" is?

So then of course the question arises, what is "two?" And is that a mathematical object? (Shorthand for S(S(0)) doesn't work either because S(S(0)) is clearly just another numeral system, not a number itself, indeed arguably any formalism is just a numeral system, rather than the number itself) And how does that comport with the reality of natural numbers in our universe (i.e. the incompleteness argument)?

> how do you know what a "sheep" is?

Right the usual answer is to tackle the map-territory head-on and say some of it is essentially "out of scope" but I'm curious if you had another idea.

> what is "two?"

Two is the property shared by all of the things that behave the same as the collection of things which contain all collections of things which contain all collections of things which contain nothing, with respect to a set of rules that allow you to make collections of things that contain some things and not other things, and check to see if a given collection of things contains some things or no things (and with the proviso that collections count as things that can be parts of collections of things).

:-)

> I'm curious if you had another idea.

See:

http://blog.rongarret.info/2015/03/why-some-assumptions-are-...

This needs some revision, but it contains the essential idea.

> Two is the property shared by all of the things that behave the same as the collection of things which contain all collections of things which contain all collections of things which contain nothing, with respect to a set of rules that allow you to make collections of things that contain some things and not other things, and check to see if a given collection of things contains some things or no things (and with the proviso that collections count as things that can be parts of collections of things).

That is the standard Zermelo definition. But why not the von Neumann definition (or indeed a non-set-theory definition altogther?). By your definition, and presumably, your definition of "one" I could claim that the statement "two contains one" is true. And yet by the von Neumann definition (i.e. the property shared by the collection of things which contains the collection of things that contain nothing and which contains all collections of things that contain the collection of things that contain nothing), "two contains one" is false.

And yet most mathematicians would tell you that this statement is a category error without a true or false value.

How do you distinguish between all of them? What is the truth value of "two contains one?"

All of these are hallmarks (turning category errors into sensible questions and having different answers to nonessential questions depending on the vagaries of the definition) that the definition you've provided is the definition of a numeral system, not a number itself.

And again more to the central point, how can you reconcile all of this with Godel's incompleteness theorem and the reality of natural numbers as expressed by physical machines in our universe?

> This needs some revision, but it contains the essential idea.

Right I think you're making the same appeal in your article as the one made here: https://www.lesswrong.com/posts/X3HpE8tMXz4m4w6Rz/the-simple... which is basically the map-territory definition is "out of scope" and more a question of prediction of reality.

EDIT: I must make a modification to the "two contains one" statement to instead be "any collection that has the property called 'two' contains a collection that shares the property 'one'" since we are indeed talking about the properties rather than the collections themselves.

> the statement "two contains one" is true

No. Look more carefully at my definition. On my definition, two is not a set, it is a property of a set.

Right sorry I just edited my comment (presumably just as you were making your reply).

EDIT: As an aside, I think most mathematicians at this point would be shaking their heads vigorously at calling "two" a property and would be counting that as a category error.

> any collection that has the property called 'two' contains a collection that shares the property 'one'

Yes. That is a true statement. It is (more or less) the observation that given a set S with two elements, you can make a subset of S that has one element. (Indeed you can make two such subsets :-)

Note that the reason I put it in terms of a property was specifically go get around these kinds of objections. My definition of two is, essentially, "anything that behaves effectively the same as this definition of two". I picked the ZF definition of two as "this definition" but I didn't have to. I could have picked something else, and then the phraseology would have been different. But it all eventually bottoms out in the theory of computability. You can start with ZF or Turing Machines, or the lambda calculus, or Godel numbers, or Brainfuck. It doesn't matter. All starting points lead to the same destination.

> It is (more or less) the observation that given a set S with two elements, you can make a subset of S that has one element.

But your formal definition and your explanation do not match. Effectively you are conflating defining the "cardinality of a set is 2" with "2." To illustrate this, your formal definition can be satisfied by a set that only has one member ({{0}} with 0 as the empty set)! It has no strict subset with one element! Your formal definition is but a numeral system, your informal definition is the number itself. Indeed nothing changes in spirit if we start with "{0}" as 0 instead of the empty set itself, but your formal definition fails in another way.

Another more dramatic illustration is just to define this entirely with natural number objects in category theory, which do away with properties of sets and collections entirely. It leads to statements which are also true, but cannot even be phrased in the language of set theory.

> I picked the ZF definition of two as "this definition" but I didn't have to. I could have picked something else, and then the phraseology would have been different. But it all eventually bottoms out in the theory of computability. You can start with ZF or Turing Machines, or the lambda calculus, or Godel numbers, or Brainfuck. It doesn't matter. All starting points lead to the same destination.

This is the entire point, Turing Machines, lambda calculus, Godel numbers are all different sets of rules and symbols. Yet we recognize that they are all the "same destination" somehow and that destination is the thing we want, not the rules and symbols themselves, which are, as you say, only the "starting point." In effect, none of these definitions of natural number can fully capture what a natural number is. They each come with their own infelicities and ultimately end up as numeral systems when fully formalized rather than the natural numbers themselves.

I agree that computability is one of the key insights behind mathematics, but moving symbols around is only one encoding of computability.

EDIT: Another thing you still haven't addressed is the issue of the reality of natural numbers and Godel's incompleteness theorems and how we are allowed to talk about "natural numbers" as mathematical objects in that scenario.

> that destination is the thing we want

Agreed. But...

> moving symbols around is only one encoding of computability

I think we may be using different definitions of the word "symbol" then. Can you give me an example of an instantiation of this "destination" that is not moving symbols around?

> Can you give me an example of an instantiation of this "destination" that is not moving symbols around?

Visual proofs are a good example of this: https://mathoverflow.net/questions/8846/proofs-without-words The main objection against them is that diagrams can be subtly manipulated to lie or that they are overly specific in what they prove. Which leads me to...

A clearer but unfortunately unrealistic example of this would be if we could have direct mind-to-mind communication in the future. I think the bulk of mathematicians would argue we would still have mathematics in that world and any symbol written down would be only an encoding of the mathematical ideas rather than the mathematics themselves, which are transmitted directly as abstract thought.

Put another way, symbolic proofs are a crutch for mathematics born out of the frailties of the human body. If humans were capable of perfect, repeatable actions or otherwise directly transmit information, then symbols would not be needed to do mathematics. We could perform computations "directly" as it were. We use symbols because that is not possible, but that is a contingent concern, not a necessary one.

But again the most thorny question of what mathematics is about comes from the reality of the natural numbers and Godel's incompleteness theorem. Most mathematicians would agree we study the natural numbers as they exist in our reality, and yet those can't be fully captured by symbols, so what exactly are we studying then?

Now this whole line of argumentation feels a bit unnatural to me (I don't believe all of what I've been saying in these replies). As I mentioned at the beginning of all this that I'm actually quite a bit of a formalist myself despite all this and I'm really hiding behind a realist costume while I dance around here. In the absence of direct mind-to-mind communication, formalism is a great practical benchmark for rigor in mathematics (can you see how to turn this prose description into a symbol manipulation game? No? Well then the description is probably not precise enough). And that's good enough for me to put on a formalist hat most of the time when I do mathematics (there are certain intuitions in mathematical logic that are harder to grasp with a formalist hat on so that's when I take it off). I think it's as good a backup to have as "shut up and calculate" is in physics.

But as a philosophical understanding of what mathematics "really is all about," I still find it wanting, especially if I believe that there is only one "true" set of natural numbers in our universe (which sometimes I don't, but if you believe that the natural numbers are physically realized by real objects, then it's a compelling conclusion), because if you believe that, then Godel's Incompleteness Theorems are a philosophical death blow for formalism.

But... speaking for myself personally and leaving aside the persona I've adopted for the purposes of this discussion, I don't usually care what mathematics "really is all about." I do have a formalist counterargument for Godel's Incompleteneess theorems as well, but I've already written too much already and will only reply to that if anybody is interested.

EDIT: This is an even better example of visual proofs: https://moodle.tau.ac.il/2018/pluginfile.php/403616/mod_reso... again the fact that they are not 100% trustworthy is more a problem of physical deficiencies in humans and materials than anything else.

EDIT EDIT: Another example of non-symbolic computation would be using physical processes for computation. Again as a practical matter, this is an inversion of results, since we do not have a perfect understanding of physical principles, but assuming we were perfect physicists and there are indeed entirely featureless pure particles (which it seems like there are), then we could do computation via physics directly instead of symbols.

EDIT EDIT EDIT: Although this can be rephrased in a formalist framework of symbols and manipulation rules, a great practical example of this is diagram chasing in category theory. Although, unlike the case of pure particles, you can argue that this is just a symbol game in 2-d, it certainly doesn't feel like one in the heat of the moment and comes closest to approximating what rigorous, but non-symbolic proofs would "feel" like.

> symbolic proofs are a crutch for mathematics born out of the frailties of the human body

Well, that's an interesting perspective. I disagree, but I don't have the energy to make my case right now so we'll have to take this up some other time. I would like to hear what you have to say about the incompleteness theorems some time.

RE the incompleteness theorems, first let's lay out why I think it's a death blow for formalism if you believe in a single true set of natural numbers and you believe that these natural numbers are a mathematical object.

The basic argument of formalism is that it is the symbols and manipulations that are the essential bedrock of mathematics and everything else is just an application of those symbols. Yet at the same time, the natural numbers as they truly exist in our universe seems to be a valid, if not foundational, object of mathematical study and they occupy a curious epistemological state. On the one hand, they are abstract entities that do not exist in the same way that physical objects exist in our universe. On the other hand, they seem to have an objective truth value. It is false to say that 2 + 3 = 4 if we interpret 2, 3, 4, and + in the usual way, in a stronger way than just symbols on a piece of paper would imply. If 2 + 3 = 4, then we just simply say that the symbols 2, 3, and 4 must not represent natural numbers, or else 4 really is just a weird way of writing five. This is of course just rehashing the difference between numerals and natural numbers, but I think most mathematicians would agree they are interested in studying the latter, and not the former except as a tool to understand the latter.

But again we hit this weird issue, where it seems like mathematics now is trying to study some objective thing that exists in our universe, because after all consequences of the natural numbers have real ramifications for physical behavior in our universe. However, by Godel's incompleteness theorems we cannot capture the natural numbers in their entirety with symbols. There will always be a new axiom you need to add. And most importantly, if there's truly a single set of natural numbers, that axiom is determined. Unlike an arbitrary axiom system, you can't willy-nilly just add the opposite of the correct axiom because then you aren't describing the natural numbers anymore.

Now you can say that the "natural numbers" as they exist in our universe are not a proper object of mathematical study. Rather our limited formalization of them in say the first order Peano axioms is the proper object of mathematical study. And since we're talking about definitions here, there's no way to say that's wrong: one can always draw the lines in mathematics wherever one prefers. But then the question arises, what field is responsible for studying the "true" natural numbers, rather than the pale reflection of them we see in the Peano axioms? If not math, then what? Physics? And yet natural numbers seem far too abstract a thing to study in physics. And it sounds so weird that something as foundational as the natural numbers is not an object of mathematical study.

And moreover it's fairly clear if you talk to mathematicians that the "true" natural numbers hold primacy over the Peano Axioms. If we found something about the natural numbers that contradicted the Peano Axioms, we'd throw out the latter and keep the former, not the other way around. That is we use the Peano Axioms to study the natural numbers, which necessarily exist outside of symbols by Godel's incompleteness theorems, not the other way around.

Indeed the very idea of there being objectively "correct" rather than only "useful" or "applicable" axioms is a very different intuition of how to do mathematics than what is implied by formalism. And it's very hard to escape that there are objectively "correct" axioms that can only be accessed through some form of informal intuition (since by nature of being axioms they cannot be accessed by formal proofs) if you believe that there is a single true set of natural numbers and take Godel's incompleteness theorems into account. (Note that this is not just hypotheticals, if you read the papers of set theorist giants such as Hugh Woodin, they are content to declare certain axioms as "true" or "false" based on certain argumentation). In that world those informal lines of arguments are extremely important and the centerpiece of mathematics and the best that formal arguments can do is serve as a backstop to make sure we aren't making egregious mistakes. That is in the case of natural numbers our main mission is to make a variety of mathematical, but ultimately informal arguments to declare the validity of a new axiom, and then just keep an eye out that we haven't introduced inconsistency accidentally (note that often, except with rather conservative extensions, we can't even formally check whether we've preserved consistency, even if we assume our original axioms were consistent, by Godel's incompleteness theorems! We've really got our hands tied if we only accept formal arguments)

Now if I put my formalist hat on, how would I respond? Well first, let's consider plausible discoveries that would seriously cast doubt on a single "true" set of natural numbers.

Most mathematicians think that something like the Continuum Hypothesis doesn't have an absolute truth value (although most set theorists probably would disagree). It is an empirical observation that mathematicians are broadly in agreement that every arithmetic statement about the natural numbers so far has an absolute truth value. However, the potential remains that someone could come up with an arithmetic statement independent of PA that large portions of the mathematical community view in the same way as the Continuum Hypothesis. The mathematical realist would argue that this is not possible, but it's not clear to me a priori that that's the case (although the fact that it has not yet shown up after so much scrutiny about the natural numbers is perhaps a good indication that it is the case). If such a statement were to be found, that would be very strong empirical evidence that the notion of "one true set of natural numbers" is wrong.

Another, perhaps more likely, possibility is if through various exotic physical processes we had access to nonstandard natural numbers, i.e. if we found evidence that the physical Church-Turing thesis was wrong and that there are physical computations which cannot be simulated by symbolic ones. This would be a philosophical triumph for Godel's completeness theorem and by extension formalism (formalism bears a close relationship to semantic completeness, stated roughly as the idea that if a formal axiom system is consistent, there is some conceivable world where that axiom system is applicable) and would also be a strong case against the thesis that there is one true set of natural numbers. Simultaneously though it would jeopardize the primacy of symbolic computation and the same arguments would probably be rehashed about the equivalent "hyper natural numbers."

So as a formalist my first instinct would be to question the idea of "one true set of natural numbers" and to say that in fact natural numbers in the objective sense do not exist. That they seem to exist is only an artifact of limited human experience. (to be fair this not completely absolve the formalist. An intuitionist could cut in at this point and say "I believe everything you've said so far, but I'm still not a formalist")

But anyways, that's enough for now. I'll return if there's interest.

EDIT: Another possibility is to deny the reality of the natural numbers as an entity altogether to various different degrees which is roughly what something like finitism or ultrafinitism is. In this world, you would basically deny that statements such as "for all natural numbers" or "there exists a natural number" have a coherent physical interpretation or at least that portions of those statements don't make sense.

Are you aware of the Löwenheim–Skolem theorem?

Yes although I don't think it's particularly important for the natural numbers. While countability is not absolute, being countable is upwards absolute, so if you expand your universe enough you eventually get the natural numbers and never lose them so to speak.

The more interesting theorem here is Tennenbaum's Theorem, which gives strong philosophical ammunition to realists that there is only one true set of natural numbers (and is a good argument why my hypothetical arithmetical statement might never exist). It's also another good reason why PA is so special (e.g. Robinson arithmetic is enough for Godel's incompleteness theorems but it fails Tennenbaum's Theorem).

> Tennenbaum's Theorem

Heh, that's interesting. I'm going to have to ponder that for a while. Thanks for the pointer.

> No, it isn't. You're just wrong about that.

> Now, it is true that there are different kinds of numbers in math, but each different kind of number is formally defined. All of those definitions are interrelated, and collectively they define what it means to be a number. There is nothing informal about it.

> though explaining why is much too long for an HN comment.

I'm surprised you're still arguing here, basically from first principles and ad hoc arguments, as to whether there exists in current professional discourse a formal model for the concepts you want. You can just Name. The. Model.

Imagine I said that a bacon McDonalds burger doesn't exist, and you decided to argue from internal and ad hoc reasonings as to whether McDonalds most assuredly has a bacon burger — when you could've just Named. The. Burger. You know, like the Bacon McDouble.

Anyways, to be clear, "number" is an informal concept among mathematicians.

If we were arguing about bacon burgers then I would not argue from first principles. I would just invite you to a restaurant and show you a bacon burger.

You're right that there is no formal definition of "number" but there is a formal definition of "natural number" and "real number" and "imaginary number" and "complex number" and "rational number" and integers and quaternions and octonians, all of which are kinds of numbers even though their names don't contain the word "number"). All of those different kinds of formally defined numbers collectively are what a "number" is. If you want to call that informal, fine, but that seems to me like a pretty tiny garnish of informality on top of a mountain of formality.

> If you want to call that informal, fine, but that seems to me like a pretty tiny garnish of informality on top of a mountain of formality.

Perhaps you forgot, but you decided that this sin of sprinkling a tiny garnish of informality was so circular and awful that you had to stop reading right there, and provide paragraphs of explanation as to how "number" is a formal concept.

If you really thought this was just a tiny garnish of informality atop a mountain of formalism, then I don't know why this is the point you had to stop reading to warn people that this work sucks.

Because it's not the informality of the definition that's the problem, it's the circularity.

If that's your definition of formal, then "thingy" is also a formal term in mathematics.

This is will be inaccessible for anyone who actually starts from scratch, it spends too much time on topics that are largely irrelevant and it isn't the best resource for anyone who has the time to spend on the irrelevant parts.

I don't think there's a good target audience for it.

Know of any resources that do accomplish the goal of starting from scratch? I’d be interested

Mathematics: From the birth of numbers by Jan gullberg

A worthwhile investment

I recently started working through Basic Mathematics by Lang. It has exercises with solutions to about half of them. It starts basic but introduces simple proofs early. Although what counts as 'from scratch' depends on the reader I guess. The book doesn't define numbers via sets, maybe check the table of contents.

I am kinda disappointed the "math foundations" (first part) does not have the word "logarithm" anywhere on the page, as this is for cs / AI.

The CS part of this thing (different page) does talk briefly about logarithms, but just a little. In one part it says "A (base-10) logarithm of a number is essentially how many digits a number has."

The final (AI) part then ends up needing exponential functions and logarithmic functions. It references a book

"Obtain the book Mathematical Modeling and Applied Calculus listed in the beginning of this page via library genesis or other means, and skip to chapter 1.4 Exponential Functions. There is a short demonstration here that assumes you have a basic idea what a derivative is from the other above videos. Read chapter 1.5 on inverses, then 1.6 Logarithmic Functions. By read I mean leisurely skim through the material knowing when to go practice or read more in depth later if needed."

I skimmed the three chapters of the book to see how they teach it, and i really kinda didn't like the way they approached it and what they leave out. I think even if you understood everything there, you wouldn't really have a good understanding of log and exponent, what e actually is, how logs relate to each other, how they relate to polynomials, how exponential funcs relate to doubly exponential ones and so on... just like good roots to be able to understand complexity really well. I dunno.

It's a draft right? Maybe they will fix that up and make it more self-contained. I don't think i would really want to do it like this from scratch. Math is too often taught in such a mechanical way without really explaining how everything relates. I want people to actually really have good intuition about the things i teach them before i use them as requisite to teach them something that requires them.

Is AI really a good place to start? Why even do it this way?

I'm currently searching for good math resources.

My girlfriend started studying CS and didn't have math for 15 years now.

It's quite hard to fill the gaps. Here some trigonometry is missing, there some logarithms, etc.

Also, finding a good amount of exercise tasks to work through isn't so easy.

Disclaimer: I'm the author of these books.

I have a book that might be of interest to you and/or your girlfriend as a review of math fundamentals like high school math (with exercises): https://nobsmath.com/ see also free concept maps https://minireference.com/static/conceptmaps/math_concepts.p... and book preview https://minireference.com/static/excerpts/noBSmath_v5_previe...

For university level stuff, I have a Linear Algebra book: https://minireference.com/static/excerpts/noBSLA_v2_preview.... and a MECH+CALC book too https://minireference.com/static/excerpts/noBSmathphys_v5_pr... Check out the reviews on the amazons https://www.amazon.de/dp/0992001005/ and https://www.amazon.de/dp/0992001021/

Good luck with re-learning math topics... it's very cool stuff. A good source of knowledge buzz ;)

If you truly want to look at Math Foundations "from scratch", you might check out the Metamath Proof Explorer:

http://us.metamath.org/mpeuni/mmset.html

This starts from axioms of logic & set theory, & proves everything else.

Real and complex numbers info here: http://us.metamath.org/mpeuni/mmcomplex.html

I don't see a single inline exercise? This is like being taught to bike by being told to watch videos about wheels

Only a math graduate student could understand this very terse, Cliff's Notes presentation of basic math.

And the math grad student wouldn't need it, because they would have already learned it at an appropriate pace from much better sources.

My "Warehouse/Workshop Model" is a IT architecture based on mathematical prototype that the simple, classic, and widely used in social production practice, elementary school mathematics "water input/output of the pool".

Software and hardware are factories that manufacture data, so they have the same "warehouse/workshop model" and management methods as the manufacturing industry.

the "Warehouse/Workshop Model" will surely replace the "von Neumann architecture" and become the first architecture in the computer field, and it is the first architecture to achieve a unified software and hardware. Because "von Neumann architecture" lacks mathematical model support, it is impossible to prove its scientificity.

It is an epoch-making theoretical achievement in the IT field, it surpasses the "von Neumann architecture", unifies the IT software and hardware theory, and raises it from the manual workshop production theory level to the modern manufacturing industry production theory Level. Although the Apple M1 chip has not yet fully realized its theory, Apple M1 chip has become the fastest chip in the world.

It has a wide range of applications, from SOC to supercomputer, from software to hardware, from stand-alone to network, from application layer to system layer, from single thread to distributed, from general programming to explainable AI, from manufacturing industry to IT industry, from energy to finance, from the missile's "Fire-and-Forget" technology to Boeing aircraft pulse production line technology .

The Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model: https://github.com/linpengcheng/PurefunctionPipelineDataflow

Why my "warehouse/workshop model" can achieve high performance and low power consumption (take Apple M1 chip, Intel AVX-512, Qualcomm as examples) https://github.com/linpengcheng/PurefunctionPipelineDataflow...

Reply on news.ycombinator.com