Hilbert space: Treating functions as vectors | Hacker News Reader

Hilbert space: Treating functions as vectors

eli.thegreenplace.net

150 pointssigna1155 comments

As evidenced by the confusion of at least one commenter, I do not think it is a good didactic way to introduce vectors by how they can be written in a particular basis.

It is just unhelpful in many ways. It fixates on one particular basis and it results in a vector space with few applications and it can not explain many of the most important function vector spaces, which are of course the L^p spaces.

In most function vector spaces you encounter in mathematics, you can not say what the value of a function at a point is. They are not defined that way.

The right didactic way, in my experience, is introducing vector spaces first. Vectors are elements of vector spaces, not because they can be written in any particular basis, but because they fulfill the formal definition. And because they fullfil the formal definition they can be written in a basis.

> It fixates on one particular basis and it results in a vector space with few applications and it can not explain many of the most important function vector spaces, which are of course the L^p spaces.

Except just about all relevant applications that exist in computer science and physics where fixating on a representation is the standard.

Most relevant applications use L^2 spaces which can not be defined point wise.

If you want to talk about applications, then this representation is especially bad. Since the intuition it gives is just straight up false.

Fwiw, my favourite textbook in communication theory (Lapidoth, A Foundation in Digital Communication) explicitly calls out this issue of working with equivalence classes of signals and chooses to derive most theorems using the tools available when working in ℒ_2 (square-integrable functions) and ℒ_1 space

In physics it is common to work explicitily with the components in a base (see tensors in relativity or representation theory), but it's also very important to understand how your quantities transform between different basis. It's a trade-off.

Haha, this works if you already know what a vector space is. But I think pedagogy needs to provide motivating examples. I'll quote one section of a text by Poincaré (translated by an LLM since most here do not speak French).

> We are in a geometry class. The teacher dictates: “A circle is the locus of points in the plane that are at the same distance from an interior point called the center.” The good student writes this sentence in his notebook; the bad student draws little stick figures in it; but neither one has understood. So the teacher takes the chalk and draws a circle on the board. “Ah!” think the students, “why didn’t he say right away: a circle is a round shape — we would have understood.”

> No doubt, it is the teacher who is right. The students’ definition would have been worthless, since it could not have served for any demonstration, and above all because it would not have given them the salutary habit of analyzing their conceptions. But they should be shown that they do not understand what they think they understand, and led to recognize the crudeness of their primitive notion, to desire on their own that it be refined and improved.

The learning comes from making the mistake and being corrected, not from being taught the definition, I think.

Anyway, it's from Science and Method, Book 2 https://fr.wikisource.org/wiki/Science_et_m%C3%A9thode/Livre...

There's more to the section that talks about the subject. I just find this particular paragraph amusingly germane.

I have nothing against starting out with motivating examples, obviously they are needed for understanding. But they should motivate the definition of a vector space. Not the definition of vectors as mappings of indices.

Functions are actually a great motivating example for the definition of a vector space, precisely because they are first look nothing like what student think of as a vector.

Thinking about this specific case, I think you are right. The manner of describing actually confuses the concept more than if it never tried to introduce the index-mapping.

It's trivial to provide motivating examples for vector spaces, and there's no reason you can't do so while explaining what they actually are, which is also very simple for anyone who understands the basic concepts of set, function, associativity and commutativity. The notion of a basis falls out very quickly and allows you to talk about lists of numbers as much as you like without ever implying any particular basis is special.

I hesitate to call anything pedagogically "wrong" as people think and learn in different ways, but I think the coyness some teachers display about the vector space concept hampers and delays a lot of students' understanding.

Edit: Actually, I think the "start with 'concrete' lists of numbers and move to 'abstract' vector spaces" approach is misguided as it is based on the idea that the vector space is an abstraction of the lists of numbers, which I think is wrong.

The vector space and the lists of numbers are two equivalent, related abstractions of some underlying thing, eg. movements in Euclidean space, investment portfolios, pixel colours, etc. The difference is that one of the abstractions is more useful for performing numerical calculations and one better expresses the mathematical structure and properties of the entities under consideration. They're not different levels of abstraction but different abstractions with different uses.

I'd be inclined to introduce the one best suited to understanding first, or at least alongside the one used for computations. Otherwise students are just memorising algorithms without understanding, which isn't what maths education should be about, IMO. (The properties of those algorithms can of course be proved without the vector space concept, but such proofs are opaque and magical, often using determinants which are introduced with no better justification than that they allow these things to be proved.)

For many students, it is not so simple to grasp the concept of an abstract vector space. They could be taking linear algebra as college freshmen, without having seen any formal algebraic structures before. Many are unfamiliar with the formal notion of a set (and certainly have not seen the actual axioms of a set before). Most linear algebra students are not actually math majors; they are typically studying engineering, computer science, or some other physical science. Examples of abstract vector spaces are most often function spaces of some form (for example, polynomials of at most a given degree). These examples are not so motivating for non-math students.

The main reason why people care about linear algebra is that it lets you solve linear systems of equations (and perform related operations, such as projections). A linear system of equations has an immediate correspondence with a matrix of coefficients, a right-hand side vector, and a solution vector. For this reason, it is very natural to first talk about matrices and vectors (they can be used to represent concretely a linear system of equations), and then introduce the concept of vector space in cases where the abstract view can be clarifying or help with understanding.

From my perspective, the "right" way to teach linear algebra depends on the mathematical maturity of the students. If they are honors math majors, they can easily handle the definition of an abstract vector space right away. If they have less mathematical maturity, the abstract viewpoint isn't helpful for them (at least not without first familiarizing themselves with the more concrete concepts). Think about it this way: we don't teach school children about natural numbers and arithmetic by first listing the Peano axioms.

I think at least in the UK the lack of "mathematical maturity" among early undergraduates is partly the result of this very coyness about mathematical concepts. Enormous time at A-Level is spent rote learning algorithms, and very little on grasping the basic concepts of mathematics, so it's hardly surprising students turn up unprepared for such simple notions as "vector space".

I don't have first hand experience of the French system, but from what I understand the approach there is more along the lines I'm thinking of, and the relative over-representation of French graduates among my more mathematical colleagues suggests it may be rather effective in practice.

That's awful - just an awful way to teach. It's from more than a century ago when the point was to tame the children and turn them into good Prussian soldiers.

You don't have to start with anxiety, shame, and dominance - you can start with curiosity, a base of common understanding, and then experiment and problem solving.

>It's from more than a century ago when the point was to tame the children and turn them into good Prussian soldiers.

If you judge by the outcome, that is probably the greatest education system of all time.

>You don't have to start with anxiety, shame, and dominance - you can start with curiosity, a base of common understanding, and then experiment and problem solving.

You can. The kids will learn nothing though.

School nowadays is a joke. An absolute waste of time. In a single semester of rigorous mathematics I learned more than in years in school. It is cruel to waste childrens time like that.

School needs to be authoritarian, rigorous and selective.

Yikes

What's awful is thinking that making mistakes is a form of shame, and that being corrected is a form of dominance. That view is something that is taught and acquired and I am very thankful that the people who taught me never made me feel that way. I make mistakes all the time and I never have to feel ashamed about it, nor do I feel that the people in my life who I've learned from hold some kind of position of power over me.

It's good that no one did it to you, but are you sure that you never try to do it to anyone else?

You win my comment-of-the day award with grace and subtlety.

Completely agree. In uni, I (re)-learned about vectors in linear algebra, and for a good chunk of the course, we didn't write anything in "standard vector notation". We learned about vector axioms first, and then vectors were treated as "anything that satisfies the vector axioms". (When doing more practical examples, we just used the reals instead of something like R^3, but the entire time it was clear that for any proof, anything that can be added and multiplied in the way that the vector axioms describe would fit.) I think adopting this structuralist view really helps with a lot of mathematical studies.

Now I’m thinking that I have missed the point of the article. I didn’t read it as an introduction to vector spaces, but rather that the introduction served as to give an intuition how functions may be viewed as vectors (going back to the article, it’s even in the section heading). I found the next parts well written and to the point, leading along the steps to show that indeed the requirements for a Hilbert space are met by L^2 (even though those requirements are only spelled out in the end). I’m not actively working with mathematics any more, but I didn’t notice any major corner cutting. It’s not text book rigorous but lays out the idea in an easy to follow way. I took something away from it - or not, depending on whether I missed some inconsistency.

> In most function vector spaces you encounter in mathematics, you can not say what the value of a function at a point is.

Could you spell out what you mean by that? Functions are all defined on their domains (by definition)

Are you referring to the L^p spaces being really equivalence classes of functions agreeing almost everywhere?

Yes, the L^p spaces are not vector spaces of functions, but essentially equivalent classes of functions that give the same result in an Lebesgue integral. For these reason, common operations on functions, like evaluating at a point or taking a derivative are undefined.

If you care about these you need something more restrictive, for example to study differential equations you can work in Sobolev spaces, where the continuity requirement allows you to identify an equivalent class with a well-defined function.

Thanks for the clarification

reminded me of "tensor is a bunch of numbers that transform in a certain way"; this should be illegal to teach, especially in physics

> In most function vector spaces you encounter in mathematics, you can not say what the value of a function at a point is. They are not defined that way.

That is because they are not vector spaces of function but a quotient of one

Tangential question that's been nagging me; in spaces of functions defined pointwise, is there a natural way to lift algebraic structures (group/ring/whatever) from the base set up to the function space? It's generally true across many separate objects in math, but I don't know a clear explanation of why. (Category theory, I assume).

Many of the propositions in the author's Appendix A are of this form.

I.e., if you look at how addition on function spaces is defined pointwise, (f+g)(x) = f(x)+g(x) -- that's different meanings of (+) on either side -- that looks exactly like the defining relation of a group homomorphism, except that the symbols are backwards.

ianam but I'm not sure there needs to be a "why" -- these things are defined to have certain properties, and so they have those properties. That's the purpose for which they were defined.

You are observing the evaluation map ev_p: C(M, R) -> R, ev_p(f) = f(p), is a ring homomorphism. In that spirit, you might find this Terry Tao article on the Yondea lemma interesting: https://terrytao.wordpress.com/2023/08/25/yonedas-lemma-as-a...

Let T^S = {f| f:S->T} be all functions from a source set S to a target structure (set + operations + relations + axioms) T.

You can lift all the operations and relations from T to T^S, and you'll get a structure with the same type signature.

Universal equations involving operations remain true when lifted. Therefore if T is a variety[0], T^S is a variety of the same type.

So for example if S a set with 2 elements, then T^S is TxT + lifted properties. If T is an Abelian group then TxT is also an Abelian group. If T is a ring, TxT is also a ring. If T is a field, TxT is not a field since (0,1) has no inverse.

What about the relations, what types of identities remain true when lifted form T to T^S?

[0] : https://en.wikipedia.org/wiki/Variety_(universal_algebra)

> But we can take it even further; what if we allow any real number as an index?

How can an uncountably infinite set be used as an index? I was fine with natural numbers (countably infinite) being an index obv, but a real seems a stretch. I get the mathematical definition of a function, but again, this feels like we suddenly lose the plot…

We do it all the time. An index is just indicative that there is a mapping (a function), usually from the integers. However we don't use the subscript notation when indexing by a continuum due to the discomfort you describe.

The point is that we need some way to deal with objects that are inherently infinite-dimensional.

I'm probably ignorant of how indexes work at a nuts-and-bolts level, but intuitively this seems like a good idea for certain situations. E.g if we want to keep entries in a specific order but don't know ahead of time how many entries will be added between two existing ones. House numbers in areas with a lot of development are an example of the kind of problem this seems ideal to solve, when there's a clear 'order' based on geography but no clear limit on the number of addresses that could be added 'between' existing addresses.

That’s kind of how I understand it as well.

I think you're still describing a countably infinite set: there's a bijection between the natural numbers and the set of houses.

One way to think about it is that, even though you're defining an index that permits infinite amounts of subdivision, from any given house there's always a "next house up" in the vector: you can move up one space.

In a real-indexed vector, that notion doesn't apply. It's "infinity plus one" all the way down: whatever real value you pick to start with, x, there's no delta small enough to add to it such that there's no number between x and x+d.

> In a real-indexed vector, that notion doesn't apply. It's "infinity plus one" all the way down: whatever real value you pick to start with, x, there's no delta small enough to add to it such that there's no number between x and x+d.

Just to clarify, uncountability isn't necessary for this. It's true for the rational numbers too, which are countable.

Yes. Indexes in infinite sets are counterintuitive, and real numbers even more so.

The famous counterexample to all of this sort of thinking is Hilbert’s hotel, which I’m sure you know but want to point it out for people who haven’t seen it before because it’s pretty mind-blowing when you first encounter it.

Say you have a hotel with an infinite number of rooms numbered 1,2,3,… and so on and they are all occupied. A guest arrives- how do you accommodate them? Well you ask the person in room one to move to room 2, the person in room 2 to move to room 3, and in general the person in room n to move to room n+1. So every existing guest has a room and room 1 is now free for your new guest.

Ok but what if an infinite number of prospective guests arrive all at once and every room in your hotel is full. How do you accommodate them? Still no problem. You ask the guest in room 1 to move to room 2, the one in room 2 to move to room 4, and in general the guest in room n to move to room 2n. Now all your existing guests still have a room but you have freed up an infinite number of (odd-numbered) rooms for your infinite number of new guests to move into.

These are all countable infinities, and Cantor showed that if the number of rooms in your infinitely-roomed hotel is ℵ_0, then the number of real numbers is 2^ℵ_0, which is obviously quite a lot more.

c.f.: fractional indexing

The author is stretching an analogy, it's a price to pay for starting with R^3 as a motivational example. There is nothing in the general definition of a vector space that requires it's elements to be "indexed"

I think that’s why the author put “vector” in quotes. I kind of imagine it as an ephemeral, infinite list where for some real, when we use that real value as an index into our “vector”/function, we get the output value as the item in this infinite, ephemeral list.

I think the only thing that matters is that the indices have an ordering (which the reals obviously do) and they aren’t irrational (i.e. they have a finite precision).

Imagine you have a real number, say, e.g. 2.4. What stops us from using that as an index into an infinite, infinitely resizable list? 2.4^2 = 5.76. Depending on how fine-grained your application requires you could say 2.41 (=5.8081) is the next index OR 2.5 (=6.25) is the next index we look at or care about.

I could be misunderstanding it, though.

A vector is always a vector -- an element of something that satisfies the axioms of a vector space. The author starts with the example of R^n, which is a very particular vector space that is finite-dimensional and comes with a "canonical" basis (0,...,1,...,0). In general, a basis will always exist for any vector space (using the axiom of choice), but there is no need to fix it, unless you do some calculations. The analogy with R^n is the only reason the "indices" are mentioned, and I think this only creates more confusion.

> and they aren’t irrational (i.e. they have a finite precision)

No, if you want only rational "indices", then your vector space has a countable basis. Interesting vector spaces in analysis are uncountably infinite dimensional. (And for this reason the usual notion of a basis is not very useful in this context.)

> and they aren’t irrational (i.e. they have a finite precision).

I'm not sure if I'm misunderstanding what you mean by 'finite precision' but the ordinary meaning of those words would seem to limit it to rational numbers?

In practice you're always computing with finite precision. (Even computing with symbolic expressions is just a preliminary step to what's ultimately a numerical result with finite precision.) The whole point of real numbers is to abstract away from detailed considerations of precision, and figure out what happens if you only ever care about putting satisfactory bounds on the output and are willing to bound your input to the extent required.

What is the benefit of using R if it's really Q?

The idea of R is that it allows you to reason about things like "I need more than X input precision to achieve Y bound on my output". Just sticking with naïve computation in Q does not suffice for that.

The only difference of note, I think, is that you can't enumerate the elements. Instead of being able to say "for each element, ..." you'd have to say "for all elements, ...", like the example of vector length defined as an integral over the full number range.

To a mathematician “each” and “all” are synonyms.

What do you understand “index” to mean here? To me, a family indexed by some set is mostly just a different notation for, and attitude towards, a function with domain the indexing set.

Consider a function on R as an |R|-dimensional vector...

Which defeats the purpose of thinking about functions as a vector space. It's all smokes and mirrors

You can look at use cases for an index, and see how well they hold up.

Asking where the smallest greater number (next number) is no longer makes sense.

Taking two numbers and asking whether one is greater than the other still makes sense. (and hence also whether they are equal)

Taking two numbers and asking how far separated from each other still makes sense.

You may already observe some uses for indexes in programming that don't use all of these properties of an index. For example, the index of a hash set "only cares about equality", and "the next index" may be an unfilled address in a hash set.

This Has it's use. The continouus Fourier Transform is is based on that. You are asking what frequencies is this continouus signal made of. Time is normally defined as a real number in that context, but If you have a continouus time you need continouus frequencies to map time space to frequency space. You can think about an Index as a lego Block, that you need to construct Something.

I think getting hung up on words (in this case index) in mathematics is a trap. They are often stretched to their breaking point and you just kind of go with the flow.

> When I use a word,’ Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’

> ’The question is,’ said Alice, ‘whether you can make words mean so many different things.’

> ’The question is,’ said Humpty Dumpty, ‘which is to be master — that’s all.

In mathematics it is the author's privilege.

Well there's no law against it.

Okay I suppose the axiom of choice is somewhat necessary to make it make sense. But only because otherwise such an indexed object may fail to exist.

Anyway arbitrary indexes are useful, you often end up doing stuff like covering a space by finding a covering set for each individual point. And then using compactness to show you only need finitely many to cover the whole space. It is doable without uncountable indices, but it makes it very difficult to write down.

In ZFC set theory, indexed family over a set (possibly uncountable or even bigger), is just syntactic sugar for a function.

So let's say you have a set U (possibly uncountable). To say let {u_i}, i in I (another set, possibly uncountable) is equivalent to asserting existence of function f:I -> U, such that f(i) = u_i. Note that this does not even require axiom of choice, since you are allowed to postulate that a function exists.

Of course if I is uncountable you can't list the elements of I, but that is not important in this case.

Wish I read this years ago

Hey thanks for writing this article. I found it to be really great.

Reply on news.ycombinator.com