For the notes, I found serious problems in just the first half of the first column on the first page.
F'get about the four pages.
If enough people want an outline in a few pages, then I'll consider knocking one out and putting up somewhere as a PDF file.
1) "Linear algebra is the math of vectors and matrices" - I think this would be better phrased as "the math of vectors and linear transformations." Matrices are a convenient way to represent finite dimensional linear transformations, but I think it's putting the cart before the horse to say that linear algebra is the math of matrices. This is a minor problem, though.
2) "A vector ⃗v ∈ R^n is an array of n real numbers" - In general vectors are not arrays of numbers, real or otherwise. This kind of definition will run into problems when you get into Hilbert/Banach spaces.
3) "the function f(x) = ln(x) has the inverse f^−1(x) = e^x" - this is only true for x > 0 if we're talking about the real-valued logarithm function, since it's not defined for x <= 0.
Generally I consider those sorts of things to be the difference between writing 'proofs' versus explaining concepts.
Surely this is ok if it is parsed as "(a vector in R^n) is an array of n real numbers".
No, such an array can be either a vector or the representation (which also has to be a vector) you mentioned.
To make such things make sense, need a definition of a vector space. It can be tempting to omit that definition from an outline, but the definition is just crucial, e.g., when talking about subspaces, which is often crucial. E.g., for m x n matrix A and n x 1 'vector' x, the set of all x such that Ax = 0 is a subspace, but to know this need the definition of a vector space.
Better: Let n be a positive integer, and let R be the set of real numbers. Then R^n is the set of all n-tuples over the real numbers. Such an n-tuple is a list of n numbers; e.g., (1, 3, 2.6, 1, 7) is an n-tuple for n = 5.
For positive integers m and n, an m x n (read "m by n") (real) 'matrix' as a rectangular array of real numbers with m rows and n columns. Typically for integer i = 1, 2, ..., m and integer j = 1, 2, ..., n, in m x n matrix A = [a_{ij}] a_{ij} is the number, 'component', in row i and column j. An example of a 2 x 3 matrix is
1 9 4
4 0 -1
Typically an n-tuple x in R^n is regarded as an n x 1 matrix. In this way, with m x n matrix A, n x 1 vector b, and matrix multiplication (we have yet to define) Ax, we can write Ax = b as a system of m linear equations in n unknowns x, and this is the most important foundation and application of linear algebra.
(2) Similarly for positive integer n and the set of complex numbers C, C^n is the set of all n-tuples of complex numbers. And, we can have an m x n matrix of complex numbers.
Once get very far in linear algebra, complex numbers become essential if only from the fundamental theorem of algebra (that is, from factoring a polynomial). In particular, eigenvalues are important, and they are often complex numbers.
(3) R^n and C^n are examples of 'vector spaces', but there are also many other examples.
For a 'vector space', we have a non-empty set V of 'vectors' and a 'field' F. Note: A 'field' is defined carefully in courses in abstract algebra, but the two leading examples in practice are R and C. Most commonly F = R or F = C, but there is also interest in other fields, e.g., the integers modulo a prime number. Here we assume that F = R or F = C.
On V we have an operation 'addition' denoted by +. So, for vectors x in V and y in V, there is a vector x + y in V. Addition is commutative so that x + y = y + x. Vector addition is also associative, that is, for x, y, z in V,
(x + y) + z = x + (y + z)
Also there is a vector 0 in V so that for any x in V we have 0 + x = x.
Also for x in V, there is -x in V so that x + (-x) = x - x = 0.
Or set V with operation + forms a commutative group (defined carefully in courses in abstract algebra).
On V with F we have an operation 'multiplication'. Then for x in V and a in F, with this 'multiplication' the 'product' ax in V.
For x, y in V and a, b in F, we have two distributive laws, (a + b)x = ax + bx and a(x + y) = ax + ay. Also, for 0 in F, 0x is the vector 0. For the vector 0, a0 is also the vector 0.
We have that (-1)x = -x.
For some examples of vector spaces, (A) let F = R and let V be the set of all monophonic audio signals exactly 10 seconds long, (B) let F = R and let V be the set of all real valued random variables X where E[X^2] is finite, and (C) let F = R and for m x n matrix A and n x 1 vector x and m x 1 vector b, let V be the set of all x such so that Ax = b (so far the matrix product Ax has not been defined, but it will be defined so that Ax = b is a system of m linear equations in n unknowns, the components of x).
(3) For set S, x in S means that x is an 'element' of S.
(4) The notation (R, R, R) = R^3 is neither good nor standard.
The notation R^3 = R X R X R as the set theory Cartesian product is standard.
Actually, we already know just what (R, R, R) is -- a 3-tuple and not a set of all 3-tuples of real numbers.
(5) R^{m x n} is questionable notation and, anyway, needs a definition.
(6) For the "power" of that mathematics, might illustrate that with some applications. One of the most important is solving systems of linear equations. For more, a matrix can describe a linear transformation, and linearity is a powerful assumption that commonly holds with good accuracy in practice. So, vectors and matricies become powerful means of working with linear systems where a matrix describes the system and the inputs and outputs of that system are vectors.
From G. F. Simmons, the two pillars of analysis in mathematics are continuity and linearity, and linear algebra emphsizes both, especially linearity.
More examples of linearity include both differentiation and integration in calculus and the Fourier transform.
(6) The definition of inverse function is wrong. That definition holds only when the given function f is 1-1. Even if f is not 1-1, the inverse function is still useful, but its values can be a set of values, that is, the inverse function is multiply valued. E.g., just take f(x) = x^2. Then the f^{-1}(1) = {-1, 1}.
It was just true. Why? It was an 'advanced' course in linear algebra and matrix theory, and I'd never had even a first course in the subject.
But long before the course, in total I'd done much more than just such a course.
I had had two good courses in abstract algebra, each of which had provided a good foundation for linear algebra and also touched on the subject. I had taken a reading course in theoretical physics based heavily on matrix theory. And I had written my undergraduate honors paper on group representation theory, essentially in linear algebra.
I'd worked carefully through two of the best books on linear algebra, one by E. Nearing and the other by P. Halmos. The Halmos book is famous as the crown jewel of books on linear algebra. Why? It can be good to look at linear algebra as a baby version of Hilbert space theory, heavily from von Neumann, and Halmos wrote his book when he was an assistant to von Neumann at the Institute for Advanced Study. So, deliberately, the Halmos book is an introduction to Hilbert space theory. It also has the profound perspective of one of the best mathematicians of all time and the elegance of thought of von Neumann.
For applications, in my career and in my own study, I'd worked through stacks of books and papers on applications in numerical linear algebra, multivariate statistics, optimization, curve fitting, signal processing, especially around the fast Fourier transform, differential equations, advanced calculus, deterministic optimal control theory, exterior algebra, etc.
At one point I wrote my own notes on much of linear algebra, with black ink on white paper, 100% cotton I got especially for the notes. My work was very careful although not elegant.
For review, just before the course, I wrote out from memory 80 pages covering about 2/3rds of the course. As Horn wrote, I knew the subject cold, long before I took his course. Horn's homework, tests, and final exam were all challenging for the other students, and I effortlessly blew them away, by large margins on all three. E.g., I never studied for the tests or either the mid-term or final exam. When doing the homework, I nearly never looked at my notes from class and, instead, just wrote out the solutions based on what I already knew.
I didn't belong in the course and had told the department that at the beginning. The department's reaction was to smile and say "Take the course anyway.". So, I did.
For me, the course was a huge waste: From a long commute to campus and taking care of my wife, I was short on time, but the Horn homework had me up nearly all night and then exhausted the next day once a week and, thus, ate into a big fraction of the best time I had each week. Some of my other, more important work suffered. When I did the Horn homework, I was usually just exhausted. I'd get the papers back, graded, in about two weeks, recognize my handwriting, but, from having been so tired, actually had no memory of having done the work. Bummer. It was a trial by fire and exhaustion, not education. I was being asked to 'prove myself', not learn something.
In the program, I only wanted to learn some good material and had no desire at all to 'compete', certainly not in that linear algebra course beneath me. I didn't know I was blowing away the other students until the grader showed me the scores on homework and tests, all except the final exam.
In many ways Horn was a very good teacher, but, in total, he blew away nearly all the students. If I hadn't already known the material coming into the course, then I couldn't have learned it in the course. The way I had learned the material was from some slow, careful, thoughtful study and a lot of work with applications, and the course offered neither.
Also, what background do you think is necessary for these texts?
As in Simmons (in this list), the two most important pillars of analysis are continuity and linearity.
The first place to learn linearity well is 'linear algebra' -- or, much the same thing, matrix theory or finite dimensional vector spaces.
Then the algebraic aspects can be emphasized and can lead to applications in, say, the theory of error correcting codes.
For the analysis aspects, can continue with Hilbert and Banach spaces and more and also various applications.
Much of the mathematics in applied mathematics, science, engineering, and technology is linear algebra.
Here is a list of books relevant to linear algebra and/or to the course I took from Roger Horn. This list also contains maybe 60% of what I'd worked from before Horn's course that let me do well in that course.
========================================
Linear Algebra
----------------------------------------
Horn's course was close to
Roger A. Horn, Charles R. Johnson, 'Matrix Analysis', 0-521-38632-2, Cambridge University Press, 1990.
with also a few topics from
Roger A. Horn, Charles R. Johnson, 'Topics in Matrix Analysis', 0-521-46713-6, Cambridge University Press, 1994.
----------------------------------------
The course also used for reference and some topics and exercises
Richard Bellman, 'Introduction to Matrix Analysis: Second Edition', McGraw-Hill, New York, 1970.
This book is just packed with little results; at some point can get the impression that the author went on and on ... writing. Bellman was a very bright guy in mathematics, engineering, and medicine.
----------------------------------------
Relatively easy to read and relatively close to applications, and another book Horn's course used as a reference, is
Ben Noble, 'Applied Linear Algebra', Prentice-Hall, Englewood Cliffs, NJ, 1969.
Some edition of this book may be a good place to start for a student interested in applications now.
----------------------------------------
About the easiest reading in this list, and my first text on linear algebra, was
D. C. Murdoch, 'Linear Algebra for Undergraduates', John Wiley and Sons, New York, 1957.
This book is awfully old, but what it has are still the basics.
----------------------------------------
For my undergraduate honors paper I made some use of, and later read carefully nearly all of,
Evar D. Nering, 'Linear Algebra and Matrix Theory', John Wiley and Sons, New York, 1964.
The main part of this book is a relatively solid start, maybe a bit terse and advanced for a first text. The book also has in the back a collection of advanced topics, some of which might be quite good to know at some point and difficult to get elsewhere.
One of the topics in the back is linear programming, and for that I'd recommend something else, e.g., Chv'atal and/or Bazaraa and Jarvis in this list.
----------------------------------------
Likely the crown jewel of books on linear algebra is
Paul R. Halmos, 'Finite-Dimensional Vector Spaces, Second Edition', D. Van Nostrand Company, Inc., Princeton, New Jersey, 1958.
Halmos wrote this in about 1942 when he was an assistant to von Neumann at the Institute for Advanced Study. The book is intended to be a finite dimensional introduction to Hilbert space theory, or how to do linear algebra using mostly only what also works in Hilbert space. It's likely fair to credit von Neumann with Hilbert space.
The book is elegant.
Apparently at one time Harvard's course Math 55, with a colorful description at,
http://www.american.com/archive/2008/march-april-magazine-co...
used this text by Halmos and also, as also in this list, Rudin's 'Principles' and Spivak.
----------------------------------------
Long highly regarded as a linear algebra text is
Hoffman and Kunze, 'Linear Algebra, Second Edition', Prentice-Hall, Englewood Cliffs, New Jersey, 1971.
========================================
Numerical Methods
----------------------------------------
If want to take numerical computations in linear algebra seriously, then consider the next book or something better if can find it
George E. Forsythe and Cleve B. Moler, 'Computer Solution of Linear Algebraic Systems', Prentice-Hall, Englewood Cliffs, 1967.
========================================
Multivariate Statistics
My main start with multivariate statistics was
N. R. Draper and H. Smith, 'Applied Regression Analysis', John Wiley and Sons, New York, 1968.
Apparently later editions of this book remain of interest.
----------------------------------------
A relatively serious book on 'regression analysis' is
C. Radhakrishna Rao, 'Linear Statistical Inference and Its Applications: Second Edition', ISBN 0-471-70823-2, John Wiley and Sons, New York, 1967.
----------------------------------------
Three famous, general books on multivariate statistics are
Maurice M. Tatsuoka, 'Multivariate Analysis: Techniques for Educational and Psychological Research', John Wiley and Sons, 1971.
William W. Cooley and Paul R. Lohnes, 'Multivariate Data Analysis', John Wiley and Sons, New York, 1971.
Donald F. Morrison, 'Multivariate Statistical Methods: Second Edition', ISBN 0-07-043186-8, McGraw-Hill, New York, 1976.
========================================
Analysis of Variance
A highly regarded first book on analsis of variance and experimental design is
George W. Snedecor and William G. Cochran, 'Statistical Methods, Sixth Edition', ISBN 0-8138-1560-6, The Iowa State University Press, Ames, Iowa, 1971.
and a famous, more mathematical, book is
Henry Scheff'e, 'Analysis of Variance', John Wiley and Sons, New York, 1967.
========================================
Linear Optimization
A highly polished book on linear programming is
Vav sek Chv'atal, 'Linear Programming', ISBN 0-7167-1587-2, W. H. Freeman, New York, 1983.
----------------------------------------
Nicely written and with more emphasis on the important special case of network flows is
Mokhtar S. Bazaraa and John J. Jarvis, 'Linear Programming and Network Flows', ISBN 0-471-06015-1, John Wiley and Sons, New York, 1977.
----------------------------------------
A grand applied mathematics dessert buffet, based on Banach space and the Hahn-Banach theorem is
David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York, 1969.
========================================
Mathematical Analysis Relevant to Understanding Linearity
----------------------------------------
Long the first place a math student gets a fully serious encounter with calculus and closely related topics has been
Walter Rudin, 'Principles of Mathematical Analysis, Third Edition', McGraw-Hill, New York, 1964.
The first chapters of this book do well as an introduction to metric spaces, and that work applies fully to vector spaces.
----------------------------------------
A nice place to get comfortable doing mathematics in several dimensions is
Wendell H. Fleming, 'Functions of Several Variables', Addison-Wesley, Reading, Massachusetts, 1965.
Some of the material here is also good for optimization.
----------------------------------------
Another place to get comfortable doing mathematics in several dimensions is
Michael Spivak, {\it Calculus on Manifolds: A Modern Approach to Classical Theorems of Advanced Calculus,\/} W.\ A.\ Benjamin, New York, 1965.\ \
----------------------------------------
The first half, the 'real' half of the next book has polished introductions to Hilbert and Banach spaces which are some of the most important vector spaces
Walter Rudin, 'Real and Complex Analysis', ISBN 07-054232-5, McGraw-Hill, New York, 1966.
----------------------------------------
An elegant introduction to how to get comfortable in metric space is
George F. Simmons, 'Introduction to Topology and Modern Analysis', McGraw Hill, New York, 1963.
========================================
Ordinary Differential Equations
----------------------------------------
Linear algebra is important, as some points crucial, for ordinary differential equations, a polished introduction from a world expert is
Earl A. Coddington, 'An Introduction to Ordinary Differential Equations', Prentice-Hall, Englewood Cliffs, NJ, 1961.
In my first post here, I misspelled the name. As in my first response to your post, the correct spelling is Nering.
> Also, what background do you think is necessary for these texts?
The background to get started in linear algebra is essentially just high school algebra. In college, linear algebra is commonly the next course after calculus.
For more, after linear algebra, commonly there is a course in 'analysis' such as Rudin's 'Principles' in my list. Without a good course, this book would be tough reading.
The applications of linear algebra are so broad that for a good sampling of those applications the prerequisites in total are essentially at least a good undergraduate major in mathematics.
That's good, because you clearly wouldn't have been the star student in English class.
Any suggestions?