Statistical Computing for Scientists and Engineers

Thank you so much for sharing this!!!

It looks like a graduate class. It's not expected that this is the first time someone in the course has come across most of the concepts and methods.

The syllabus could almost be from an inverse theory class in my field (geology), albeit one with more focus on the underlying mathematics. I don't think it's trying to do too much, it's just not trying to be a intro course.

The professor himself probably agrees with you. In lecture 18 the professor states

At 3:40 - For this topic he actually has has 6 or 7 lectures, 100 pages of notes. But it's compressed to an introduction.

At 7:10 - He states HMM is 2-3 lectures, but he's going to compress it to half a lecture.

Honestly, I had courses like this in grad school. The were typically seminars and usually graded on a curve because people crammed stuff into their brains as quickly as possible and barely understood anything! They were meant to give you a broad coverage of a field, not comprehensively cover any particular set of topics.

It’s been a while since I’ve done serious statistical work (I’m a logician. There’s many different kinds of distributions (i.e. lines through a field), from which you would make very different inferences - relationships which satisfy a power law can of look like a Gaussian/normal distribution. Finding the “right” distribution when you know you’re working with incomplete data isn’t simple.

I can assure you they aren’t trying to make the math look scary - to be a bit blunt, a comment like that’s usually a sign that someone hasn’t made a serious attempt to engage with a field. That sort of bad faith regarding a core discipline of the mathematical sciences probably should be downvoted.

I have seen people in physics and other physical sciences work with statistics. My sense is that the less someone has to worry about "measurement error", the less intuitive statics seems to them.

For example, in some areas of research, once you have the right instrument take measurements, you can just plot the results, and your measurements (or some transformation of them) will show up as a linear function of whatever you manipulated.

But really, I'm not totally sure what you mean, since in any situation where there's uncertainty, it makes sense to me that you'd want to try to capture that uncertainty in your analysis, and that's statistics. Make those analyses more complex (e.g. taking measurements from a field with obvious spatial dependence between measures), and the models become more complex too.

Ah, but not just any curve will do, will it. You need a lot of "scary looking math" to make your case for where you draw that curve.

There are some problems with the field of statistics. They may stem partially from its role in "certifying" results in other sciences, which induces a certain conservatism, and a desire by many less scrupulous practitioners to just crank out the result they want, never mind understanding how the method works, or whether it is appropriate. It also hasn't (perhaps until recently) been that "sexy" a field, so it may attract fewer really bright people. Finally, there is a tendency for bright people who come into the field from math to think that statistics is a sub-field of math, when in fact there are philosophical issues of inductive inference that are not just math, and practical skills in data analysis that are not necessarily easy just because you're good at math.

Some particular problems:

1) Scary math and pointlessly obscure terminology are indeed a problem. For unnecessarily scary math, the early literature on Dirichlet process mixture models is a good example - almost like they were designed to be incomprehensible to most people who do actually have enough background to use the results. At a lower level, there are pointlessly obscure and misleading terms like "score function", "coefficient of determination", and worst of all, "standard error of the estimate" for the estimated standard deviation of residuals in a regression model (worst since it is not in fact a "standard error" by the general definition of that term).

2) Introductory statistics is generally taught from a naive "frequentist" perspective, because that's been the tradition for the last century or so. The justifications offered in such courses for using p-values and confidence intervals are not defensible - they just sound plausible if you don't know better. There is no good solution, since more sophisticated frequentist arguments will be beyond the level of the course, and shifting to a Bayesian perspective cuts the students off from the scientific literature with p-values, etc. that they will need to be able to read.

3) Outsiders coming into the field often have strange ideas. You might think that physicists capable of building a billion dollar accelerator would be able to recognize when a statistical method they think of is nonsense, but you'd be wrong. There's a tendency for anyone who learns information theory before statistics to think that information theory is tremendously relevant - but no, rephrasing maximum likelihood or Bayesian methods in information theory terms may sometimes be slightly helpful in thinking about them, but doesn't really add anything fundamental. And no, there's nothing particularly special or interesting about distributions that maximize entropy subject to some (generally arbitrarily selected) constraint.

4) There's a tendency to want more than you can get. There is no one "objectively correct" model/prior/analysis for a data set. Subjective assessments are unavoidable. But a lot of people don't want to accept this fact, and devote great efforts to ways of trying to pretend otherwise.

However, if you think statistics is just a simple matter of running a curve through a cloud of points, you're very wrong. Even running a curve through a could of points is a complicated and subtle enough task that deep issues arise, and these issues become much more obvious if you're trying to fit a function of hundreds of variables rather than just one. And if you're trying to not just "fit" data but come to valid conclusions about cause and effect, or about underlying latent variables that will provide useful information in new contexts, then you really do need to know a lot.

In general that is an inaccurate and naive criticism of a vast area of work. I also think you are confusing statistics with its application by scientists.

However, there is some truth in this regarding the application by scientists. I did my PhD in the field of computational statistics applied to population genetics. There you are usually trying to infer past history, and therefore have no hope of experimental verification. Although some important advances in computational statistics originate in that field, there were also lots of papers just throwing increasingly fancy Bayesian MCMC software at datasets without the honesty of experimental physics about what is knowable and what sadly might not be.

Also, many scientists, away from physics, simply do not have the mathematical maturity to understand most of statistics.

>To me this also seems intentional, like someone is trying to hide the lack of real content.

You are half correct. Something is intentionally hidden, but it's not the lack of real content.

The real stuff is called 'Mathematical Statistics' and what you learn in school is 'Statistic's for Scientists and Engineers' or 'Applied Statistics'.

Statistics loos to you random because it's useful everywhere and it must be taught to so many people in other fields. You get a basic set of per-selected tools that are useful and the real stuff is hidden.

If you really want to spend the time and effort to see it as fundamental from ground up, you have to learn mathematic background like set theory, Borel sets, Lebesque measures and integrals, Fourier integrals, etc. Then you start working up from fundamental axioms of random variables. The payday for practically oriented mind comes later and learning it may not be as motivating.

As a statistician I will try to defend my field.

Statistic is a field to represent the noise and uncertainty by quantifying it.

It is unclear to you or lacking in brutal honesty but the honest truth is the world is not perfect. If Kobe can shoot 100% of the free throw then you can perfectly model it without statistic but sometime he misses and because of that we need statistic to take into account the fact that while we expect him to make freethrows sometime he miss his shot. Without statistic you cannot take into account the sometime Kobe miss shot.

It is also the field that define in definite what significant means. Statistically significant is the difference between this med really works versus it's a placebo/not sig enough.

Another concrete example of not perfect is data. Either collection of data (missing data, measurement, etc..) is not perfect and statistic helps. You can do ordinary least square without caring about statistic but finding outliers in the data, fixing the missing data, etc.. require statistic iirc.

I glanced at the lecture notes and this looks much different (imo better) than your usual stats course. I didn't even see any mention of the usual hypothesis/significance testing mumbojumbo that has been causing all the trouble. They say:

"It is a course intended for those that value the role of Bayesian inference and machine learning on [sic] their research."

It's certainly a field plagued by some intentional obfuscation and annoying jargon (krigging anyone?), but there's real value in it like anything else. You can prove that to yourself easily. Choose a noise process and generate x,y samples off a line + noise. try ordinary lst-squares and then some gaussian processes...

Basically the value of statistics lies in the fact that there's some human-tractable set of noise processes and correlations which appear universally in a wide spectrum of real phenomena.

Knowing the real mechanism which generates the noise is always better, but not always tractable.

> Let the down-voting begin

It violates the HN guidelines to include this sort of baity noise in comments here. We'd appreciate it if you'd read and follow the rules. They're at https://news.ycombinator.com/newsguidelines.html.