Hacker News Reader

Well yeah MC is usually meant for problems that are otherwise intractable, e.g. sampling complex Bayesian posteriors, or solving stochastic differential equations.

Using it for an answer you could easily get analytically isn't very useful outside of education/fun.

I agree. Here's a Python notebook that illustrates this problem on a classical gambling problem, the Saint Petersburg paradox. The notebook explains why stochastic simulation methods such as Monte Carlo are not suitable for situations where a very unlikely event has a significant impact on the output of interest in your simulations.

https://risk-engineering.org/notebook/saint-petersburg.html

While I generally agree with what your saying, I think the MCM can be a great way to validate more accurate approaches. The primary advantage for validation is you can get a reasonably accurate answer with minimal effort.

Really dependents on your philosophy. Would you rather do error propagation using this method http://lectureonline.cl.msu.edu/~mmp/labs/error/e2.htm

or would you rather perturbate your variables and resolve mean and variance. My philosophy drives me towards the computational approach over the analytical mathematical approach.

Are you sure there is even a philosophical difference there? I'm still going to use a program to do the heavy lifting of calculating the errors. I'm happy to argue that there is no philosophical difference; if both approaches are feasible and take comparable amount of time to implement then one gets the exact answer to the question at hand and the other is a poor man's shortcut for people who aren't confident in their maths skills.

The calculations should be arriving at the same numbers in the end, and implementing a simulation vs. an error propagation tree isn't going to favour the simulation for simple scenarios.

MCM is great for when doing usual error propagation is infeasible. It is sloppy when the error propagation could be done with a one or two analytic formula. Simulation isn't appropriate for problems that are easy analytically. For example, in practice, most of the examples in the article are not suitable problems for breaking out MCM (and although the author probably knows that, maybe not everyone on HN does). The Pi one might be a standard application, but even there Pi in particular is not a great choice for a showcase because Pi has some very nice analytic approximations.

I think the value is in checking assumptions - if the analytical approach assumes a normal distribution and then you do MCM and get a vastly different result then it is probably worth checking the assumptions made by the analytical model.

That is an inappropriate way of checking your assumptions. Assuming a normal is assuming a model (a model is a simplification of a situation by only considering the relative parts). Writing a simulation is also assuming a model, although the process is a lot less formal than traditional statistical regression.

If a modeler has assumed two different models of a situation and are getting vastly different results then that is evidence something is wrong, but it is also evidence that the modeler is out of their depth. It is not appropriate to fit two different models and then claim that the differences are delivering insight. The differences are revealing big gaps in the modelers understanding of key influences, rendering both models highly suspect. They should not be creating models, they should be putting more time into understanding the thing they are modeling.

There are times when a modeler would have two different models, but they should certainly not be surprised that they give different results. Indeed, they should be assuming two different models because they are going to give completely different results. There will probably be other edge cases, but in my experience they are rarer than people just making mistakes about where MCM is appropriate.

If there is an analytic model, using Monte Carlo methods is extremely useful to validate the theoretical result. If both agree, it is unlikely that there is a mistake in the theoretical result: which is valuable information.

For something as straightforward as Bernoulli, where you plug values into a formula and are done, there is no reason to use MCM. However, I am in agreement that in more complicated cases (e.g. a probability distribution is based on some result of one Markov or Markov-like calculation — or possibly multiple), and the theory is slightly or much more complicated then there is a lot of value in Monte Carlo methods.

Can I ask if I understand your use of `tail` in tail variance? Do you mean tail as in extremes of a distribution, or tails, as in losses. I'm very interested in using MC techniques to model extremes of a distribution, like 1-in-1000 year events from a Weibull distribution, but my (naive) algorithms spend a lot of time in the fat belly of the curve rather than out in the tails, but perhaps (probably) I'm holding it wrong. Is there a way to constrain the MC sampling to the tails of a distribution?

You want to look up about importance sampling. There are lots of ways to do it, the two most common are via scaling or translation.

I meant tail as in the extremes of the distribution. My point was more that once you've assumed that your events follow a Weibull distribution then the opportunities for MCM to add more insight are limited if the task at hand is to predict what the distribution and its parameters are. Obviously if you simulate a Weibull distribution you will get results in line with the info box on the Wikipedia page for the Weibull distribution, and you need to effectively assume parameters to simulate. Simulating a distribution just creates a very error-prone calculator the obscures the assumptions. I've seen people do exactly that thinking it was somehow helping - it is surprisingly easy to accidentally simulate a normal distribution, for example, because if you add a whole bunch of uncorrelated random things together (with identical distributions) then the result is normally distributed. You don't need a simulation to tell you that, and it is easier to mull over without simulation.

If I were using simulation it is because I think that something fishy happens in extreme events (eg, maybe stock returns all starting to become highly correlated in a liquidity crisis, destroying i.i.d assumptions). Or as other commenters mentioned because the thing being simulated has a distribution that is not analytically tractable. But there has to be some phenomena in there that is more complicated than standard distributions or a steady state Markov model, because they are more productive models when they work.

> Is there a way to constrain the MC sampling to the tails of a distribution?

You could sample 100 numbers at a time and drop out the middle 95? The question is maybe not well posed.

I once was in a situation where I had to hammer in a nail with a screwdriver (grabbing it by the front, hitting with the grip). The feeling I had back then is one I all too often experience again in programming.

This example isn't so much different; MCM = screw driver, problem = nail and learning proper statistics = driving to the hardware store and buying a hammer.

The only aspect where the comparison is somewhat lacking, is that the costs of having a hammer (both aquisition and storage) are minimal compared to in-depth knowledge about statistics.