While a good set of initial conditions is indeed critical, having a smaller model is helpful for modeling micro climates such as the ones you see in the Bay Area. At this resolution you can have a much more detailed representation of relief and water, which are two of the biggest drivers behind the beautiful dynamics we observe here.

Kalman filtering is only one part of the process, and plays a critical role during the data assimilation part. Classical Kalman filtering is optimal for Gaussian-distributed linear dynamical systems, but needs tweaks for non Gaussian distributions and non linear systems.

Classical NWP models for instance will integrate the primitive partial differential equations in time and space and run various parameterizations (which can be in some cases even more expensive than integrating the primitive equations). ECMWF on their end use IFS, which is a spectral method for solving the PDEs.

The whole process of solving these models accurately has definitely been some of the most fascinating science and engineering I’ve had the pleasure to work with. It’s extremely humbling :)