The Analytic Garden

The secret of change is to focus all of your energy, not on fighting the old, but on building the new. ~ Socrates On 2021-09-24, I downloaded the COVID-19 metadata file from GISIAD . The file contains a whopping 3,795,999 records. The records refer to individual SAR-COV-2 genome sequences uploaded. This is an unprecedented scientific effort and an amazing resource. To get a since of whir can be don with such a large collection  of data I suggest checking out this YouTube video by Trevor Bedford from the Fred Hutchinson Cancer Research Center. In this post, I want to continue exploring variants of concern (VOC) in the US. There are 1,085,764 sequences with Location labeled as "USA". First, the log of the sequence counts. We see a drop-off in the counts of the Delta variant starting around the end of August 2021. This is probably  because of a lag in the time from collection to upload of the sequences. This is a log plot of sequence counts, so the linear increase in ...

In a  previous post , I looked at the problem of combining the Bayesian Change Point algorithm with the Kalman filter. The results of applying the method to temperature anomaly data were less than spectacular. With Bayesian change point algorithm, we assume that a change point represents a change in the marginalized probability of the data. There are other ways of thinking about what a change point means. A change point could be considered an anomaly or outlier in the data. To get an idea of how outliers could be detected, consider a simple linear trend Kalman filter. \[\begin{array}{l}{y_t} = {\theta _t} + {\upsilon _t}\,\,\,\,\,\,{\upsilon _t} \sim N(0,V)\\{\theta _t} = {\theta _{t - 1}} + {\omega _t}\,\,{\omega _t} \sim N(0,{W_t})\end{array}\] In this model, we are assuming that the variances don't vary with time, but are drawn from some distribution, typically an inverse-gamma distribution . \[{V_t} = \frac{1}{{{\phi _y}}}\,\,\,\,{W_t} = \frac{1}{{{\phi _\theta }}}\] ${\ph...

One should clearly distinguish between two aspects of the estimation problem: (1) The theoretical aspect. Here interest centers on: (i) The general form of the solution. (ii) Conditions which guarantee a priori the existence, physical realizability, and stability of the optimal filter. (iii) Characterization of the general results in terms of some simple quantities, such as signal-to-noise ratio, information rate, bandwidth, etc... (2) The computational aspect. The classical (more accurately, old-fashioned) view is that a mathematical problem is solved if the solution is expressed by a formula. It is not a trivial matter, however, to substitute numbers in a formula. The current literature on the Wiener problem is full of semi-rigorously derived formulas which turn out to be unusable for practical computation when the order of the system becomes even moderately large...  ~ Rudolf E. Kalman In a pair of previous posts ( here and here ) I looked at examples of simple Kalman Filter...