You are receiving a time series whose elements belong to a finite set. Assume the time series is distributed as a Discrete-Time Markov Chain. You receive one element at each time step.
For each time step, your goal is to produce the best possible approximation of the underlying Markov Chain, ideally through a minimal-computation update algorithm.
Has this problem ever been formulated? Does an algorithm for this problem exist? Any little bit of help is very welcome.
Thanks a lot.
P.S.: The update algorithm could include a prediction phase (as Kalman filtering does).
P.S. 2: Bonus points if the solution includes a way to discard the null hypothesis (i.e. to detect it isn’t likely to be a Markov chain, after all).