Since current actions in a game of rummy depend on how you got there, i.e. if several 7's have been played, you are unlikely to try to collect 7's I would say no. That is, the next event depends on previous events and in a Markov chain, the next event (state) does not depend on previous events (states).