This paper analyzes reinforcement learning algorithms and Markov property. To answer this question well-known game TicTacToe had been chosen and the model was played in different situations. The model is based on reinforcement learning algorithms SARSA and HMM. After performing several experiments, it is found that the model’s best and random modes were considered equally important for training. Although the model plays excellent when choosing the best available options, random moves do not lose their importance.