It seems clear that Marcus Aurelius believed in god(s). It is possible to rationalize some of his generic references to them as not necessarily reflecting faith, but rather a generic piety. This one, for instance: “To the gods I am indebted for having good grandfathers, good parents, a good sister, good teachers, good associates, good kinsmen and friends, nearly everything good.” (I.17).
Dorothea Frede’s chapter in The Cambridge Companion to the Stoics is about determinism, and it’s a must read for anyone seriously interested in the philosophy of Stoicism.
Frede’s approach is to contrast the Stoics with Aristotle, because the two schools share a number of positions, and yet differ markedly in certain crucial respects. The idea is that it is easier to appreciate the differences if one also has an understanding of the similarities.