Stochastic Decision Making

Full course description

Any realistic model of a real-world phenomenon must take into account the possibility of randomness. That is, more often than not, the quantities we are interested in will not be predictable in advance but, rather, will exhibit an inherent variation that should be taken into account by the model. Mathematically, this is usually accomplished by allowing the model to be probabilistic in nature. In this course, the following topics will be discussed:

(1) Basic concepts of probability theory: Probabilities, conditional probabilities, random variables, probability distribution functions, density functions, expectations and variances.

(2) Finding probabilities, expectations and variances of random variables in complex probabilistic experiments.

(3) Discrete and continuous time Markov chains and related stochastic processes like random walks, branching processes, Poisson processes, birth and death processes, queueing theory.

(4) Markov decision problems.

(5) Multi-armed bandit problems, bandit algorithms, contextual bandits, cumulative regret, and simple regret