Université Paris 6
Pierre et Marie Curie | Université Paris 7
Denis Diderot | |

CNRS U.M.R. 7599
| ||

``Probabilités et Modèles Aléatoires''
| ||

**Auteur(s): **

**Code(s) de Classification MSC:**

- 62L20 Stochastic approximation
- 93C40 Adaptive control systems
- 68T05 Learning and adaptive systems
- 92J40 Memory and learning, See also {68T05}
- 90A80 Resource allocation

**Résumé:** We investigate the asymptotic behaviour of the so-called two-armed
bandit algorithm. It is an example of stochastic approximation
procedure whose associated ODE has both a repulsive and an
attractive equilibrium, at which the procedure
is noiseless. We show that if the gain parameter is constant or goes
to $0$ not too fast, the algorithm does fall
in the noiseless repulsive equilibrium with positive probability
whereas it always converges to its natural
attractive target when the gain parameter goes to zero at some
appropriate rates depending on the parameters of
the model. We also elucidate the behaviour of the constant step
algorithm when the step goes to
$0$. Finally, we highlight the connection between the algorithm and
the Polya urn. An application to asset allocation
is briefly described.

**Mots Clés:** *Two-armed bandit algorithm ; Stochastic Approximation ;
learning automata ; Polya urn ; asset allocation*

**Date:** 2002-11-06

**Prépublication numéro:** *PMA-769*

**Pdf file : **PMA-769.pdf