Periodic finite state controllers for efficient POMDP and DEC-POMDP planning
Tutkimustuotos › › vertaisarvioitu
Yksityiskohdat
| Alkuperäiskieli | Englanti |
|---|---|
| Otsikko | Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011 |
| Tila | Julkaistu - 2011 |
| OKM-julkaisutyyppi | A4 Artikkeli konferenssijulkaisussa |
| Tapahtuma | 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011 - Granada, Espanja Kesto: 12 joulukuuta 2011 → 14 joulukuuta 2011 |
Conference
| Conference | 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011 |
|---|---|
| Maa | Espanja |
| Kaupunki | Granada |
| Ajanjakso | 12/12/11 → 14/12/11 |
Tiivistelmä
Applications such as robot control and wireless communication require planning under uncertainty. Partially observable Markov decision processes (POMDPs) plan policies for single agents under uncertainty and their decentralized versions (DEC-POMDPs) find a policy for multiple agents. The policy in infinite-horizon POMDP and DEC-POMDP problems has been represented as finite state controllers (FSCs). We introduce a novel class of periodic FSCs, composed of layers connected only to the previous and next layer. Our periodic FSC method finds a deterministic finite-horizon policy and converts it to an initial periodic infinitehorizon policy. This policy is optimized by a new infinite-horizon algorithm to yield deterministic periodic policies, and by a new expectation maximization algorithm to yield stochastic periodic policies. Our method yields better results than earlier planningmethods and can compute larger solutions than with regular FSCs.