A real-world application of Markov chain Monte Carlo method for Bayesian trajectory control of a robotic manipulator

Tavakol Aghaei, Vahid and Ağababaoğlu, Arda and Yıldırım, Sinan and Onat, Ahmet (2022) A real-world application of Markov chain Monte Carlo method for Bayesian trajectory control of a robotic manipulator. ISA Transactions, 125 . pp. 580-590. ISSN 0019-0578 (Print) 1879-2022 (Online)

This is the latest version of this item.

Full text not available from this repository. (Request a copy)

Abstract

Reinforcement learning methods are being applied to control problems in robotics domain. These algorithms are well suited for dealing with the continuous large scale state spaces in robotics field. Even though policy search methods related to stochastic gradient optimization algorithms have become a successful candidate for coping with challenging robotics and control problems in recent years, they may become unstable when abrupt variations occur in gradient computations. Moreover, they may end up with a locally optimal solution. To avoid these disadvantages, a Markov chain Monte Carlo (MCMC) algorithm for policy learning under the RL configuration is proposed. The policy space is explored in a non-contiguous manner such that higher reward regions have a higher probability of being visited. The proposed algorithm is applied in a risk-sensitive setting where the reward structure is multiplicative. Our method has the advantages of being model-free and gradient-free, as well as being suitable for real-world implementation. The merits of the proposed algorithm are shown with experimental evaluations on a 2-Degree of Freedom robot arm. The experiments demonstrate that it can perform a thorough policy space search while maintaining adequate control performance and can learn a complex trajectory control task within a small finite number of iteration steps.
Item Type: Article
Uncontrolled Keywords: Bayesian learning; Intelligent control; Markov chain Monte Carlo; Policy search; Reinforcement learning
Divisions: Faculty of Engineering and Natural Sciences > Academic programs > Industrial Engineering
Faculty of Engineering and Natural Sciences > Academic programs > Mechatronics
Faculty of Engineering and Natural Sciences
Depositing User: Sinan Yıldırım
Date Deposited: 30 Aug 2022 18:24
Last Modified: 30 Aug 2022 18:24
URI: https://research.sabanciuniv.edu/id/eprint/43666

Available Versions of this Item

Actions (login required)

View Item
View Item