Learning Locomotion with Little Dog

with M. Kalakrishnan, P. Pastor, S. Schaal
We developed an autonomous walking planner and controller architecture for a quadruped robot. This controller uses model based planning and control, including floating base inverse dynamics and force control to achieve fast, robust and compliant walking over very rough terrain.		Selected publications: M. Kalakrishnan, J. Buchli, P. Pastor, M. Mistry, and S. Schaal. Learning, planning, and control for quadruped locomotion over challenging terrain. Int. J. Robotics Research, 2011. link M. Kalakrishnan, P. Pastor, J. Buchli, M. Mistry, and S. Schaal. Fast, robust quadruped locomotion over challenging terrain. IEEE Int. Conference on Robotics and Automation (ICRA) 2010, pages 2665-2670, 2010. M. Kalakrishnan, J. Buchli, and S. Schaal. Learning locomotion on rough terrain using terrain templates. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 167-172, 2009 J. Buchli, M. Kalakrishnan, M. Mistry, P. Pastor, and S. Schaal. Compliant quadruped locomotion over rough terrain. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 814-820, 2009 J. Buchli, M. Kalakrishnan, M. Mistry, and S. Schaal P. Pastor. Compliant ZMP control for quadrupedal walking over rough terrain. Proceedings of the international conference on climbing and walking robots (CLAWAR) 2010

Control of dynamic locomotion

with C. Semini, D.G. Caldwell
We are developing a fully autonomous torque controlled hydraulic quadruped robot for dynamic outdoor locomotion. We test ideas on compliant model based locomotion and impedance control for dynamic motion and locomotion.		Selected publications: C. Semini, J. Buchli, M. Frigerio, T. Boaventura, M. Focchi, E. Guglielmino, F. Cannella, N.G. Tsagarakis and D.G. Caldwell. “HyQ - A Dynamic Locomotion Research Platform”. Int. Workshop on Bio-Inspired Robots Extended Abstract Selected publications on design of HyQ: C. Semini, “HyQ - Design and Development of a Hydraulically Actuated Quadruped Robot”, Dissertation, Italian Institute of Technology and University of Genoa, Italy, 2010. link C. Semini, N. G. Tsagarakis, E. Guglielmino, M. Focchi, F. Cannella, and D. G. Caldwell, “Design of HyQ - a hydraulically and electrically actuated quadruped robot,” IMechE Part I: J. of Systems and Control Engineering, vol. 225, no. 6, pp. 831–849, 2011.link

with E.Theodorou, F. Stulp, S. Schaal
We have shown how we can use a path integral based Reinforcement Learning algorithm (PI2) to learn not only reference trajectories but also gain schedules. Opening up new possibilities in model free/learning impedance and force control for practical robot applications.		Selected publications: J. Buchli, F. Stulp, E. Theodorou, and S. Schaal: Learning Variable impedance control, Int. Journal of Robotics Research, in print. link J. Buchli, E. Theodorou, F. Stulp, S. Schaal, Variable Impedance Control - A Reinforcement Learning Approach In Proceedings Robotics, Science and Systems, p20, 2010.

with E.Theodorou, S.Stulp, S.Schaal
We have developed a novel reinforcement learning algorithm based on path integral approach to stochastic optimal control. The resulting algorithm PI2 (Policy Improvements with Path Integrals) has a single open parameter, scales to high DOF and real robotic problem with very arbitrary cost functions.		Selected publications: E. Theodorou, J. Buchli, S. Schaal, Reinforcement Learning in High Dimensional State Spaces: A Path Integral Approach., Journal of Machine Learning Research 11 (2010) 3137-3181 E. Theodorou, J. Buchli, S. Schaal, Learning Policy Improvements with Path Integrals, In Proceedings AISTATS 2009.M. Mistry, J. Buchli, and S. Schaal. Inverse dynamics control of floating base systems. IEEE Int. Conference on Robotics and Automation (ICRA), pages 3406–3412, 2010. E. A. Theodorou, J. Buchli and S. Schaal, Path Integral-Based Stochastic Optimal Control for Rigid Body Dynamics, In 2009 IEEE International Symposium on Adaptive Dynamic Programming and Reinforcement Learning F. Stulp, J. Buchli, E. Theodorou, and S. Schaal. Reinforcement learning of full-body humanoid motor skills. In proceedings IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2010 Source code: MATLAB STOMP Motion Planner Policy Learning

with M.Mistry, L.Righetti, R.Gregg
We study model based approaches to the control of articulated robots with a floating base (such as a walking robot or a mobile manipulator). We investigate both theoretical aspects and implementation aspects of such algorithms.		Selected publications: M. Mistry, J. Buchli, and S. Schaal. Inverse dynamics control of floating base systems. IEEE Int. Conference on Robotics and Automation (ICRA), pages 3406–3412, 2010. R. D. Gregg, L. Righetti, J. Buchli, and S. Schaal. Constrained accelerations for controlled geometric reduction: Sagittal-plane decoupling for bipedal locomotion. In proceedings IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2010. L. Righetti, J. Buchli, M. Mistry, and S. Schaal. Inverse dynamics with optimal distribution of ground reaction forces for legged robots. Proceedings of the international conference on climbing and walking robots (CLAWAR) 2010. J. Buchli, M. Kalakrishnan, M. Mistry, P. Pastor, and S. Schaal. Compliant quadruped locomotion over rough terrain. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 814-820, 2009.

with F.Stulp, E.Theodorou, S.Schaal
We apply Reinforcement Learning to learning control parameters of manipulation tasks. These control parameters can be force, impedance, task space reference trajectories, etc.		Selected publications: F. Stulp, J. Buchli, E. Theodorou, and S. Schaal. Reinforcement learning of full-body humanoid motor skills. In proceedings IEEE-RAS Int. Conference on Humanoid Robots (Humanoids), 2010. J. Buchli, E. Theodorou, F. Stulp, S. Schaal, Variable Impedance Control - A Reinforcement Learning Approach In Proceedings Robotics, Science and Systems, p20, 2010.

with L.Righetti, A.J.Ijspeert
We developed a novel class of oscillators, adaptive frequency oscillators (AFO), that are able to adapt to frequencies of perturbing signals. These AFOs have been applied to robotic dynamic locomotion, signal processing and control of rehabilitation devices.		Selected publications: L. Righetti, J. Buchli, A.J. Ijspeert: Dynamic Hebbian learning in adaptive frequency oscillators, Physica D, 216(2):269-281, 2006. J. Buchli, L. Righetti, A.J. Ijspeert: Frequency Analysis with Coupled Nonlinear Oscillators, Physica D, 237(13), 1705-1718, 2008. J. Buchli, A.J. Ijspeert: Self-organized adaptive legged locomotion in a compliant quadruped robot, Autonomous Robots, Vol 25(4), 331-347, 2009. J. Buchli, L. Righetti, A.J. Ijspeert: Entrainment and Adaptation in Limit Cycle Systems, Biological Cybernetics, Vol. 95, No. 6, 2006.