Imitation_Learning

IL.md

Useful material for Imitation Learning for Control

• An algorithmic perspective on Imitation Learning https://arxiv.org/pdf/1811.06711.pdf Book, very useful

• Tutorials/Implementation of IL algorithms https://imitation.readthedocs.io/en/latest/algorithms/bc.html

• Semi-Supervised Trajectory-Feedback Controller Synthesis for Signal Temporal Logic Specifications Karen Leung, Marco Pavone https://arxiv.org/pdf/2202.01997.pdf LSTM, well written, USing STL to guide control synthesis They use Signal Temporal Logic (STL) and expert demonstrations to synthesize a trajectory-feedback controller that satisfies a desired spatio-temporal specification while staying consistent with intuitive human-like behaviors. Of- fline, an LSTM network and an environment encoding are optimized via an adversarial training strategy. Online, a gradient-based adaptation scheme refines the controller to further robustify against disturbances.

• Deep Imitative Models for flexible inference, plannng, and control Sergey Levine, Model-based RL, good for motivation

• Trajectory Tracking Control for Robotic VehiclesUsing Counterexample Guided Training of Neural Networks, 2019 Sriram: related, https://ojs.aaai.org/index.php/ICAPS/article/view/3555/3433

• Formal synthesis of closed-form sampled-data controllers for nonlinear continuous-time systems under STL specifications. Cees F. Verdier, Niklas Kochdumper, Matthias Althoff, and Manuel Mazo Jr. https://arxiv.org/pdf/2006.04260v2.pdf They propose a counterexample-guided inductive synthesis framework for the formal synthesis of closed-form sampled-data controllers for nonlinear systems to meet STL specifications over finite- time trajectories.

• https://hackmd.io/@adonze/rJC7Mu9Li

• Survey, artificial citations probably generated by ChatGPT https://arxiv.org/pdf/2303.11191.pdf

• MPC-based Imitation Learning for Safe and Human-like Autonomous Driving, 2022 Flavia Sofia Acerbo, Jan Swevers, Tinne Tuytelaars, Tong Duy Son https://arxiv.org/pdf/2206.12348.pdf

• Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving, 2021 Flavia Sofia Acerbo, Mohsen Alirezaei, Herman Van der Auweraer, Tong Duy Son https://arxiv.org/pdf/2110.04052.pdf

• Safe and Computational Efficient Imitation Learning for Autonomous Vehicle Driving, 2020 Flavia Sofia Acerbo, Herman Van der Auweraer, Tong Duy Son https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9147256

• Counterexample-Guided Synthesis of Perception Models and Control, ACC 2021 https://arxiv.org/pdf/1911.01523.pdf Seshia's group - nice clear formalization

• List: https://github.com/apexrl/Imitation-Learning-Paper-Lists

• • Trajectory Tracking Control for Robotic VehiclesUsing Counterexample Guided Training of Neural Networks, 2019 Sriram: related, https://ojs.aaai.org/index.php/ICAPS/article/view/3555/3433

• Formal synthesis of closed-form sampled-data controllers for nonlinear continuous-time systems under STL specifications. Cees F. Verdier, Niklas Kochdumper, Matthias Althoff, and Manuel Mazo Jr. https://arxiv.org/pdf/2006.04260v2.pdf They propose a counterexample-guided inductive synthesis framework for the formal synthesis of closed-form sampled-data controllers for nonlinear systems to meet STL specifications over finite- time trajectories.

• https://hackmd.io/@adonze/rJC7Mu9Li

• Survey, artificial citations probably generated by ChatGPT https://arxiv.org/pdf/2303.11191.pdf

• MPC-based Imitation Learning for Safe and Human-like Autonomous Driving, 2022 Flavia Sofia Acerbo, Jan Swevers, Tinne Tuytelaars, Tong Duy Son https://arxiv.org/pdf/2206.12348.pdf

• Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving, 2021 Flavia Sofia Acerbo, Mohsen Alirezaei, Herman Van der Auweraer, Tong Duy Son https://arxiv.org/pdf/2110.04052.pdf

• Safe and Computational Efficient Imitation Learning for Autonomous Vehicle Driving, 2020 Flavia Sofia Acerbo, Herman Van der Auweraer, Tong Duy Son https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9147256

• Counterexample-Guided Synthesis of Perception Models and Control, ACC 2021 https://arxiv.org/pdf/1911.01523.pdf Seshia's group - nice clear formalization

• List: https://github.com/apexrl/Imitation-Learning-Paper-Lists

• Parallelized Control-Aware Motion Planning With Learned Controller Proxies, 2023 https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10052750 Motivation & relatedwork for replacing MPC by NN

• Interpretable Policies from Formally-Specified Temporal Properties Jonathan DeCastro, Karen Leung, Nikos Arechiga, Marco Pavone https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9294442 pSTL

• Training Neural Network Controllers Using Control Barrier Functions in the Presence of Disturbances, 2020 Sriram, Fainekos https://arxiv.org/pdf/2001.08088.pdf

• Follow The Rules: Online Signal Temporal Logic Tree Search for Guided Imitation Learning in Stochastic Domains, 2023 Jasmine Jerry Aloor, Jay Patrikar, Parv Kapoor, Jean Oh and Sebastian Scherer Nice motivation and related work

  Most prior work focuses on offline backpropagating STL robustness along with imitation learning loss to improve the trained policy’s constraint satisfaction. These proposed offline methods that learn from either a margin based on the lower bound of STL satisfaction [11], reward functions [12], [13], vector representation [14], or risk metrics [15]. While offline learning has led to improved STL sat- isfaction, there are no guarantees that the resulting con- troller will produce satisfying trajectories [16] nor can it accommodate post hoc specification changes

• Learning from Demonstrations using Signal Temporal Logic, 2021 Aniruddh G. Puranic, Jyotirmoy V. Deshmukh and Stefanos Nikolaidis https://arxiv.org/pdf/2102.07730.pdf

• Imitation Learning from MPC for Quadrupedal Multi-Gait Control Alexander Reske, Jan Carius, Yuntao Ma, Farbod Farshidian, Marco Hutter https://arxiv.org/pdf/2103.14331.pdf Imitation learning on MPC

• Safe end-to-end imitation learning for model predictive control, 2019 Keuntaek Lee, Kamil Saigol and Evangelos A. Theodorou https://arxiv.org/pdf/1803.10231.pdf

• https://www.mathworks.com/help/reinforcement-learning/ug/imitate-mpc-controller-for-lane-keeping-assist.html