|
Research co-op
Department of Sustainability,
Honeywell
Develop innovative machine learning (ML) technologies to assist Honeywell in solving sustainability challenges in industrial processing and exploring robotic control techniques.
Specifically, integrate reinforcement learning (RL) and control theory to improve sample efficiency, generalization, and stability.
|
|
Research intern
Planning and Control Group,
Huawei Noah's Arc Lab
1. Developed advanced Hybrid A* planner for the company, fixed the flaw of the original version of the planner and completed the U-turn and L-turn scenarios.
2. Developed distributed planning algorithm for multi vehicle system, balancing efficiency and overall performance.
3. Received the letter of appreciation from our customer.
|
|
Research assistant
iDLab
Tsinghua University
1. Designed a scheme allowing multi agents to accelerate the exploring speed of Reinforcement Learning(RL). The acceleration is demonstrated to be proportional with the amount of agents.
2. Surveyed on Hierarchical Reinforcement Learning (HRL)
|
Selected Research
My research interests focus on the intersection of control and reinforcement learning (RL), with applications in industrial processes and robotics.
Specifically, I aim to enable RL with model-based control capabilities from the perspective of algorithmic frameworks, to improve sample efficiency, generalization and so on.
|
|
New!Empowering Neural Networks with Control and Planning Abilities
Shuyuan Wang,
Philip Loewen,
Michael Forbes,
Bhushan Gopaluni
Accepted by NeuralPS 2024 Workshop on Behavioral Machine Learning
Paper (coming soon)
We present a framework for differentiating through iLQR controllers via implicit differentiation, providing an analytical gradient solution with constant backward cost and accurate gradients for end-to-end learning.
|
|
Guiding Reinforcement Learning with Incomplete System Dynamics
Shuyuan Wang,
Jingliang Duan,
Nathan Lawrence,
Philip Loewen,
Michael Forbes,
Bhushan Gopaluni,
Lixian Zhang
Accepted by IEEE/RSJ IROS 2024
Paper | Video
We develop a novel framework that integrates partial model knowledge into RL in a decoupled manner. This approach bridges RL and control frameworks without disrupting the RL structure.
|
|
Deep Hankel matrices with random elements
Nathan Lawrence,
Philip Loewen,
Shuyuan Wang
Michael Forbes,
Bhushan Gopaluni
L4DC 2024
Paper
We study the output prediction accuracy from recursively applying the same persistently exciting input sequence to the Hankel-based model.
|
|
Stabilizing reinforcement learning control: A modular
framework for optimizing over all stable behavior
Nathan Lawrence,
Philip Loewen,
Shuyuan Wang
Michael Forbes,
Bhushan Gopaluni
Automatica
Paper
We introduce a modular framework for RL-based controller design through a 'model-free' realization of the Y-K parameterization.
Additionally, we establish a data-driven stability criterion and provide a probabilistic analysis of models using Hankel matrix structures.
|
|
Velocity Planning for Multi-Vehicle Systems via Distributed Optimization
Shuyuan Wang,
Hang Yu,
Shuai Yuan,
Shengbo Eben Li,
Zepeng Ning
IEEE ITSC 2023
Paper| Video
We present a distributed velocity planning strategy for multi-vehicle cooperation within the constraints of pre-defined paths, balancing efficiency and overall performance.
|
|
Reinforcement Learning with Partial Parametric Model Knowledge
Shuyuan Wang,
Philip Loewen,
Nathan Lawrence,
Michael Forbes,
Bhushan Gopaluni
IFAC World Congress 2023
Paper
We propose Partial Knowledge Least Squares Policy Iteration (PLSPI),
which utilizes incomplete information from a linear partial model while retaining the data-driven adaptability of RL towards optimal performance.
|
|
A modular framework for stabilizing deep reinforcement learning control
Nathan Lawrence,
Philip Loewen,
Shuyuan Wang,
Michael Forbes,
Bhushan Gopaluni
IFAC World Congress 2023
Paper
We propose a method for producing stable operators uses a non-recurrent neural network structure,
and formulate a data-driven realization of the Y-K parameterization essentially removing the prior modeling assumption.
|
|
Switching Control of A Mecanum Wheeled Mobile Robot for Vision-Based Tracking with Intermittent Image Losses
Lixian Zhang,
Shuyuan Wang,
Bo Cai, Tianhe Liu, Yiming Cheng
IEEE SMC 2019
Paper
We propose a switching control scheme to tackle tracking problem for a Mecanum wheeled mobile robot (MWMR) with camera in the presence of intermittent image losses.
|
|
TA for CHBE: Introduction of Process Control [Fall 2022 and 2023]
Led weekly tutorials and office hours. Assisted with assignment and final exam grading.
|
|