Shuyuan Wang

I am a PhD student at the University of British Columbia (UBC) focusing on control theory and reinforcement learning, where I am supervised by Prof. Bhushan Gopaluni and Prof. Philip Loewen. As part of my PhD program, I had the pleasure of collaborating with Honeywell Process Solutions.

Previously, I had an internship with a wonderful group of people at Huawei Noah's Arc Lab. I completed my M.Sc. in Automation at Harbin Institute of Technology (HIT), supervised by Prof. Lixian Zhang (Fellow of IEEE), and B.Sc. in Electrical Engineering at Beijing Jiaotong University (BJTU).

Email  /  Github /  Linkedin

profile photo
profile photo Research co-op
Department of Sustainability,

Develop innovative machine learning (ML) technologies to assist Honeywell in solving sustainability challenges in industrial processing and exploring robotic control techniques. Specifically, integrate reinforcement learning (RL) and control theory to improve sample efficiency, generalization, and stability.

profile photo Research intern
Planning and Control Group,
Huawei Noah's Arc Lab

1. Developed advanced Hybrid A* planner for the company, fixed the flaw of the original version of the planner and completed the U-turn and L-turn scenarios.
2. Developed distributed planning algorithm for multi vehicle system, balancing efficiency and overall performance.
3. Received the letter of appreciation from our customer.

profile photo Research assistant
Tsinghua University

1. Designed a scheme allowing multi agents to accelerate the exploring speed of Reinforcement Learning(RL). The acceleration is demonstrated to be proportional with the amount of agents.
2. Surveyed on Hierarchical Reinforcement Learning (HRL)

Selected Research

My research interests focus on the intersection of control and reinforcement learning (RL), with applications in industrial processes and robotics. Specifically, I aim to enable RL with model-based control capabilities from the perspective of algorithmic frameworks, to improve sample efficiency, generalization and so on.

profile photo New!Reinforce actions with half of the dynamics
Shuyuan Wang, Jingliang Duan, Nathan Lawrence, Philip Loewen, Michael Forbes, Bhushan Gopaluni, Lixian Zhang
Accepted by IEEE/RSJ IROS 2024
Paper (coming soon) | Video

We develop a novel framework that integrates partial model knowledge into RL in a decoupled manner. This approach bridges RL and control frameworks without disrupting the RL structure.

profile photo Deep Hankel matrices with random elements
Nathan Lawrence, Philip Loewen, Shuyuan Wang Michael Forbes, Bhushan Gopaluni
L4DC 2024

We study the output prediction accuracy from recursively applying the same persistently exciting input sequence to the Hankel-based model.

profile photo Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior
Nathan Lawrence, Philip Loewen, Shuyuan Wang Michael Forbes, Bhushan Gopaluni

We introduce a modular framework for RL-based controller design through a 'model-free' realization of the Y-K parameterization. Additionally, we establish a data-driven stability criterion and provide a probabilistic analysis of models using Hankel matrix structures.

profile photo Velocity Planning for Multi-Vehicle Systems via Distributed Optimization
Shuyuan Wang, Hang Yu, Shuai Yuan, Shengbo Eben Li, Zepeng Ning
Paper| Video

We present a distributed velocity planning strategy for multi-vehicle cooperation within the constraints of pre-defined paths, balancing efficiency and overall performance.

profile photo Reinforcement Learning with Partial Parametric Model Knowledge
Shuyuan Wang, Philip Loewen, Nathan Lawrence, Michael Forbes, Bhushan Gopaluni
IFAC World Congress 2023

We propose Partial Knowledge Least Squares Policy Iteration (PLSPI), which utilizes incomplete information from a linear partial model while retaining the data-driven adaptability of RL towards optimal performance.

profile photo A modular framework for stabilizing deep reinforcement learning control
Nathan Lawrence, Philip Loewen, Shuyuan Wang, Michael Forbes, Bhushan Gopaluni
IFAC World Congress 2023

We propose a method for producing stable operators uses a non-recurrent neural network structure, and formulate a data-driven realization of the Y-K parameterization essentially removing the prior modeling assumption.

profile photo Switching Control of A Mecanum Wheeled Mobile Robot for Vision-Based Tracking with Intermittent Image Losses
Lixian Zhang, Shuyuan Wang, Bo Cai, Tianhe Liu, Yiming Cheng

We propose a switching control scheme to tackle tracking problem for a Mecanum wheeled mobile robot (MWMR) with camera in the presence of intermittent image losses.

profile photo TA for CHBE: Introduction of Process Control [Fall 2022 and 2023]

Led weekly tutorials and office hours. Assisted with assignment and final exam grading.

Credits to Jon Barron for the website design.