Shuyuan Wang

I am a PhD student at the University of British Columbia (UBC) focusing on control theory and reinforcement learning, where I am supervised by Prof. Bhushan Gopaluni and Prof. Philip Loewen. As part of my PhD program, I had the pleasure of collaborating with Honeywell Process Solutions.

Previously, I had an internship with a wonderful group of people at Huawei Noah's Arc Lab. I completed my M.Sc. in Automation at Harbin Institute of Technology (HIT), supervised by Prof. Lixian Zhang (Fellow of IEEE), and B.Sc. in Electrical Engineering at Beijing Jiaotong University (BJTU).

Email  /  Github /  Linkedin

Develop innovative machine learning (ML) technologies to assist Honeywell in solving sustainability challenges in industrial processing and exploring robotic control techniques. Specifically, integrate reinforcement learning (RL) and control theory to improve sample efficiency, generalization, and stability.

1. Developed advanced Hybrid A* planner for the company, fixed the flaw of the original version of the planner and completed the U-turn and L-turn scenarios.
2. Developed distributed planning algorithm for multi vehicle system, balancing efficiency and overall performance.
3. Received the letter of appreciation from our customer.

1. Designed a scheme allowing multi agents to accelerate the exploring speed of Reinforcement Learning(RL). The acceleration is demonstrated to be proportional with the amount of agents.
2. Surveyed on Hierarchical Reinforcement Learning (HRL)

My research interests focus on the intersection of control and reinforcement learning (RL), with applications in industrial processes and robotics. Specifically, I aim to enable RL with model-based control capabilities from the perspective of algorithmic frameworks, to improve sample efficiency, generalization and so on.

We develop a novel framework that integrates partial model knowledge into RL in a decoupled manner. This approach bridges RL and control frameworks without disrupting the RL structure.

We study the output prediction accuracy from recursively applying the same persistently exciting input sequence to the Hankel-based model.

We introduce a modular framework for RL-based controller design through a 'model-free' realization of the Y-K parameterization. Additionally, we establish a data-driven stability criterion and provide a probabilistic analysis of models using Hankel matrix structures.

We present a distributed velocity planning strategy for multi-vehicle cooperation within the constraints of pre-defined paths, balancing efficiency and overall performance.

We propose Partial Knowledge Least Squares Policy Iteration (PLSPI), which utilizes incomplete information from a linear partial model while retaining the data-driven adaptability of RL towards optimal performance.

We propose a method for producing stable operators uses a non-recurrent neural network structure, and formulate a data-driven realization of the Y-K parameterization essentially removing the prior modeling assumption.

We propose a switching control scheme to tackle tracking problem for a Mecanum wheeled mobile robot (MWMR) with camera in the presence of intermittent image losses.

Led weekly tutorials and office hours. Assisted with assignment and final exam grading.

Credits to Jon Barron for the website design.