Siliang Zeng
 |
|
About me
I am a fifth year PhD student at University of Minnesota, advised by Prof. Mingyi Hong.
Before moving to Minnesota, I received the B.S. in Statistics from The Chinese University of Hong Kong, Shenzhen in 2020.
In my research, I am always thinking about how to design practical algorithms and systems for sequential decision making under uncertainty.
My recent research interests focus on Foundation Model Alignment and Agent.
Experience
Applied Scientist Intern, Amazon Web Search (AWS) AI Research and Education lab, Santa Clara, May 2024 - Oct 2024.
Design RLHF algorithm to align LLMs with high-quality demonstrations. (Mentor: Yao Liu and Rasool Fakoor)
Applied Scientist Intern, Amazon Web Search (AWS) AI Research and Education lab, Santa Clara, May 2023 - Oct 2023.
Design RLHF algorithm for LLM alignment with low compuational cost. (Mentor: Kaixiang Lin)
Selected Publications (* indicates equal contribution.)
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
Jiaxiang Li, Siliang Zeng, Hoi-To Wai, Chenliang Li, Alfredo Garcia, Mingyi Hong
Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024).
Also accepted to ICML 2024 Workshop on Theoretical Foundations of Foundation Models.
When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
Siliang Zeng*, Chenliang Li*, Alfredo Garcia, Mingyi Hong
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023). (Oral: 0.5% [67/12343] )
Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees
Siliang Zeng, Mingyi Hong, Alfredo Garcia
Operations Research.
Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees
Siliang Zeng, Chenliang Li, Alfredo Garcia, Mingyi Hong
Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022).
(A previous version accepted by Decision Awareness in Reinforcement Learning Workshop at ICML 2022)
A Stochastic Linearized Augmented Lagrangian Method for Decentralized Bilevel Optimization
Songtao Lu, Siliang Zeng, Xiaodong Cui, Mark S. Squillante, Lior Horesh, Brian Kingsbury, Jia Liu, Mingyi Hong
Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022).
Selected as an Honorable Mention in the IBM Pat Goldberg Memorial competition for best papers.
Recent News
Oct 2024: Two papers got accepted to NeurIPS 2024 Workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability (FITML 2024):
Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment.
Policy Optimization can be Memory-Efficient: LLM Alignment Through Successive Policy Re-weighting (SPR).
Sep 2024: One paper has been accepted to NeurIPS 2024:
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
August 2024: Our paper titled “A stochastic linearized augmented Lagrangian method for decentralized bilevel optimization” has been
selected as an Honorable Mention in the IBM Pat Goldberg Memorial competition for best papers!
July 2024: One paper has been accepted to Operations Research:
Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees
Jun 2024: One paper has been accepted to ICML 2024 Workshop on Theoretical Foundations of Foundation Models:
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
May 2024: I join Amazon Web Search (AWS) as an applied scientist intern. I will work on large language model alignment.
Apr 2024: I am thrilled to receive the 2024-25 Doctoral Dissertation Fellowship (DDF).
Apr 2024: I am invited to give a talk at Ai4 - Industry's Leading AI Conference. See you in Las Vegas!
Nov 2023: I am invited to give a talk at the Coordinated Science Laboratory of UIUC.
Oct 2023: I am thrilled to receive the NeurIPS 2023 Scholar Award. See you in New Orleans!
Sep 2023: One paper has been accepted to NeurIPS 2023 as an oral:
When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning.
August 2023: One paper has been accepted to Conference on Robot Learning (CoRL) 2023:
A Bayesian Approach to Robust Inverse Reinforcement Learning.
May 2023: I join Amazon Web Search (AWS) as an applied scientist intern. I will work on reinforcement learning and large language model training.
March 2023: I gave an invited talk at Amazon AWS about our recent work on offline inverse reinforcement learning:
Understanding Expertise through Demonstrations: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
Oct 2022: I am thrilled to receive the NeurIPS 2022 Scholar Award. See you in New Orleans!
Sep 2022: Two papers have been accepted to NeurIPS 2022:
Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees.
A Stochastic Linearized Augmented Lagrangian Method for Decentralized Bilevel Optimization.
Jul 2022: I have been selected for a travel grant from the DARL Workshop at ICML 2022!
Jun 2022: One paper has been accepted to Decision Awareness in Reinforcement Learning Workshop at ICML 2022:
Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees.
Jun 2022: One paper has been accepted to SIAM Journal on Optimization:
On the Divergence of Decentralized Non-Convex Optimization.
Mar 2022: One paper has been accepted to L4DC 2022:
Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees.
Sep 2021: One paper has been accepted to NeurIPS 2021:
A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum.