Postdoctoral Fellow MILA (Montreal Institute for Learning Algorithms) Email: penny.ling.pan [@] gmail [DOT] com pl17 [@] mails [DOT] tsinghua [DOT] edu [DOT] cn [Google Scholar] [LinkedIn]

About Me

I am a postdoctoral fellow at MILA supervised by Prof. Yoshua Bengio. Prior to that, I received my Ph.D. from the Institute for Interdisciplinary Information Sciences (IIIS) (headed by Prof. Andrew Yao), Tsinghua University in 2022, advised by Prof. Longbo Huang. I received my B.E. from the School of Computer Science and Engineering (ranking 1/435 for four consecutive years during my undergraduate study), Sun Yat-Sen (Zhongshan) University, Guangzhou, China in 2017.

I visited Stanford University from April 2021 to January 2022, advised by Prof. Tengyu Ma. I visited University of Oxford from September 2020 to January 2021, advised by Prof. Shimon Whiteson. I was a research intern in the Machine Learning Group at Microsoft Research Asia from July 2018 to March 2019, advised by Dr. Wei Chen. I was also a recepient of Microsoft Research Asia Fellowship (2020).

Please feel free to drop me an email if you are interested in collaborating with me.

Research Interests

My research interests mainly include theoretical understanding, algorithmic improvements and practical application of generative flow networks (GFlowNets), reinforcement learning and multi-agent systems. I focus on developing robust, efficient, and practical deep reinforcement learning algorithms. I am also interested in the application of reinforcement learning in practical problems like computational sustainability and drug discovery.

Publications

(* indicates equal contribution)

• Better Training of GFlowNets with Local Credit and Incomplete Trajectories
  Ling Pan, Nikolay Malkin, Dinghuai Zhang, Yoshua Bengio
  Preprint
  [PDF]
  

• Stochastic Generative Flow Networks
  Ling Pan*, Dinghuai Zhang*, Moksh Jain, Longbo Huang, Yoshua Bengio
  Preprint
  [PDF]
  

• Distributional GFlowNets with Quantile Flows
  Dinghuai Zhang*, Ling Pan*, Ricky T.Q. Chen, Aaron Courville, Yoshua Bengio
  Preprint
  [PDF]
  

• Generative Augmented Flow Networks
  Ling Pan, Dinghuai Zhang, Aaron Courville, Longbo Huang, Yoshua Bengio
  In Eleventh International Conference on Learning Representations (ICLR), Kigali, Rwanda, 2023
  Spotlight (Top 5%)
  [PDF] [Code]
  

• RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch
  Yiqin Tan*, Pihe Hu*, Ling Pan, Jiatai Huang, Longbo Huang
  In Eleventh International Conference on Learning Representations (ICLR), Kigali, Rwanda, 2023
  Spotlight (Top 5%)
  [PDF] [Code]
  

• E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance
  Can Chang, Ni Mu, Jiajun Wu, Ling Pan, Huazhe Xu
  In Thirty-Sixth Conference on Neural Information Processing Systems (NeurIPS), New Orleans, USA, 2022
  Spotlight (Top 5%)
  [PDF] [Website]
  

• Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
  Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu
  In Thirty-Ninth International Conference on Machine Learning (ICML), Baltimore, USA, 2022
  [PDF] [Code] [Website]
  

• Recurrent Softmax Policy Gradient for Delay-Constrained Scheduling
  Pihe Hu, Ling Pan, Yu Chen, Zhixuan Fang, Longbo Huang
  In Twenty-Third International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing (MobiHoc), Seoul, South Korea, 2022
  [PDF]
  

• Regularized Softmax Deep Multi-Agent Q-Learning
  Ling Pan, Tabish Rashid, Bei Peng, Longbo Huang, Shimon Whiteson
  In Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS), 2021
  [PDF][Code]
  

• Exploration in Policy Optimization through Multiple Paths
  Ling Pan, Qingpeng Cai, Longbo Huang
  Journal of Autonomous Agents and Multi-agent Systems (JAAMAS), 2021
  [PDF]

• Softmax Deep Double Deterministic Policy Gradients
  Ling Pan, Qingpeng Cai, Longbo Huang
  In Thirty-Fourth Conference on Neural Information Processing Systems (NeurIPS), 2020
  [PDF][Code]

• Reinforcement Learning with Dynamic Boltzmann Softmax Updates
  Ling Pan, Qingpeng Cai, Qi Meng, Wei Chen, Longbo Huang
  In Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI), 2020, Yokohama, Japan
  (Acceptance rate: 12.6%)
  [PDF]

• Multi-Path Policy Optimization
  Ling Pan, Qingpeng Cai, Longbo Huang
  In Nineteenth International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2020, Auckland, New Zealand
  Invited for fast-track publication in JAAMAS (Top 5%)
  [PDF]

• Deterministic Value-Policy Gradients
  Qingpeng Cai*, Ling Pan*, Pingzhong Tang
  In Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 2020, New York, USA
  [PDF]

• A Deep Reinforcement Learning Framework for Rebalancing Dockless Bike Sharing Systems
  Ling Pan, Qingpeng Cai, Zhixuan Fang, Pingzhong Tang, Longbo Huang
  In Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), 2019, Hawaii, USA
  (Acceptance rate: 16.2%)
  [PDF] [Slides] [Poster]
  

Selected Awards

• Outstanding Doctoral Thesis, by Tsinghua University, 2022
  Thesis: Towards Robust, Efficient, and Practical Deep Reinforcement Learning Algorithms

  The only selected computer science doctoral thesis in IIIS, Tsinghua University

• Outstanding Graduate (top 3%), by Tsinghua University, 2022
  Also Beijing outstanding graduate and IIIS, Tsinghua University outstanding graduate, 2022

• China National Scholarship (top 2%), by Ministry of Education of China, 2021

• Microsoft Research Asia Fellowship, 2020
  12 outstanding Ph.D. students in computer science in the Asia-Pacific region

• China National Scholarship (top 2%), by Ministry of Education of China, 2016

• China National Scholarship (top 2%), by Ministry of Education of China, 2015

• China National Scholarship (top 2%), by Ministry of Education of China, 2014

Professional Activities

• SPC member:

  • International Joint Conference on Artificial Intelligence (IJCAI), 2021

• PC member/Reviewer:

  • International Conference on Learning Representations (ICLR), 2022-2023
  • Conference on Neural Information Processing Systems (NeurIPS), 2021-2022
  • International Conference on Machine Learning (ICML), 2021-2022
  • AAAI Conference on Artificial Intelligence (AAAI), 2021
  • Transactions of Machine Learning Research

Selected Talks/Presentations

• Towards Robust, Efficient, and Practical Reinforcement Learning
  Computer Science and Artificial Intelligence Lab (CSAIL), MIT, December, 2021
  Berkeley Artificial Intelligence Research (BAIR), UC Berkeley, November, 2021
  

• Regularized Softmax Deep Multi-Agent Q-Learning
  Reinforcement Learning China Community (RLChina), May, 2022
  AI Time NeurIPS Session (by Tsinghua University), February, 2022
  Third International Conference on Distributed Artificial Intelligence, January, 2022
  

• Softmax Deep Double Deterministic Policy Gradients
  IJCAI-Shanghai Artificial Intelligence Industry Association (SAIA) Young Elite Symposium, July, 2021
  Second International Conference on Distributed Artificial Intelligence, October, 2020
  

• A Deep Reinforcement Learning Framework for Rebalancing Dockless Bike Sharing Systems
  First International Conference on Distributed Artificial Intelligence, October, 2019
  Nanjing University, May, 2019