Siyu Huang

Siyu Huang (黄思羽)

Assistant Professor at Clemson University
Research Interests: Computer Vision, Deep Learning, Generative Model

Address: 218 McAdams Hall, Clemson, SC 29631
Email: siyuh@clemson.edu
My Google Scholar

I am joining School of Computing at Clemson University 2023 Fall as an Assistant Professor.

I am actively looking for motivated PhD students and interns.

Education

Ph.D. at College of Information Science and Electronic Engineering, Zhejiang University
Sept 2014 - July 2019 Hangzhou, China
Supervised by Prof. Zhongfei (Mark) Zhang and Prof. Xi Li.
Visiting Scholar at Language Technologies Institute, School of Computer Science, Carnegie Mellon University
Jan 2018 - Jan 2019 Pittsburgh, USA
Supervised by Prof. Alexander G. Hauptmann.
Bachelor at College of Information Science and Electronic Engineering, Zhejiang University
Sept 2010 - June 2014 Hangzhou, China
Chu Kochen Honors College

Work Experiences

Tenure-Track Assistant Professor at Clemson University
School of Computing
Aug 2023 - Now Clemson, SC, USA
Postdoctoral Fellow at Harvard University
Visual Computing Group (VCG), School of Engineering and Applied Sciences
Nov 2021 - Aug 2023 Boston, USA
Working with Prof. Hanspeter Pfister
Research Fellow at Nanyang Technological University
Rapid-Rich Object Search (ROSE) Lab, School of Electrical and Electronic Engineering
Feb 2021 - Oct 2021 Singapore
Working with Dr. Bihan Wen
Research Scientist at Baidu Research
Big Data Lab
July 2019 - Jan 2021 Beijing, China
Working with Prof. Dejing Dou and Prof. Jun Huan
Research Intern at Baidu Research
Big Data Lab
Mar 2019 - June 2019 Beijing, China
Supervised by Prof. Jun Huan.

Teaching

CPSC 8810: Machine Learning-based Image Synthesis [2023 Fall]
This course offers a comprehensive exploration of machine learning techniques for visual data (e.g., images or videos) synthesis. The course will cover a range of topics from classical algorithms (e.g., image filtering and transformation) and deep generative models (e.g., VAEs, GANs, and Diffusion Models). Participants will learn to implement image synthesis algorithms, to understand cutting-edge image synthesis techniques, and to explore intriguing research questions. This course will be of particular interest to students seeking to delve into fields of generative AI, computer vision, and deep learning.
CPSC 4070/6070: Applied Computer Vision [2024 Spring]
This course offers an introduction to fundamental principles and real-world applications of 2D, 3D, and deep learning-based computer vision. Major topics include image filtering, feature detection and matching, recognition and tracking, scene understanding, camera imaging geometry, stereo vision, and deep learning-based vision. Students will learn to implement interesting computer vision algorithms in a series of well designed projects. Students will also explore intriguing research questions during a final project. This course will be of particular interest to students seeking to delve into fields of image processing and computer vision.

Publications

Learning Gaze-aware Compositional GAN from Limited Annotations [pdf]
Nerea Aranjuelo Ansa, Siyu Huang, Ignacio Arganda-Carreras, Luis Unzueta Irurtia, Oihana Otaegui Madurga, Hanspeter Pfister, Donglai Wei
ACM Symposium of Eye Tracking Research & Applications (ETRA), 2024
S3-TTA: Scale-Style Selection for Test-Time Augmentation in Biomedical Image Segmentation [pdf]
Kangxian Xie, Siyu Huang, Sebastian Andres Cajas Ordonez, Hanspeter Pfister, Donglai Wei
IEEE International Symposium on Biomedical Imaging (ISBI), 2024
Towards Robust Image Denoising via Flow-based Joint Image and Noise Model [pdf]
Lanqing Guo, Siyu Huang, Haosen Liu, and Bihan Wen
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2023
ContRE: A Complementary Measure for Robustness Evaluation of Deep Networks via Contrastive Examples [pdf]
Xuhong Li, Xuanyu Wu, Linghe Kong, Xiao Zhang, Siyu Huang, Dejing Dou, and Haoyi Xiong
IEEE International Conference on Data Mining (ICDM), 2023
Domain-Scalable Unpaired Image Translation via Latent Space Anchoring [pdf] [code]
Siyu Huang*, Jie An*, Donglai Wei, Zudi Lin, Jiebo Luo, Hanspeter Pfister
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023
3D Domain Adaptive Instance Segmentation via Cyclic Segmentation GANs [pdf] [project] [code] [dataset]
Leander Lauenburg, Zudi Lin, Ruihan Zhang, Marcia dos Santos, Siyu Huang, Ignacio Arganda-Carreras, Edward S. Boyden, Hanspeter Pfister, Donglai Wei
IEEE Journal of Biomedical and Health Informatics (JBHI), 2023
QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity [pdf] [slides] [code]
Siyu Huang*, Jie An*, Donglai Wei, Jiebo Luo, Hanspeter Pfister
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023
ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal [pdf] [code]
Lanqing Guo, Chong Wang, Wenhan Yang, Siyu Huang, Yufei Wang, Hanspeter Pfister, Bihan Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Making Your First Choice: To Address Cold Start Problem in Vision Active Learning [pdf] [code]
Liangyu Chen, Yutong Bai, Siyu Huang, Yongyi Lu, Bihan Wen, Alan Yuille, Zongwei Zhou
Medical Imaging with Deep Learning (MIDL), 2023
Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study [pdf] [code]
Xuhong Li, Haoyi Xiong, Siyu Huang, Shilei Ji, Dejing Dou
Machine Learning, European Conference on Machine Learning 2022 journal track (MLJ), 2023
ShadowFormer: Global Context Helps Shadow Removal [pdf] [code]
Lanqing Guo, Siyu Huang, Ding Liu, Hao Cheng, Bihan Wen
AAAI Conference on Artificial Intelligence (AAAI), 2023
Temporal Output Discrepancy for Loss Estimation-based Active Learning [pdf] [code]
Siyu Huang, Tianyang Wang, Haoyi Xiong, Bihan Wen, Jun Huan, Dejing Dou
IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022
MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts [pdf]
Weibin Liao, Haoyi Xiong, Qingzhong Wang, Yan Mo, Xuhong Li, Yi Liu, Zeyu Chen, Siyu Huang, Dejing Dou
Medical Image Computing and Computer Assisted Intervention (MICCAI), 2022
A Unified Framework for Bidirectional Prototype Learning from Contaminated Faces across Heterogeneous Domains [pdf] [code]
Meng Pang, Binghui Wang, Siyu Huang, Yiu-ming Cheung, Bihan Wen
IEEE Transactions on Information Forensics and Security (TIFS), 2022
Parameter-Free Style Projection for Arbitrary Style Transfer [pdf] [code] [Official PaddleHub]
Siyu Huang, Haoyi Xiong, Tianyang Wang, Bihan Wen, Qingzhong Wang, Zeyu Chen, Jun Huan, Dejing Dou
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2022
BM-NAS: Bilevel Multimodal Neural Architecture Search [pdf] [video] [code]
Yihang Yin, Siyu Huang, Xiang Zhang
AAAI Conference on Artificial Intelligence (AAAI), 2022 (Oral)
AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators [pdf] [video] [code]
Yihang Yin, Qingzhong Wang, Siyu Huang, Haoyi Xiong, Xiang Zhang
AAAI Conference on Artificial Intelligence (AAAI), 2022
Boosting Active Learning via Improving Test Performance [pdf] [code]
Tianyang Wang, Xingjian Li, Pengkun Yang, Guosheng Hu, Xiangrui Zeng, Siyu Huang, Cheng-Zhong Xu, Min Xu
AAAI Conference on Artificial Intelligence (AAAI), 2022
Semi-Supervised Active Learning with Temporal Output Discrepancy [pdf] [supp] [poster] [slides] [video] [code]
Siyu Huang, Tianyang Wang, Haoyi Xiong, Jun Huan, Dejing Dou
International Conference on Computer Vision (ICCV), 2021
ReLLIE: Deep Reinforcement Learning for Customized Low-Light Image Enhancement [pdf] [code]
Rongkai Zhang, Lanqing Guo, Siyu Huang, Bihan Wen
ACM International Conference on Multimedia (ACM MM), 2021
An Investigation of Containment Measure Implementation and Public Responses to the COVID-19 Pandemic in Mainland China [pdf]
Ji Liu, Haoyi Xiong, Xiakai Wang, Jizhou Huang, Qiaojun Li, Tongtong Huang, Siyu Huang, Haifeng Wang, Dejing Dou
IEEE International Conference on Digital Health (ICDH), 2021
ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows [pdf] [supp] [code]
Jie An*, Siyu Huang*, Yibing Song, Dejing Dou, Wei Liu, Jiebo Luo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Dual Low-Rank Multimodal Fusion [pdf]
Tao Jin*, Siyu Huang*, Yingming Li, Zhongfei Zhang
Findings of the Association for Computational Linguistics: EMNLP (EMNLP Findings), 2020
Neighbours Matter: Image Captioning with Similar Images [pdf]
Qingzhong Wang, Jiuniu Wang, Antoni Chan, Siyu Huang, Haoyi Xiong, Xingjian Li, Dejing Dou
British Machine Vision Conference (BMVC), 2020
Generating Person Images with Appearance-aware Pose Stylizer [pdf] [code]
Siyu Huang, Haoyi Xiong, Zhi-Qi Cheng, Qingzhong Wang, Xingran Zhou, Bihan Wen, Jun Huan, Dejing Dou
International Joint Conference on Artificial Intelligence (IJCAI), 2020
SBAT: Video Captioning with Sparse Boundary-Aware Transformer [pdf]
Tao Jin, Siyu Huang, Ming Chen, Yingming Li, Zhongfei Zhang
International Joint Conference on Artificial Intelligence (IJCAI), 2020
Stacked Pooling for Boosting Scale Invariance of Crowd Counting [pdf] [code]
Siyu Huang, Xi Li, Zhi-Qi Cheng, Zhongfei Zhang, Alexander Hauptmann
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020
Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning [pdf]
Tao Jin, Siyu Huang*, Yingming Li, Zhongfei Zhang
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019
Text Guided Person Image Synthesis [pdf] [supp]
Xingran Zhou, Siyu Huang*, Bin Li, Yingming Li, Jiachen Li, Zhongfei Zhang.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019
User-Ranking Video Summarization with Multi-Stage Spatio-Temporal Representation [pdf] [demo]
Siyu Huang, Xi Li, Zhongfei Zhang, Fei Wu, Junwei Han.
IEEE Transactions on Image Processing (TIP), 2019
Perceiving Physical Equation by Observing Visual Scenarios [pdf]
Siyu Huang*, Zhi-Qi Cheng*, Xi Li, Xiao Wu, Zhongfei Zhang, Alexander Hauptmann.
NeurIPS Workshop on Modeling the Physical World, 2018
TVT: Two-View Transformer Network for Video Captioning [pdf]
Ming Chen, Yingming Li, Zhongfei Zhang, Siyu Huang.
Asian Conference on Machine Learning (ACML), 2018
GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning [pdf] [slides] [poster]
Siyu Huang, Xi Li, Zhi-Qi Cheng, Zhongfei Zhang, Alexander Hauptmann.
ACM International Conference on Multimedia (ACM MM), 2018 (Oral)
Learning to Transfer: Generalizable Attribute Learning with Multitask Neural Model Search [pdf]
Zhi-Qi Cheng, Xiao Wu, Siyu Huang, Jun-Xiu Li, Alexander Hauptmann, Qiang Peng.
ACM International Conference on Multimedia (ACM MM), 2018
Body Structure Aware Deep Crowd Counting [pdf] [demo]
Siyu Huang, Xi Li, Zhongfei Zhang, Fei Wu, Shenghua Gao, Rongrong Ji, Junwei Han.
IEEE Transactions on Image Processing (TIP), 2018
Deep Learning Driven Visual Path Prediction From a Single Image [pdf] [dataset]
Siyu Huang, Xi Li, Zhongfei Zhang, Zhouzhou He, Fei Wu, Wei Liu, Jinhui Tang, Yueting Zhuang.
IEEE Transactions on Image Processing (TIP), 2016

Contest

AI Challenger Global Contest, 2017
Organizers: Sinovation Ventures, ByteDance, Sogou
Track: English-Chinese Simultaneous Interpretation Competition
Ranked: 2nd/675 teams [news] [short slides]
Team Members：Siyu Huang (team leader), Ming Chen, Lei Wang

Patents

Data mining system, method, and storage medium. Ji Liu, Haoyi Xiong, Siyu Huang, Dejing Dou. Filed, US20210248139A1, CN111597226A
Pose transfer system, method, and storage medium. Siyu Huang, Haoyi Xiong, Dejing Dou. Granted, CN111680623A
A text-guided person image generation algorithm. Xingran Zhou, Siyu Huang, Bin Li, Yingming Li, Zhongfei Zhang. Granted, CN110021051B
A one-dimensional sequence learning based video summarization algorithm. Siyu Huang, Xi Li, Zhongfei Zhang. Granted, CN107729821A
A pedestrian body appearance structure based crowd counting algorithm. Siyu Huang, Xi Li, Zhongfei Zhang. Granted, CN106815563B

Invited Talks and Presentations

Conference Presentation: "Parameter-Free Style Projection for Arbitrary Style Transfer".
International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Virtual. May 2022.
Conference Presentation: "Semi-Supervised Active Learning with Temporal Output Discrepancy".
International Conference on Computer Vision (ICCV). Virtual. Oct 2021.
Invited Talk: “Visual Understanding and Prediction Techniques”.
UESTC International Forum for Young Scholars. Virtual, Sept 2021.
Conference Presentation: "Generating Person Images with Appearance-aware Pose Stylizer".
29th International Joint Conference on Artificial Intelligence (IJCAI). Virtual. Jan 2021.
Open Course: “Automated Deep Learning: Theory and Applications”.
Baidu World Congress, Branch Forum. Beijing, Sept 2020.
Conference Tutorial: “A Tutorial on Neural Architecture Search”.
IEEE International Conference on Data Mining (ICDM) Tutorial on Automated Deep Learning: Theory, Algorithms, Platforms, and Applications. Beijing, Nov 2019.
Open Course: “NAS with RL and Differentiable NAS in AutoDL”.
Baidu Create 2019 - Baidu AI Developer Conference. Beijing, July 2019.
Invited Talk: “Deep Cross-Modal Knowledge Mining”.
Forum for Talented Young Scholar in Computer Science. Zhejiang University, Hangzhou, May 2019.
Invited Talk: "Text Guided Person Image Synthesis".
Baidu Research. Beijing, China. Mar 2019.
Conference Presentation: "GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning".
ACM International Conference on Multimedia (ACM MM). Seoul, Korea. Oct 2018. [photo]
Invited Talk: "Greedy Neural Architecture Search for Multi-Attribute Learning".
State University of New York (SUNY) at Binghamton, Binghamton, USA. Sept 2018.

Awards

Outstanding Research. Baidu Research. 2020.
Excellent Postgraduate Students' Award. Zhejiang University. 2019.
ACM MM Student Travel Award. ACM SIGMM. 2018.
National Scholarship for PhD Student. Ministry of Education of the P.R.C. 2017. Awarded to top 1% PhD students.
Postgraduate of Merit. Zhejiang University. 2017, 2018.
Award of Honor for Postgraduate. Zhejiang University. 2017, 2018.
Graduate of Chu Kochen Honors College. Zhejiang University. 2014.
Excellent Bachelor’s Thesis "Feature Learning and Its Applications". Zhejiang University. 2014.

Professional Services

Area chair/meta reviewer:

Journal reviewer:

Conference reviewer:

Since Nov, 2018