CV
Yiyang Huang
Summary
I am a Ph.D. candidate in Computer Engineering at Northeastern University. I study how multimodal agents use tools and learn from feedback, how vision-language models understand long videos, and why they hallucinate.
Education
- Ph.D. in Computer EngineeringSep 2024 – May 2029 (expected)Northeastern University, Boston, MAAdvised by Prof. Yun Raymond Fu.
- M.S. in Computer ScienceSep 2021 – Jun 2024Xidian University, Xi’an, ChinaAdvised by Prof. Xuefeng Liang.
- B.Eng. in Intelligence Science and TechnologySep 2017 – Jun 2021Xidian University, Xi’an, China
Research and Industry Experience
- Ph.D. CandidateSep 2024 – PresentNortheastern University, Boston, MAAdvised by Prof. Yun Raymond Fu.
- Developed RACER, a training-free agent that decomposes questions, retrieves video frames, and refines its search using visual feedback. Across four video LLMs, it improved accuracy over uniform sampling by 4.3–6.6 percentage points on Video-MME, MLVU, and LongVideoBench.
- Developed SHIELD to address encoder bias with inference-time token interventions and adversarial contrastive decoding. It reduced captioning hallucinations by 27% relative to the baseline across multiple vision-language model families.
- Developed D-CoDe to adapt image-pretrained models to video question answering through question decomposition and dynamic frame and token compression. It improved accuracy on egocentric videos by 29% relative to the LLaVA-NeXT-7B baseline.
- Research InternMay 2026 – Nov 2026Adobe Research, San Jose, CAStudied multimodal agents, VLM reasoning, and self-improvement with mentors Zhaowen Wang and Shraman Pramanick.
- Built DesignJudge, a dataset of 7,475 layout preference pairs. Post-trained Qwen3.5-9B and Qwen3.8-27B to critique and score layouts.
- Developed a loop in which a VLM evaluator critiques a layout, a Codex agent edits it, and the evaluator scores the result. Used the resulting preference pairs for iterative GRPO training.
- GRACE-27B reached 83.7% accuracy on DesignJudge. In a user study, a majority of raters preferred GRACE-guided edits to Gemini-guided edits on 70% of the Crello layouts.
- Research InternMay 2025 – Nov 2025Adobe Research, San Jose, CAMentored by Zhaowen Wang, Simon Jenni, and Jing Shi.
- Developed MASON to ground layout metadata in the image and model the 3D structure of design elements. With Qwen2.5-VL-7B as its backbone, MASON achieved relative improvements of 26% over Gemini-2.5-Pro and 15% over GPT-5/GPT-o3.
- Built CoDeLayout, a benchmark of approximately 20,000 examples, by mining compositional structure and generating multimodal questions and answers.
Skills
Languages
- Python
- LaTeX
Machine learning
- PyTorch
- Hugging Face Transformers
- Hugging Face Datasets
Training and fine-tuning
- Accelerate
- PEFT
- bitsandbytes
Reinforcement learning and post-training
- TRL
- verl
- OpenRLHF
Inference and evaluation
- vLLM
- lmms-eval
Video processing
- FFmpeg
- PyAV
- decord
Systems and infrastructure
- Linux
- Git
- Slurm
Publications
First-Author Papers
- SHIELD: Suppressing Hallucinations in LVLM Encoders via Bias and Vulnerability Defense2026
- Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models2026Yiyang Huang, Zhaowen Wang, Simon Jenni, Jing Shi, Yitian Zhang, Yizhou Wang, Yun FuECCV
- Distorted or Fabricated? A Survey on Hallucination in Video LLMs2026Yiyang Huang, Yitian Zhang, Yizhou Wang, Mingyuan Zhang, Liang Shi, Huimin Zeng, Yun FuACL
- D-CoDe: Scaling Image-Pretrained VLMs to Video via Dynamic Compression and Question Decomposition2025
- CALLip: Lipreading using Contrastive and Attribute Learning2021
- When Plausibility Fails: Learning Human-Aligned Layout Evaluation with VLMsYiyang Huang, Zhaowen Wang, Shraman Pramanick, Yun FuUnder review
- RACER: Reflective Agent Coupling Query Interpretation and Tool-Based Retrieval for Frame Selection in Long Video Understanding2026Yiyang Huang, Yitian Zhang, Yizhou Wang, Jianglin Lu, Qihua Dong, Hailing Wang, Huimin Zeng, Mingyuan Zhang, Yun FuPreprint
Selected Co-Authored Papers
- StreamVLM: Adapting Offline Vision-Language Models for Streaming UnderstandingHailing Wang, Jianglin Lu, Xu Ma, Yiyang Huang, Yun FuUnder review
- Rethinking Fine-Tuning: Unlocking Hidden Capabilities in Vision-Language Models2025
- Capturing Individual Differences of Facial Expression for Authentic Expression Generation2026
Academic Service
- Conference reviewer · ICLR, NeurIPS, ACL, EMNLP, WWW, IEEE Face & Gesture (FG)
- Journal reviewer · ACM TKDD
Awards
- 2022 · Outstanding Student, Xidian University
- 2021 · National Scholarship, China
- 2021 · First Prize, Undergraduate Computer Design Competition (National Level), China
- 2019 · Second Prize, RoboMaster National Robotics Competition, China
- 2019 · Third Prize, ICRA AI Challenge