Selected Publications (* equal contribution)

Preprints

Aria-UI: Visual Grounding for GUI Instructions

Yuhao Yang, Yue Wang, Dongxu Li, Ziyang Luo, Bei Chen, Chao Huang, and Junnan Li

2025

VideoAutoArena: An Automated Arena for Evaluating Large Multimodal Models in Video Analysis through User Simulation

CVPR 2025

Ziyang Luo, Haoning Wu, Dongxu Li, Jing Ma, Mohan Kankanhalli, and Junnan Li

ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use

ICLR 2025 Workshop

Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang and Tat-Seng Chua

ScratchEval: Are GPT-4o Smarter than My Child? Evaluating Large Multimodal Models with Visual Programming Challenges

NAACL 2025

Rao Fu, Ziyang Luo, Hongzhan Lin, Zhen Ye, Jing Ma

CodeHalu: Code Hallucinations in LLMs Driven by Execution-based Verification

AAAI 2025

Yuchen Tian, Weixiang Yan, Qian Yang, Xuandong Zhao, Qian Chen, Wen Wang, Ziyang Luo, Lei Ma, Dawn Song

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?

COLING 2025

Yuwei Zhao*, Ziyang Luo*, Yuchen Tian, Hongzhan Lin, Weixiang Yan, Annan Li, and Jing Ma

GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse

ACM Transactions on Intelligent Systems and Technology (TIST)

Hongzhan Lin*, Ziyang Luo*, Bo Wang, Ruichao Yang, and Jing Ma

MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models

ICLR 2025 Workshop

Shengkang Wang*, Hongzhan Lin*, Ziyang Luo*, Zhen Ye, Guang Chen, and Jing Ma

2024

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

ICLR 2024

Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang

AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation

EMNLP 2024

Ziyang Luo, Xin Li, Hongzhan Lin, Jing Ma, and Lidong Bing

Towards Low-Resource Harmful Meme Detection with LMM Agents

EMNLP 2024

Jianzhao Huang, Hongzhan Lin, Ziyan Liu, Ziyang Luo, Guang Chen, and Jing Ma

MMCode: Evaluating Multi-Modal Code Large Language Models with Visually Rich Programming Problems

EMNLP 2024

Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, and Jing Ma

CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models

ACL 2024

Zixin Chen, Hongzhan Lin, Ziyang Luo, Mingfei Cheng, Jing Ma, and Guang Chen

Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models

WWW 2024

Hongzhan Lin, Ziyang Luo, Wei Gao, Jing Ma, Bo Wang, and Ruichao Yang

2023

LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval

ICCV 2023

Ziyang Luo, Pu Zhao, Can Xu, Xiubo Geng, Tao Shen, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang

Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models

EMNLP 2023

Hongzhan Lin*, Ziyang Luo*, Jing Ma, and Long Chen

Zero-Shot Rumor Detection with Propagation Structure via Prompt Learning

AAAI 2023

Hongzhan Lin, Pengyao Yi, Jing Ma, Haiyun Jiang, Ziyang Luo, Shuming Shi, Ruifang Liu

2022

A Coarse-to-fine Cascaded Evidence-Distillation Neural Network for Explainable Fake News Detection

COLING 2022

Zhiwei Yang, Jing Ma, Hechang Chen, Hongzhan Lin, Ziyang Luo, Yi Chang

Conditioned Masked Language and Image Modeling for Image-Text Dense Retrieval

EMNLP 2022

Ziyang Luo, Yadong Xi, Rongsheng Zhang, Gongzheng Li, Zeng Zhao, and Jing Ma

DecBERT: Enhancing the Language Understanding of BERT with Causal Attention Masks

NAACL 2022

Ziyang Luo, Yadong Xi, Jing Ma, Zhiwei Yang, Xiaoxi Mao, Changjie Fan, Rongsheng Zhang

Easy and Efficient Transformer: Scalable Inference Solution for Large NLP Model

NAACL 2022

Yadong Xi, Gongzheng Li, Jingzhen Ding, Duan Wang, Ziyang Luo, Rongsheng Zhang, Bai Liu, Changjie Fan, Xiaoxi Mao, Zeng Zhao

2021

Positional Artefacts Propagate Through Masked Language Model Embeddings

ACL 2021

Ziyang Luo, Artur Kulmizev, and Xiaoxi Mao

Smoothing with Fake Label

CIKM 2021

Ziyang Luo, Yadong Xi, and Xiaoxi Mao

Gender Bias Hidden Behind Chinese Word Embeddings: The Case of Chinese Adjectives

GeBNLP 2021

Meichun Jiao, Ziyang Luo

Have Attention Heads in BERT Learned Constituency Grammar?

EACL 2021 SRW

Ziyang Luo

Selected Open-Source Projects

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

VideoLLaMA Team@Alibaba

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

WizardLM Team@Microsoft

AURORA-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order

Aurora-M Open-Source Community

Master Thesis

Analyzing the Anisotropy Phenomenon in Transformer-based Masked Language Models

Supervised by Artur Kulmizev, Uppsala University, 2021