Publications
Transformers as Provable Approximators of Sparse Principal Component Analysis
Qinyan Liu, Yan Chen, and Canhong Wen
STAI-X 2026 Strong Accept (with top review scores)
Paper (pdf) | Code | OpenReview
Extended the theoretical understanding of Transformers to fully unsupervised statistical learning, using Sparse Principal Component Analysis (SPCA, or sparse PCA) as a representative statistical task.
Established a constructive approximation framework showing that ReLU Transformers can emulate truncated power iterations, with error guarantees under a noisy spiked covariance model and necessary assumptions.
Derived high probability statistical error bounds under standard learning-theoretic assumptions and an additional restricted sparse eigengap condition, quantifying the expressiveness-capacity tradeoff.
Validated the method on high-dimensional genomic datasets and noisy synthetic datasets. Results demonstrate that unsupervised Transformers can achieve SPCA performance comparable to baselines and efficient batched inference.
Advisor: Prof. Canhong Wen, School of Management Sciences, USTC
Working papers
Hypergraph Generation from Hyperbolic Latent Embeddings
Advisor: Prof. Ji Zhu and Prof. Gongjun Xu, Department of Statistics, University of Michigan, Ann Arbor (UMich)
Mentor : Dr. Shihao Wu, incoming Assistant Professor at University of California, Davis (UCD)
Projects
Leveraging Zero-Inflated Data via Double Machine Learning
Advisor: Prof. Kan Xu, W. P. Carey School of Business, Arizona State University
- Analyzed zero-inflated and censored demand patterns from real-world online marketing data.
- Surveyed literature on Double Machine Learning and Doubly Robust Estimation, and built preliminary models to explore the latent structure in sparse click and purchase data.