Looped Transformer & Test-Time Training
We want to study how lightweight test-time parameter updates interact with recurrent-depth computation in looped Transformers, aiming to jointly reduce operator mismatch and finite-iteration error.
01 — Research
Questions I’m interested in.
I’m a slow learner, but I keep learning and moving forward.
R / 01
Deep Learning
DL / Architecture / Recurrent Model
R / 02
Reinforcement Learning
RL / Theory / Bandits Theory
Beyond Scalar Leaderboards: Adaptive Sampling for Pareto Frontier Identification in Multi-Objective LLM Arenas
We study Pareto frontier identification algorithms for evaluating models from human preferences, especially in AI Arena, through binary pairwise-feedback bandits with the Bradley-Terry model and Borda-score successive elimination.
R / 03
Large Language Model
LLM / Interpretability / Chain-of-Thought Reasoning
CoT Reasoning Mechanism for Counting
We investigate how Chain-of-Thought improves LLMs' counting ability by tracing the mechanism from controlled synthetic tasks on GPT-2-like Transformers to realistic needle-in-a-haystack settings in open-weight large models.
R / 04
Diffusion Language Model
DLM / Interpretability / Latent Reasoning
DLM Reasoning Mechanism for Sudoku
We want to study how diffusion language models enable efficient planning and reasoning on tasks such as Sudoku, and how different adaptive decoding orders affect their reasoning behavior.