Yao Lu

Yao Lu, Dengdong Fan, Shixun Zhang, Yonghong Tian

arXiv, 2026

We propose PowerStep, a memory-efficient optimizer that achieves coordinate-wise adaptivity without storing second-moment statistics. Across Transformer models from 124M to 235B parameters, PowerStep matches Adam's convergence speed while reducing optimizer memory by half—and by roughly $8\times$ with int8 quantization.

Yao Lu*, Dengdong Fan*, Jianzheng Nie*, Fan Xu, Jie Chen, Bin Zhou, Yonghong Tian (*equal contribution)

arXiv, 2026

We show superior results for reasoning LLMs can be achieved with offline RL, which is much more stable and efficient than online RL methods such as GRPO. All experiments were conducted on Huawei Ascend 910C NPUs.

Yao Lu, Si Wu

Neural Networks, 2024

We pointed out the problem of recurrent networks of only visible binary neurons in generating sequences and solved it by adding hidden neurons. A three-factor learning algorithm with theoretical guarantees is proposed to learn sequences as attractors for the networks with hidden neurons.

Yao Lu, Stephen Gould, Thalaiyasingam Ajanthan

Neural Networks, 2023

We theoretically solve the vanishing/exploding gradients problem in feedforward neural networks. Key idea: constraining signal propagation in both directions via a new class of activation functions and orthogonal weight matrices based on high-dimensional probability theory.

Yao Lu, Jack Valmadre, Heng Wang, Juho Kannala, Mehrtash Harandi, Philip Torr

IEEE Winter Conference on Applications of Computer Vision (WACV), 2020

We pointed out the problem of image warping in estimating optical flow and proposed the deformable cost volume to solve the problem.

Yao Lu, Jukka Corander, Zhirong Yang

Pattern Recognition Letters, 2019

We proposed a new method for data visualization, making it on spheres!

Yao Lu, Zhirong Yang, Juho Kannala, Samuel Kaski

International Joint Conference on Neural Networks (IJCNN), 2019

We proposed a new neural network module, Contrast Association Units, to model the relations between two sets of input variables.

Yao Lu, Mehrtash Harandi, Richard Hartley, Razvan Pascanu

ICML workshop on Modern Trends in Nonconvex Optimization for Machine Learning, 2018

We proposed a new matrix approximation method which allows efficient matrix inversion. We then applied the method to second order optimization algorithms for training neural networks.

Yao Lu

International Joint Conference on Artificial Intelligence (IJCAI), 2016

We found visual attributes of object classes can be unsuperisedly learned by applying Independent Component Analysis on the softmax outputs of a trained ConvNet. We showed such attributes can be useful for object recognition by performing zero-shot learning experiments on the ImageNet dataset of over 20,000 object classes.

Yao Lu, Kaizhu Huang, Cheng-Lin Liu

Pattern Recognition, 2016

We designed a fast graph matching algorithm with time complexity $O(n^3)$ per iteration, where $n$ is the size of a graph. We proved its convergence rate. It takes within 10 seconds to match two graphs of 1,000 nodes on a PC.

Yao Lu, Horst Bunke, Cheng-Lin Liu

Graph-Based Representations in Pattern Recognition (GbR), 2013

We designed a fast Maximum Common Subgraph (MCS) algorithm for Planar Triangulation Graphs. Its time complexity is $O(mnk)$, where $n$ is the size of one graph, $m$ is the size of the other graph and $k$ is the size of their MCS.

Yao Lu, Yuzuru Sato, Shun-ichi Amari

Neural Computation, 2011

We found a unique traveling bump solution, which was unknown to exist before, in a set of two-dimensional neural network equations.