Clustering algorithm is one of the hot research technologies in digital recognition. It is a kind of unsupervised learning method to look for clustering classes with similar features. Clustering method is used to design handwritten number recognition system in VC ++6.0 environment. Through image segmentation and feature extraction, pattern similarity measurement and clustering analysis, the recognition work is completed. The results show that the recognition rate of the handwritten number recognition system based on the clustering recognition algorithm can reach 100%, which meets the practical application requirements.