The common feature selection methods utilize the entire region of the sample space to extract an optimal subset of features. In contrast, this paper proposes a new local feature selection method, in which each region of the sample space is associated with a different optimal feature set, which can optimally adapt to the local variation of the sample space. At the same time, when solving the subspace corresponding to the optimal feature set, based on the concept of the nearest neighbor, this paper proposes a method to measure the similarity between the test data and each class to classify the test samples. The method proposed in this paper can be expressed as a linear programming optimization problem, so the global optimal solution can be solved by simple convex optimization. The experimental results on three real datasets and three mainstream methods demonstrate the feasibility and effectiveness of the proposed algorithm.