Strong supervisory recognition algorithm requires a large amount of annotation information and consumes a lot of manpower and material resources. In order to solve the above problems and meet the practical requirements, two image recognition methods based on weak supervisory information are proposed for finegrained vision classification (FGVC). One is the combination of ResNet and Inception network, which improves the ability of capturing finegrained features by optimizing the network structure of convolutional neural network. The other is to improve the Bilinear CNN model, feature extractor selects Inceptionv3 module and Inceptionv4 module proposed by Google, and finally gathers different local features for classification. The experimental results on CUB200-2011 and Stanford Cars finegrained image datasets show that the proposed method achieves classification accuracy of 883% and 942% on the two data sets, and achieves better classification performance.