Aiming at the problem that traditional gait recognition methods based on silhouette are limited by the ability to extract input features and model features, which leads to low recognition accuracy, an improved GaitSet method for gait recognition via fusion of silhouette enhancement and attention mechanism is proposed. Firstly, the outline of the pedestrian is obtained by preprocessing, and its average value is obtained. Then the GEI energy diagram is synthesized, which is used as the input feature of the neural network model to enhance the representation of human appearance. Secondly, the attention mechanism is introduced in the process of feature extraction to enhance the feature extraction ability of the model, so as to improve the accuracy of gait recognition. Finally, experiments are carried out on the CASIA-B and OU-MVLP benchmark data sets, and the average Rank-1 accuracy of the proposed method is 87.7% and 88.1%, respectively. Especially under the most complex walking conditions with overcoat, compared with GaitSetv2 algorithm, the accuracy is improved by 6.7%, indicating that the proposed method has stronger accuracy. Notably, the proposed innovative method adds almost no additional parameter number, computational complexity, and inference time, which proves the rapidity of its individual modules.