As an important protection equipment in distribution system, the fault diagnosis of universal circuit breaker (ACB) is very important for the stable operation of power system. However, the traditional single-modal model cannot fully describe the characteristics of the data when extracting features, resulting in a decrease in the accuracy of fault diagnosis. To solve this problem, this paper proposes an improved osprey algorithm to optimize the gated recurrent unit-graham angle and field-recurrence plot-vision transformer (GRU-GASF-RP-ViT) universal circuit breaker fault diagnosis model. The model combines one-dimensional signal and two-dimensional image features to describe the characteristics of the data more comprehensively from the perspective of time series and space. The accuracy of fault classification and recognition is improved. Firstly, the one-dimensional vibration signal is converted into two sets of two-dimensional images by GASF and RP respectively. Then, the two-branch ViT is used to effectively learn the spatial and local features of the two sets of two-dimensional images. The other branch captures the dynamic changes and trends in the one-dimensional time series signal through the GRU, and realizes the parallel combination of GRU and the new two-branch ViT. For the hyperparameters that are difficult to determine in the model. The improved osprey algorithm is introduced to optimize the parameters to make the model more reasonable. Finally, a circuit breaker fault simulation experiment platform is built. By comparing with the other four models, the accuracy rate has increased by 3.3%~13.3%, it is verified that the proposed model has higher diagnostic accuracy.