Geomagnetic map is an important factor affecting the performance of geomagnetic matching navigation. The random errors of the geomagnetic map seriously affect the accuracy of the matching and positioning, even lead to mismatching. In order to improve the geomagnetic matching performance, a method for modeling and compensating the random errors of geomagnetic data is proposed. This method establishes a non-stationary time series model of the data based on the analysis of the random error characteristics of the geomagnetic map data, uses Kalman filter, which the state equation is the times series model and the measurement is the real-time data, to filter the geomagnetic map data and compensate the random error. The effectiveness of the filtering method is indirectly proved through the navigation and positioning experiments based on the geomagnetic data before and after filtering. The actual geomagnetic map data processing results show that the geomagnetic map data filtered by the method in this paper can improve the positioning accuracy by 54. 7% when it’s used for geomagnetic navigation.