When microwave RF chips are measured in the time domain, the broadband probe of digital oscilloscope is a key subsystem used to test the chip, and the transmission performance of the oscilloscope probe system must be clear for accurate measurement of the chip. So, the calibration system containing a grounded coplanar waveguide is designed to calibrate a broadband probe of digital oscilloscope. According to the signal transmission characteristics of the calibration system, the frequency response of the digital oscilloscope broadband probe system is obtained by deconvolution, and its bandwidth is calculated, which realizes the calibration of the probe system. The Keysight Infiniimax high-frequency differential active probe system is calibrated by this method, and the bandwidth value of about 12 GHz is in accordance with the nominal bandwidth of 12 GHz, indicating that the calibration system is reasonable and feasible.