In the context of time-varying operating conditions, fault diagnosis often exhibits high dynamism, while the limited model learning under small samples makes the issue more challenging. For the above situation, a fault diagnosis method based on contrastive deep convolutional networks is proposed. Firstly, considering the characteristic of small data samples, take advantage of differences in vibration data distribution caused by speed changes, and naturally realize data enhancement without manual operation. Subsequently, in the process of data processing, the vibration data of the same healthy state at different rotational speeds are used as positive samples, while the vibration data from different health states are used as negative samples. The key features are extracted by comparing the similarity between the samples so as to reduce the distance between the positive samples while increasing the distance between the negative samples. Finally, the feature extractor is trained and optimized by comparative training method, where a weighted combination of contrastive loss and cross-entropy loss is used as the composite loss function, enabling the model to effectively perform classification tasks while learning feature representations. The method is applied to two different bearing failure datasets at different time-varying rotational speeds for case studies respectively. The experimental results show that the proposed model not only performs well in the feature extraction and classification tasks, but also realizes high accuracy fault diagnosis under both data scarcity and time-varying speed conditions. It is verified that the proposed model shows high feasibility and effectiveness in dealing with time-varying small-sample data, and outperforms other advanced diagnostic methods.