To address the challenges of detecting multi-scale and deformed road defects in complex road scenarios, an improved YOLOv8n model for road defect detection, named DMS-YOLO, is proposed. First, an adaptive context-aware feature pyramid network is designed to achieve global fusion and dynamic weighting of multi-scale features, significantly enhancing the model’s ability to perceive and express complex defects. Compared to existing mainstream feature pyramid networks, this approach demonstrates clear advantages in both accuracy and computational efficiency. Second, an adaptive multi-scale dynamic detection head is introduced, leveraging deformable convolution (DCNv3) to improve the model’s capability in capturing complex shape features, and a Collaborative Attention Mechanism is designed to integrate scale and task attention, enhancing the model’s understanding of multi-scale information. Finally, the CIoU loss function is improved using the Focaler-IoU idea to enhance the detection of small targets. Experimental results show that, with reduced computational cost, the DMS-YOLO model achieves a mAP@0.5 of 87.9% on the RDD2022 dataset, a 3% improvement over the baseline model. The model has 3.67×106 parameters, 8 GFLOPs of computational cost, and a model size of only 7.3 MB, demonstrating its lightweight nature and ease of deployment. Additionally, on the SVRDD dataset, DMS-YOLO improves on all performance metrics, further validating the model’s generalization and robustness. Compared to other mainstream models and state-of-the-art detection algorithms, DMS-YOLO shows superior overall performance, demonstrating its practical application value in road defect detection.