In order to solve the problems of low detection accuracy and high complexity of existing models due to the variety of defect types, significant size differences, and high complexity of existing models in the detection of steel surface defects, a lightweight detection algorithm YOLOv8n-CSG with improved YOLOv8n was proposed. Firstly, the design of the CG Block module was introduced C2f_CG which enhanced the ability to capture the surrounding features and enhance the information relevance. Secondly, a C2f_Star module is designed by adding the Star Block module, which maps the input data to the high-dimensional nonlinear feature space and generates rich feature representations, which makes the model more effective in dealing with subtle defects. Finally, a lightweight detector GSE_Detect integrating GSConv and EMA attention mechanisms was designed to maintain the high efficiency of the original detector and reduce the complexity. Multiple sets of experiments on the NEU-DET dataset show that the improved YOLOv8n-CSG network model mAP@0.5 reaches 76.8%, compared with YOLOv8n, mAP@0.5 is improved by 6.9%, the accuracy is increased by 11.3%, the calculation cost is reduced by 37%, and the parameter quantity is reduced by 35.2%, showing a better detection ability for steel surface defects, and balancing the performance and complexity of the model.