Aiming at the problem that the imaging features are inconspicuous and the proportion of objects is small in the optical remote sensing small ship detection under the complex marine scenes, such as sea-lean boundary and near-shore rocky reefs, an improved small ship detection method based on YOLOv8s is proposed. Firstly, the prediction layers are modified based on the introduction of shallow feature maps in the neck layers, which balances the weights of shallow locational information and deep semantic information, and enhances the attention of the model to small objects. Secondly, the C2f-FE module is adopted to utilize the channel grouping and the cross-channel information interactions, enhance the feature extraction of small ships, and reduce the model parameters, which merges the FasterNet Block and the efficient multi-scale attention mechanism. Finally, the dynamic detection head module is employed to improve detection capability of the model on different spatial scales and object tasks at different prediction layers. The experimental results show that compared with the original YOLOv8s model, the improved model reduces the number of parameters by 42.3%, the detection accuracy mAP50 and mAP50:95 values are improved by 4.2% and 2.2% on the MASATI dataset, and mAP50:95 values are improved by 1.7% and 1.4% on the DOTA-Ship and DOTA-Small Vehicle datasets, respectively. It can be concluded that the improved model not only achieves lightweight and accurate detection of small ships, but also satisfies the high-accuracy detection for the generalized of small objects in remote sensing scenarios.