To address the problem of insufficient grasp detection accuracy of existing grasp detection methods in complex unstructured grasping scenarios due to the conflict of angle training labels and the non-consistency between graspable regions and object regions, this paper proposed an adaptive feature fusion grasp detection network, AFFGD-Net. The network firstly adopted the angle prediction method based on the partition method, which encoded the angle values into two parts, namely, angle category and offset for learning and prediction. The conflict angle values were divided into the same category to reduce the conflict of angle training labels, and the offset was used to compensate for the loss of accuracy in the classification part to improve the prediction accuracy of the network for grasp angle. Secondly, the adaptive receptive field block ARFB and attention skip connection module ASCM are introduced. ARFB enhanced the network’s ability to characterise the features of multi-scale graspable regions, and improved the grasp detection ability of multi-scale objects by adaptively fusing features of different scales. ASCM recovered the edge features of the graspable regions by adaptively fusing the low-level spatial features and the high-level semantic features, which improved the network’s grasp angle and grasp width prediction accuracy. Finally, the effectiveness of the proposed network was verified by experiments. The accuracy of AFFGD-Net reached 98.9% and 97.7% in the image segmentation and object segmentation test modes in the Cornell dataset, respectively, and 95.2% in the Jacquard dataset. The detection speed of the network reached 111 FPS, which showed good real-time performance. The experimental results showed that AFFGD-Net outperformed the existing methods in terms of both accuracy and real-time crawl detection, confirming the effectiveness of the proposed method.