The graph-structured motion segmentation method (GS-Net) for geometric information learning is proposed to address the shortcomings of existing motion segmentation methods in terms of their practicality in traffic scenarios, and the difficulty in balancing performance and validation time. GS-Net consists of a point embedding module, a local context fusion module, a global bilateral regularization module, and a classification module. The point embedding module maps the original key feature point data from a low-dimensional linearly difficult-to-differentiate space to a high-dimensional linearly easy-to-differentiate space, which is conducive to the network learning the relationship between moving objects in the image; the local context fusion module utilizes the dual-branching graph structure to extract local information from both the feature space and the geometric space, and then fuses the two types of information to obtain a more powerful local feature representation, The global bilateral regularization module uses point-by-point and channel-by-channel global sensing to enhance the local feature representations obtained by the local context fusion module; the classification module maps the enhanced local feature representations back to the low-dimensional classification space for segmentation. GS-Net’s mean and median misclassification rates on the KT3DMoSeg dataset are 2.47% and 0.49%, respectively, which are 8.15% and 7.95% lower than those of SubspaceNet, and 7.2% and 0.57% lower than those of SUBSET. Meanwhile, GS-Net improves the network inference speed by two orders of magnitude compared to both SubspaceNet and SUBSET. GS-Net’s recall and F-measure on the FBMS dataset are 82.53% and 81.93%, respectively, showing improvements of 13.33% and 5.36% compared to SubspaceNet, and 9.66% and 3.71% compared to SUBSET, respectively. The experimental results demonstrate that GS-Net can quickly and accurately segment moving objects in real traffic scenes.