Nonlinear elements cause significant modeling errors and control delays in speaker control processes, which affect the precise control of the speaker voice coil’s motion. This not only improves sound quality but also reduces mechanical wear and aging. This paper addresses the problems of modeling errors and control delays in the fine control of the speaker’s voice coil. We design a backstepping sliding mode controller based on an improved RBF-MLP neural network, solving the issues of control interference caused by nonlinear elements in electric speakers and the insufficient accuracy of the classical RBF network in fitting complex nonlinear models. By introducing perception layers, adaptive learning mechanisms, and generalized radial basis function, the improved RBF-MLP network reduces the mean squared error of nonlinear function fitting by more than 5% compared to the classical network, enhancing its ability to capture complex nonlinear characteristics of the speaker system and improving model fitting accuracy. A simulation environment was built to evaluate the control performance of the speaker system under different frequency, amplitude, and load conditions, focusing on control precision, system delay, and chattering problems. The experimental results show that under varying frequency and load conditions, the control delay is reduced to an average of 0.15 ms, and control errors are decreased by 39%. Furthermore, the improved control method maintains excellent robustness and stability under complex load and frequency variations. These results demonstrate the broad application potential of the improved controller in electric speaker control systems.