The image feature is an important branch of texture feature, which reflects the different images and object shape, size, distribution, direction, and other important parameters and plays a decisive factor on image characteristics recognition. But the texture feature extraction process is very complex and time cost. In order to solve the problem, a new method to extract texture feature based on FPGA is implemented. First, the texture feature extraction method is optimized with parallel algorithm, then the error is analyzed and controlled based on numerical range and representation accuracy, so the method can operate on FPGA efficiently. Also, a method to improve the data stream transmission on FPGA is designed, which employs pipeline optimization on main modules and register allocation model. The system on FPGA can modify parameters online to adapt for different environmental variables, such as image size, convolution kernel and so on. The results show that the proposed model extracts image texture feature up to ten times faster than CPU under the same power consumption, and it is an ideal system to fast extract image texture feature based on FPGA.