Accurate and efficient semantic description of human posture is an important part of human behavior recognition and a key step in understanding human behavior. In recent years, human key point detection technology has made significant progress, but most key point detection methods do not achieve semantic judgment of human posture. Therefore, a semantic description method of human posture based on geometric statistics is proposed. This method adopts the idea of set to improve the efficiency of attitude recognition and has strong robustness. Firstly, the obtained human key points are divided into several sets, and then the geometric distribution characteristics of each key point set are calculated to describe the human posture. Finally, tests were conducted on multiple real scene datasets. The experimental results show that the average accuracy rate of the proposed method for recognizing human posture on the PASCAL dataset reaches 77.1%, which is superior to other methods. In difficult situations such as occlusion and missing key points, it can still achieve high accuracy in human posture judgment.