Human Foot Keypoint Dataset

Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, Yaser Sheikh

Carnegie Mellon University

Foot examples


Existing human pose datasets contain limited body part types. For both MPII and MS-COCO datasets, foot annotations are limited to ankle position only. However, graphics applications such as avatar retargeting or 3D human shape reconstruction require foot keypoints such as big toe and heel. Without foot keypoint information, these approaches suffer from problems such as the candy wrapper effect, floor penetration, and foot skate. To address these issues, a subset of about 15K human foot instances has been labeled using the Clickworker platform. The dataset is obtained out of the over 100K person annotation instances available in the COCO dataset. It is split up with 14K annotations from the COCO training set and 545 from the validation set. A total of 6 foot keypoints have been labeled, consisting of the big toe, small toe, and heel for each foot (see the figure below). We consider the 3D coordinate of the foot keypoints rather than the surface position. For instance, for the exact toe positions, we label the area between the connection of the nail and skin, and also take depth into consideration by labeling the center of the toe rather than the surface.

The foot keypoint ordering in the annotation file is as follows:

  1. Left big toe.
  2. Left small toe.
  3. Left heel.
  4. Right big toe.
  5. Right small toe.
  6. Right heel.


Foot Keypoint Annotations (Training: ~13.5k annotations, Validation: ~0.5k annotations)
Download the JSON zip file.
Download the JSON zip file.
Download the image dataset from the MS-COCO website (in particular 2017 Train images [118K/18GB] and 2017 Val images [5K/1GB])
Trained foot detector models and testing code released in OpenPose

The JSON files follow standard COCO format, analogously to 2017 Train/Val annotations [241MB] from the MS-COCO website.


Foot examples .


Please cite the following paper (arXiv link) in your publications if the dataset helps your research.
    author = {Zhe Cao and Gines Hidalgo and Tomas Simon and Shih-En Wei and Yaser Sheikh},
    booktitle = {arXiv preprint arXiv:1812.08008},
    title = {Open{P}ose: realtime multi-person 2{D} pose estimation using {P}art {A}ffinity {F}ields},
    year = {2018}


The dataset follows OpenPose license, it is freely available for non-commercial use. Please, see the license for further details. For commercial queries, use the Directly Contact Organization section from the FlintBox link and also send a copy of that message to Yaser Sheikh.