Intelligent Interactive Systems Group at Harvard

Data Set for PlateMate



This data set includes 16 out of 18 images used in the evaluation of the accuracy of PlateMate. Included is also a spreadsheet providing ground truth nutritional information for a meal shown in each photo (as reported by a preparer or manufacturer) as well as PlateMate estimates and estimates by 3 expert nutritionists. We also include estimates provided by the MealSnap iPhone app (evaluated in April of 2011).

This data set accompanies the following paper:

Jon Noronha, Eric Hysen, Haoqi Zhang, and Krzysztof Z. Gajos. PlateMate: Crowdsourcing Nutrition Analysis from Food Photographs. In Proceedings of the 24th annual ACM symposium on User interface software and technology, UIST '11, pages 1-12, New York, NY, USA, 2011. ACM.
[Abstract, BibTeX, Authorizer, etc.]


Example of a meal photograph used in the evaluation of accuracy of PlateMate estimates

The data set is provided as a single .zip file. It contains a directory with 16 food photographs and an Excel spreadsheet containing the analysis results. Note that some photographs present complex meals: PlateMate divides the analysis of such meals into analyses of individual component foods. For example, with the photograph shown here, PlateMate will start by dividing the meal into three components: pulled chicken sandwich, broccoli and potato wedges. The Excel spreadsheet includes a separate column for each food shown in each of the photographs. The comparisons to other systems, however, are performed at the level of entire meals.


The primary contact person for this project is Krzysztof Gajos.

This page was last modified on April 26, 2022.