In recent years, it’s become popular on social media to share photos of meals but who would’ve guessed that these foodie photos would offer a more than a tantalizing tease to followers and maybe free advertising for restaurants?
Now, research from CSAIL, the Computer Science and Artificial Intelligence Laboratory at MIT, has produced an AI system that can analyze these photos to help track nutrition, predict recipe ingredients, and even offer similar recipes to the meals pictured.
Although previous similar systems have been created before, the food information databases that the algorithms could draw from were too small to be very helpful. The Swiss created ‘Food-101’ recognized food in photos with only 50 percent accuracy until its databanks were greatly increased, which improved the system’s accuracy to near 80 percent. A similar project at Hong Kong’s City University has an impressively large dataset, with 65,000 recipes and more than 110,000 pictures, but it is restricted to Chinese cuisine at this point.
The new system, designed by the CSAIl team in collaboration with Qatar Computing Research Institute, or QCRI, is called Pic2Recipe and is inspired by the work of those previous projects. Their first step was to build their database of more than 1 million recipes, which they named ‘Recipe 1M’ and built through a combination of websites like Food.com and All Recipes. Once all the entries were annotated, researchers then trained their neural network to connect images with recipes and ingredients.
When tested Pics2Recipe did very well at identifying photos of muffins and cookies but struggled when food was not as clear or had mixed in components, like smoothies or sushi. Additionally, the program ran into issues when there was more than one recipe for a single dish. As an example, there are numerous recipes for something like lasagna and the pictures in general will look very similar.
However, researchers are still working to improve the system with abilities like distinguishing between methods of cooking and preparation or telling one type of mushroom from another. Ultimately they’d like Pics2Recipe to become a meal and nutritional aid providing fridge inventories, dietary preferences, as well as nutritional information when labels are lacking.
The paper with the complete findings will be presented at the Computer Vision and Pattern Recognition conference this month.
Here is the Online Demo.
More News to Read
- New ESA Mission: Manmade Spacecraft to Orbit the Sun and Detect Gravitational Waves
- UM Solar Car Race Team Wants to Win The American Solar Challenge With Novum
- Could Bacteria-Coated Nanofiber Electrodes be Key to Cleaning Polluted Water?
- First Earth to Orbit Quantum Teleportation Completed
- Quantum Particle Discovery Destroys Previous Results
