Flawed Food Compass metric could misguide tax and investment decisions

Flaws in a new food metric could hinder public health efforts and misguide tax and investment decisions across the US and other markets, a global team of scientists has found.

They are concerned about the Food Compass Nutrient Profiling system, published in Nature Food in October. It rates the cereal Frosted Mini-Wheats and chocolate-covered almonds in the highest score bracket, “to be encouraged”, while relegates millet, whole wheat bread, boiled eggs and skinless chicken breast in the lower bracket “to be moderated”.

The system could potentially displace nutritious foods in low- and middle-income country markets, to promote less healthy, ultra-processed products in favour of local items, the scientists say.

In video

Dr Ty Beal discusses his concerns on Diet Doctor

Dr Ty Beal, Research Advisor at the Global Alliance for Improved Nutrition (GAIN), led a critique of the profiling system. His work was declined for publication by Nature Food and the Food Compass authors have not replied to his queries. The editor did invite Ty Beal and his co-authors to submit a broader critique of Nutrient Profiling Systems, which they plan to do.

“The possibility of an open scientific discussion has been shut down,” he says. “That’s concerning, particularly when there was no response from the authors.

“Just adding some nuts on top of an ice cream cone is not going to make that somehow healthier”

“According to Food Compass, you add nuts to an ice cream cone and somehow it’s a whole category above the score of some minimally processed, animal-sourced foods.

“To me that should reveal that there is something wrong with the algorithm, that it’s not sufficiently penalising this high sugar, high fat, low fibre, ice cream cone. Just adding some nuts on top is not going to make that somehow healthier.

“The table weighting ultra-processed, minimally processed, and added ingredients describes a certain set of values, and then the footnote has a different weighting. This is where we saw some errors in the paper.

“I spoke with the group who created the NOVA classification system [grouping foods according to the industrial processes they undergo] on how these misclassifications occurred, and confirmed with them that many of the processed foods were wrongly classified.

“An example of this, freshly cooked dark leafy greens in vegetable oil were classified by Food Compass as ultra-processed, which is not the correct classification.

“There seems to be a favouring of big industry, highly-processed products”

“That seems like something that needs to be taken seriously.

“We found that the system presents itself as very sophisticated and overcoming the barriers of reductionist nutrient profiling systems.

“But when the components get aggregated there’s an imbalance in what takes precedence. There’s a negative penalty for ultra-processing in the score, but it’s a tiny fraction of the overall score, dwarfed by all the other components.”

He is concerned by the potential impact of misguidance from the algorithm. Ty Beal says, “It’s thought that Food Compass will influence taxation, company ratings, investment decisions and ultimately there may be intentions to use this is in a more global context.

“There seems to be a favouring of a lot of big industry products for some highly processed foods. There is a lot of effort from industry to retain the dominance of these foods as staples in society.

“It may give multinationals free rein to try and displace local, traditional foods”

“It doesn’t really do enough to combat some of those foods which can be problematic.

“The risk is that multinationals are encouraged to keep producing their ultra-processed products and have them labelled with a stamp of approval.

“If you think about this in low- and middle-income countries, it may give multinationals free rein to go in and try to displace the more local traditional foods.

“That’s a concern for health and for food sovereignty and culture, which we should push back against.

“If you look at the multinationals, their control of what foods are consumed, it’s already an uphill battle for small and medium sized businesses to compete.

“It’s hard to not feel as though vested interests are having an influence”

“I hate to see extremes, because in general around these topics there’s an exaggeration that happens on both sides and there needs to be nuance and transparency.

“But it’s hard to not feel as though vested interests are having an influence. There is a lot of industry funding that could benefit from this type of a rating.”

Food Compass was proposed as the first major nutrient profiling system to score across food groups, allowing mixed dishes to be rated, for example rating a pizza rather than the individual components, such as cheese, tomato paste and flour.

This mixed scoring helps companies rate their portfolio of foods.

Co-author Renata Micha, who worked on Food Compass as a faculty member at the Friedman School, has said the rating system can guide consumer behaviour, nutrition policy, scientific research, food industry practices, and socially based investment decisions.

“It feels like the prestige of the authors is preventing a public discussion”

The study is part of the Food-PRICE (Policy Review and Intervention Cost-Effectiveness) project, a National Institutes of Health-funded research collaboration looking at cost-effective nutrition strategies in the US.

Ty Beal says, “It feels like the prestige of the authors is preventing a public discussion about this.

“The editor of Nature Food, on declining to publish our critique, suggested reaching out to the authors again. Firstly, they are not replying. Secondly, I don’t want a private discussion, I want a scientific public discussion about this.

“There are differences of perspective which the public should be able to view so they can make their own decisions.

“Myself and my colleagues have devoted a lot of time to review the methods and write this up for the Matters Arising section of Nature Food. We did it because we were concerned and wanted to raise attention.

“Some donors have reached out, they’re not happy with Food Compass”

“I’ve had colleagues reach out who were considering using Food Compass and are not really happy with it. They are interested in alternatives.

“I think it’s telling. Even though our critique has not been published in an academic journal at this point, our concerns are shared by many. I’ve even had donors reach out to me, which I think is important. If donors wanted some sort of food rating and they’re not happy with Food Compass that’s a sign that our message has been received.”

Ty Beal and his team found the classification and scoring of ultra-processing was incorrect for many foods – listing them as receiving 100 – the top score.

“So clearly the algorithm says it penalises ultra-processing but there are several examples where it doesn’t,” he explains.

Ultra-processed and minimally-processed food choices might have the same calorie content, but the ultra-processed choice is likely to induce over-eating because it is addictive and doesn’t trigger a sense of feeling full. Most ultra-processed food choices are dangerous to public health, as obesity causes early deaths through non-communicable diseases such as diabetes, heart attack, stroke and cancer.

“In the US about 20 per cent of women aged 15 to 49 are iron deficient”

People who die from obesity in the US are, on average, 15.4 years younger than those without obesity.

Ty Beal adds, “The lack of attention to nutrient density is problematic, particularly of nutrients connected to bigger public health issues, like iron for example. In the US about 20 per cent of women aged 15 to 49 are iron deficient.

“Food Compass doesn’t pay a lot of attention to that. The consequence of not incorporating it in the scoring is one thing in the US. But global scoring encompasses markets where nutrient deficiencies are much more common, stunting and other under-nutrition indicators are really a high burden. It could discourage people from consuming foods that would improve their health a lot.

“There are a lot of ways the Food Compass could help in the US, promoting fruits, vegetables and pulses, which are allocated high scores.

“But there are a lot of foods in those high score categories as well that may lead to a higher risk of non-communicable diseases.

“Really sugary cereals got top scores. They include 12 grams of added sugar per small serving size”

“I’m not of the opinion that all ultra-processed foods are harmful, but there’s a general trend. Ultra-processed foods tends to be hyper palatable, they send a strong reward signal to the brain making it easy to over eat.

“Really sugary cereals, Frosted Mini-Wheats and Honey Nut Cheerios were getting the top scores. They include 12 grams of added sugar per small serving size. They have whole grains, but these are reconstituted, the food matrix degraded, and they’re packaged together in a sweet and appealing food that makes minimally processed foods potentially less appealing.

“There are added nutrients. But from my perspective they weren’t really scrutinised or penalised for some of the negative aspects.

“If something had nuts it always scored well. Items without any sort of food matrix like orange juice with added calcium were awarded a top score.

“Foods like millet which is minimally processed or whole wheat bread without any added sweeteners were ranked in the next score down.

“A highly processed, fried egg substitute scores better than the egg itself”

“There’s also a trend where you see animal-sourced foods scoring poorly because of the way the algorithm was set up. Skinless chicken breasts scored “moderate”, the same as Lucky Charms, a very sugary cereal.

“Foods like whole milk received a low score, eggs scoring lower even than Lucky Charms. A highly processed egg substitute fried in vegetable oil scores better than the egg itself.

“Almond M&Ms (“to be moderated”) scored above cheddar cheese or ground beef (“to be minimised”), these minimally processed foods.

“In the US we don’t need to be consuming more animal-sourced food. We don’t need to encourage unnecessary consumption. On the other hand, when you look at the Food Compass rating, it doesn’t seem like this is going to ultimately lead to what we need.

“Obesity is high in the US. A lot of these foods are appealing and easy to over-consume. So it misses the point of processing and the importance of satiety; consideration of which foods fill you up and keep you from eating an hour later – that type of thing.”

“Consumers, policy makers, are looking for simple tools to guide healthier choices”

The study’s lead author, Dariush Mozaffarian, dean of the Friedman School has said, “Once you get beyond ‘eat your veggies, avoid soda,’ the public is confused about how to identify healthier choices.

“Consumers, policy makers, and even industry are looking for simple tools to guide everyone toward healthier choices.”

The Food Compass system used a national database of 8,032 foods and beverages consumed by Americans. It scores 54 characteristics linked to major chronic diseases such as obesity, diabetes, cardiovascular problems, and cancer, as well as risk of undernutrition, especially for mothers, young children, and the elderly.

Across major food categories, the average Food Compass score was 43.2%

The lowest scoring category was snacks and sweet desserts – average score 16.4%
The highest scoring categories were vegetables – average score 69.1%, fruits – average score 73.9%, with nearly all raw fruits receiving a score of 100%, and legumes, nuts, and seeds – average score 78.6%
Among beverages, the average score ranged between 27.6% for sugar-sweetened sodas and energy drinks and 67% to 100% fruit or vegetable juices
Starchy vegetables scored an average of 43.2%
The average score for beef was 24.9%; for poultry, 42.67%; and for seafood, 67.0%

Anne Mullen, Chief Editor of Nature Food, told Quota, “Nature Food welcomes scientific discussion and debate regarding papers published in the journal and recognises the importance of post-publication commentary as necessary to advancing scientific discourse.

“For confidentiality reasons, we are unable to discuss the specifics of the editorial history or review process of submissions that have been made to the journal.

“However, we would like to make clear that whenever concerns are raised about any paper we have published, we look into them carefully, following an established process, consulting the authors and, where appropriate, seeking advice from peer reviewers and other external experts.

“Once such processes have concluded and we have the necessary information to make an informed decision, we will follow up with the response that is most appropriate (where necessary) and that provides clarity for our readers as to the outcome.”

Quota has invited the team at Food Compass to contribute to this article and will update the piece to include any additions from them. There is an Author Correction pending with the Nature Food paper – which the peer reviewers are involved in.