A UK-based team of researchers and practitioners developed an evaluation toolbox for local food strategies.
The process of developing the toolbox proved as important as the final result.
It is flexible enough to be able to work without the “perfect” data but still inspire action.
A UK-based team gathering a researcher and practitioners (Cardiff University, UK Sustainable Food Cities network) developed a toolbox that captures cities’ progress towards sustainable food. Their work shows that evaluation is not only about gathering data: it also means building a common narrative that inspires action.
Your city’s action in a snapshot
The team worked with around a hundred practitioners from 41 different places. They then applied the toolbox in Cardiff and Bristol.
The toolbox covers four dimensions (governance, health and well-being, economy, environment). For each dimension, stakeholders identified :
- A goal, i.e. an overarching aim (for example “healthy cities”);
- A desired outcome, i.e. a state or position which is reached in order that the goal is achieved (for instance, low incidence of diet-related illnesses);
- An indicator that can be used to measure progress towards delivery (for example, the decrease in the incidence of diet-related illnesses), and
- Activity-based indicators,e. activities that can potentially contribute to improve indicators (for instance, the increase in portions of vegetables in school meals). These activities were classified as follows: partnership & collaboration, policies & strategies, infrastructure & planning, public services & support, knowledge & awareness, and market-based mechanisms.
This systematic mapping of activities allows the identification of key gaps in action. For example, in Cardiff, it showed that the city had not fully embraced levers such as market-based instruments or infrastructure planning.
Linking activities to impacts
A key challenge in the elaboration of the toolbox was to establish formal links between actions developed at the local level (activity-based indicators) and impacts on the food system (the overarching goal). In other words, it is always difficult to attribute causality.
The team carried out an extensive literature review to check whether activities could be linked to impacts. The toolbox presents actions that have proven to support the objective. For instance, mapping local food assets (such as green spaces and brownfield sites that can be used by food businesses) contributes to support the local food economy. However, Ana Moragues-Faus, the article’s lead author, was surprised by the general lack of systematic analyses on the impact of sustainable food initiatives.
Therefore, the toolbox cannot say that the actions it presents are enough to achieve the objectives (see our previous article “We have done a lot, but what have we accomplished? Lessons from New York urban food policy”). It can, however, trigger a conversation about why the city cannot see the impacts of its programs. This can be due to other negative trends that are happening at the same time. For instance, great efforts to introduce organic food in schools can be counterbalanced by an increase in the number of unhealthy outlets in the city, or by the fact that parents do not have the time to cook when kids are at home.
Beyond data
During the whole process, data collection proved particularly challenging, as cities do not always collect the perfect indicator that nicely fits into the assessment framework, and it is difficult to devote funds to collect new data. Indeed, data collection is often a limitation in processes aiming to capture the impact of local food action (see our article Evaluating the economic impacts of local food systems).
In the UK, the team decided to go around this difficulty through secondary data, interviews with local actors, and also by selecting indicators that are easy to measure and accessible. Ana Moragues-Faus is grateful of how people shared their knowledge about the local food system, although she thinks the project could have gone further in collecting the diversity of voices. This qualitative information proves extremely useful to avoid being trapped in narrow indicators. For instance, only looking at the tonnage of food waste can be misleading: indeed, if fruit and vegetable consumption increases, then food waste is bound to increase as well, but that is not necessarily bad news. More qualitative approaches can qualify such numbers.
Moreover, the researcher stresses that the toolbox they came up with is flexible enough to work without the “perfect data”. The mere fact of mapping existing actions and using this to connect stakeholders is already a major step forward for cities.
The process is as important as the toolbox itself
Indeed, another key takeaway from this research is that the process of developing the toolbox proved to be as important as the toolbox itself.
First, because it made actors spell out what the objectives of a sustainable food policy should be, and agree upon these goals. For instance, in the realm of health and well-being, the goal they decided upon was: “Improving physical and mental health and wellbeing by reducing food poverty; improving access to culturally acceptable, affordable healthy food for all; promoting healthy diets; and increasing participation in food related physical and social activity”. This is key to make ideas travel from one organisation to the other, and help each actor align its work with that of its counterparts.
This is why, according to Ana Moragues-Faus, cities should ask themselves what they expect from such an evaluation toolbox when they start developing one. If inspiring action and connecting stakeholders is their main objective, then it might be worth investing in the process and not so much in the data collection. If they want to compare their action to their international counterparts, then it will be important to align with the indicators that are being used in international toolboxes provided by the FAO Food-for-Cities (see our article The A to Z of urban food analysis) or the Milan Food Policy Pact.
Indeed, the benefits of developing an evaluation toolbox are manifold: making your city’s action more visible, shaping our understanding of the problems, enabling comparison, supporting decision-making, evaluating and predicting trends. Provided cities are clear about why they are developing a toolbox, it will prove an extremely useful base for developing integrated food policies.
Albane GASPARD
January 2020
Urban Food Futures would like to thank Ana Moragues-Faus for her inputs and comments.
Picture credits: Pixabay
Going further
“We’ve done a lot, but what have we accomplished?” Lessons from New York urban food policy
How much should food sustainability rely on indicators?
Quick-scanning your food policy