In 2014, the Netherlands Environmental Assessment Agency (Planbureau voor de Leefomgeving, PBL) commissioned the Athena Institute to contribute to the process design of the evaluation of the Natuurpact agreement and to review its value for nature policy processes. In the Natuurpact, national government and the provinces agree on the decentralisation of nature policy and on broadening the ambition of nature policy beyond conserving biodiversity to also include increasing societal engagement with nature. In light of this, PBL required expertise on learning processes and reflexive monitoring in complex multi-level governance and multi-stakeholder contexts. This expertise was found at the Athena Institute. The first three-year evaluation period was concluded in 2017, and the project has continued on to a second evaluation period, for which the Athena Institute is once more contributing to the process design.
The Athena Institute adopted an emergent, responsive and participatory research approach to support the researchers (from PBL and partner Wageningen University and Research) in evaluating nature policy, emphasizing the facilitation of interactive reflection and learning by the provinces, national government and societal partners. Athena researchers met with the project team bi-weekly. Interventions included:
- organising participatory workshops to gather learning needs of governments and societal partners to establish alignment between the evaluation research and nature policy practice
- contributing to interview designs that facilitate reflection and learning
- contributing to the design of group review sessions for joint interpretation and shared meaning
- facilitating interactive reflection by the project team
- reviewing the evaluation process and its manifested values by interviewing policymakers and evaluation researchers
An article on our research was selected for a special issue of the journal Environmental Innovation and Societal Transitions, and two reports were published as a result of our research: a literature review on characteristics of reflexive evaluation (Van Veen, et al., 2016) and a review of the value of the reflexive evaluation of Natuurpact (Verwoerd, et al., 2017). The review demonstrated that the reflexive evaluation has value for policy practice on five different levels: network, affective, conceptual, instrumental and strategic. Based on the review, six recommendations were formulated to inform the process design of the second evaluation period. These recommendations included further increasing stakeholder diversity, aligning evaluation research and policy practice, supporting the learning policy network, balancing learning and accountability, embedding knowledge integration, and developing more shared understanding of the role and goal of reflexive evaluation.
The action-oriented character of the study ensured that observations and recommendations could be readily implemented by the evaluation researchers. Also, the findings from the first review were used to inform the process design of the following evaluation period. We have found that this way of evaluating a policy program was highly appreciated by the end users of the evaluation. The evaluation process led to increased understanding of different perspective and network-building, and the evaluation report (Folkert, et al., 2017) was highly instrumental in the policy development process at provincial level. As a result of our contribution to the evaluation process, we were asked to share our insights through a presentation to the Commission on Policy Quality and Evaluation (Beleidskwaliteit- en Evaluatiecommissie, BEC) of the Ministry of Economic Affairs. We were also asked to provide training on reflexive evaluation for researchers at WUR in the context of a Community of Practice on Policy Evaluation.