Assessment of Organizational Evaluation Capacity

Research

Assessment of Organizational Evaluation Capacity

Organizational Evaluation Capacity (EC) can be generally defined as an organization’s visible, enacted evaluation practices and processes. In a nutshell, organizations with a high level of evaluation capacity conduct quality evaluation studies in a timely manner, use evaluation findings regularly as part of program and organizational decision-making, and adopt an evaluative culture founded on the principles of organizational learning.

The Organizational Evaluation Capacity Assessment Instrument is based on a framework of organizational evaluation capacity developed by Bourgeois and Cousins (2013). The instrument is meant to highlight organizational evaluation capacity in six main dimensions, which constitute the main sections of the instrument (divided into “capacity to do” and “capacity to use” evaluation) Although the framework suggests that an organization must first develop its capacity to do evaluation in order to foster use (push model), the opposite can also be true, where an interest in using evaluation prompts an organization to increase its capacity to produce it (pull model).

One instrument should be completed per organization. In many instances, the person completing the instrument may need to draw on the knowledge and opinions of other organizational members. This can include (but is not limited to): senior executives, program managers, corporate staff, etc. These individuals can be consulted in a number of different ways, depending on the individual and the situation (e.g., formal or informal interviews, departmental evaluation committee meetings, written communications, etc.). We encourage you to complete it in a workshop format, where the group can discuss the ratings to be given to each item. We have often found that these discussions end up being as valuable as the completed instrument results.

Our current research projects in this line of investigation focus on various aspects of evaluation capacity:

  • Psychometric validation of the Organizational Evaluation Capacity Assessment Instrument: This project, led by Jessica Deschênes Cervantes, seeks to establish the psychometric properties of the instrument. Data collected previously will be used to confirm the instrument’s structure and to identify needed modifications to items.
  • Longitudinal analysis of instrument data: This project will be led by Léandre Leblanc and will examine the evolution of evaluation capacity in Ontario public health units since 2014.
  • Needs assessment in Ontario public health units: We are currently participating in a project led by the University of Waterloo. Isabelle Bourgeois and David Buetti are both part of this project team and will use a modified version of the Organizational Evaluation Capacity Assessment Instrument to measure evaluation capacity specific to chronic disease prevention programs.