Assessment of Organizational Evaluation Capacity


Assessment of Organizational Evaluation Capacity

Organizational Evaluation Capacity (EC) can be generally defined as an organization’s visible, enacted evaluation practices and processes. In a nutshell, organizations with a high level of evaluation capacity conduct quality evaluation studies in a timely manner, use evaluation findings regularly as part of program and organizational decision-making, and adopt an evaluative culture founded on the principles of organizational learning.

The Organizational Evaluation Capacity Assessment Instrument is based on a framework of organizational evaluation capacity developed by Bourgeois and Cousins (2013). The instrument is meant to highlight organizational evaluation capacity in six main dimensions, which constitute the main sections of the instrument (divided into “capacity to do” and “capacity to use” evaluation) Although the framework suggests that an organization must first develop its capacity to do evaluation in order to foster use (push model), the opposite can also be true, where an interest in using evaluation prompts an organization to increase its capacity to produce it (pull model).

One instrument should be completed per organization. In many instances, the person completing the instrument may need to draw on the knowledge and opinions of other organizational members. This can include (but is not limited to): senior executives, program managers, corporate staff, etc. These individuals can be consulted in a number of different ways, depending on the individual and the situation (e.g., formal or informal interviews, departmental evaluation committee meetings, written communications, etc.). We encourage you to complete it in a workshop format, where the group can discuss the ratings to be given to each item. We have often found that these discussions end up being as valuable as the completed instrument results.

Our current research projects in this line of investigation focus on various aspects of evaluation capacity:

  • Measurement of organizational evaluation capacity in public health: This project, conducted between 2014 and 2018, sought to adapt the capacity assessment instrument in order to measure organizational evaluation capacity in Ontario public health units. This project also enabled us to develop and test various evaluation capacity building strategies in participating health units.
  • Psychometric validation of the Organizational Evaluation Capacity Assessment Instrument: This project seeks to establish the psychometric properties of the instrument. Data collected previously will be used to confirm the instrument’s structure and to identify needed modifications to items.