What is a monitoring and evaluation strategy?

A monitoring and evaluation strategy enables at a high level, an effective strategy to allow for the ongoing review, analysis and understanding of the performance of a project (or program) through its life.

This sets a framework to enable correct and accurate reporting, provide a basis for continuous improvement and mechanisms to evaluate the successes and challenges faced by programs in the hope of learning from these issues.

There are several aspects to this strategy;

1. Defining measurable benchmarks

If something has not been defined it cannot be measured.

The underlying benefit of establishing a monitoring and evaluation strategy is to provide an indication of how well a project or program is tracking.  If the requirements have not first been defined, then what are the actual results going to be compared against to provide an accurate picture of performance.

To initiate the definition of benchmarks, it is important to consider the following;

• the baseline data; either set by the client, sponsor or other relevant authority or the data that existed prior to the implementation of the project- showing the situation prior to that being impacted upon by the project
• the stakeholders and impacted community; through consultation, their needs, expectations and any other important metrics should be considered. These will provide useful ‘outside’ perspectives which can be used an input into the strategy and later, where relevant translated into objective outcomes for measuring and reporting
• Where possible use quantifiable, objective measures to allow for parity of understanding when being monitored, evaluated or reported amongst multiple parties. Where quantitative information is not practical, clearly explaining explicit requirements; with the inclusion of tests for what is and what is not acceptable will assist.

2. Communicating the benchmarks

There is no point establishing quality control limits if they are not known.

Project personnel as well as in some cases external stakeholders need to understand the benchmarks which will be used in monitoring the effectiveness of the project.

These benchmarks not only provide a basis for the evaluation of project performance, but in many cases, are linked to performance bonuses or related outcomes. If these are being used in your projects, they must be clear- or this will lead to issues.

Important considerations for communicating benchmarks:

• Have the benchmarks been approved? There are cases where overcommunication or early communication can be destructive for your program.  Take into consideration this scenario; you have worked with your stakeholders and established some benchmarks, prior to gaining approval for these benchmarks they are communicated to your project team, contractors and other relevant parties.  Can you identify the issues?  What if these benchmarks are not approved, what if the parties feel the benchmarks are unfair, what if the parties commence work to meet these benchmarks.
• Have the benchmarks been validated? In many cases benchmarks are developed by a senior team through the use of analogous (mainly historic) methods- past program performances are considered as are previous industry performance.  In many cases it is hard to draw a direct link between past programs and current ones; as after all a project (composite of a program) is a unique endeavour.  It is as a result important to validate the benchmarks and standards to ensure currency and relevance to the program.
• Communicate prior to establishment; whilst I have addressed the issues associated with premature communication, it is important that benchmarks are communicated and accepted by relevant personnel prior to being bound by them. The best case (although initially costly) is to involve stakeholders through the entire process; identify the benchmarks, validate and accept as opposed to simply being instructed that these need to be satisfied.

3. Establishing monitoring plans

A monitoring plan should start with the end in mind; what do we want to know?  This could be based upon:

• reporting requirements; “we need to communicate program progress with the board on a monthly basis”;
• quality requirement; “we need to know if a defect variance of greater than 2% exists within our products”
• business requirements; “we need to know if our member base drops below 10,000”
• risk requirements; “we need to know if this risk rating reaches above 17”
• client needs, legislation and a multitude of other factors.

It is important to establish what is important, what we are going to monitor, how it will be monitored, by whom and the frequency.  These aspects form the basis of most monitoring plans.

4. Establish review plans

Review plans are similar to monitoring plans; they provide for an analysis of the actual performance against the baseline or agreed benchmarks.  The difference normally lies in the fact that whilst monitoring occurs concurrent to the program, reviews are generally associated with being performed at the conclusion of a stage, gate or project.

Monitoring is generally the inspection to report, control and where required take corrective active actions to bring the performance to plan; or in some cases adapt the plan.  Reviewing is a retrospective analysis of performance where causation factors (reasons) are identified and linked to deviations from expectations. The review is looking at past information to improve future situations.

Once developed the strategy requires implementation.  The strategy should be, when newly developed, trialled in a controlled environment prior to being applied to a high-risk program.