I have written before about the logical framework to define the clear linkages between the work done on a project and it’s ultimate impact. The principle here is that project should focus on making a change in society not just delivering things. To many projects just deliver things that do not meet the needs of stakeholders so don’t get used.
However, the language gets in the way and caused confusion.
Recently I have been working on several international projects I have discovered several variants of this model. These include project lifecycle management, results-based management and the theory of change. Add to this Product Based Planning from PRINCE2 and the use of Agile product owners, APM Work Breakdown structures and product breakdown structures. Even I get confused, let alone my international project team from several nations of the world. Especially as several of the words such as output, and outcome have no direct equivalent in the languages used by the team.
So in this post, I am going to reduce the language to it’s simplest form, develop a simple everyday example. The compare this with the different models and check the translation into common languages.
A Simple Example: Building a Home
Let’s start with a very simple example, building a home for your growing family. To make it simple we are going to assume you are doing this for yourself, using your own time and effort. You are building the home because your family is growing and you need space to work from home. You would ideally like each of your children to have their own bedroom and a separate space you can work away from the family.
|You have better health, better family relations and better working environment.
|Longer life, less health problems, happy partner, improved job prospects.
|You family now has somewhere to live which suits your needs
|Your children have a room each and you have space to work from home.
|A finished house
|You visually check the house before you move in. Does the rook leak? Are all the electrics working and safe?
|Building the foundations, walls, roof. Installing electrics and plumbing, decorating the inside.
|Progress compared to the plan, planned cost vs actual cost.
|Wood, bricks, nails, cement, sand, your own time and that of your friends.
|Have all the necessary material been delivered to site? Have you go the necessary time and skills?
Let’s look at each of these in turn
Input resources are the things you need to get the work done. These include obvious resources such as your time, but also material that you need to get the work done and the tools, equipment and information you will need. Ideally, you would identify these resources in advance. There is not much point getting the wall built and then discovering you have to wait six months for the roof tiles before you can get the roof on.
Other words for inputs are resources and material.
The OECD define this as:
The financial, human, and material
resources used for the development intervention.
Ressources (Moyens, intrants)
Moyens financiers, humains et matériels utilisés pour l’action de développement.
Recursos financieros, humanos y materiales empleados en una intervención para
They are the sub-elements with which we are going to build the house, normally as delivered by another supplier.
This is where it can get complex because we would see a brick as an input, but to brick manufacture, bricks are the product they output. Enough of this real-world complexity lets get back to the simple example.
How to measure inputs?
I always imagine input as a check list before I start work. Have I got all the materials I need? Do I have the necessary equipment? What human resources do I need?
These are the jobs that need to get done to build the house. They take effort and time to complete. Digging the foundations, building the walls, painting the rooms. They are all practical things that need to get done. These activities change the inputs into the components (or products) used to make the house. For example, we use bricks, cement, time and effort to build the walls for the house. They normally have a cost estimate made up of the work, the material and expenses.
Other words for this level include tasks, work packages, activity.
The OECD define this level as
Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilized to produce specific outputs.
Actions entreprises ou travaux menés
en vue de produire des réalisations spécifiques. L’activité mobilise des ressources telles que des fonds, une assistance technique et d’autres types de moyens.
Acciones emprendidas o labor realizadahttps://www.oecd.org/dac/evaluation/2754804.pdf
mediante las cuales se movilizan los insumos, como son los fondos, la asistencia técnica y otros tipos de recursos, para generar productos determinados.
How to measure activity?
Activity takes effort and time to complete so we measure it by the number of hours spent or the progress archived building the wall.
The ultimate output is the finished house, but often it’s useful to define sub-components as outputs. The outputs are the completed parts produced by the work. Typically they have been through some individual testing or measurement before they get assembled into the final output. This is why separate outputs are useful, especially in projects with several suppliers. They help define who will produce what and when. All the individual outputs are combined to produce the final output.
In our house example, the outputs would be foundations, walls, roof. Importantly for them to be complete they need to be checked, even if that check is just a visual inspection of the walls to see if they meet the drawing dimensions. These all combine to produce the house as a final output.
Other names for this level are products, deliverables, interfaces, milestones.
The OECD define outputs as:
The products, capital goods and services which result from a development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes.
Biens, équipements ou services qui résultent de l’action de développement. Le terme peut s’appliquer à des changements
induits par l’action qui peuvent conduire à des effets directs
Comprende los productos, los bienes de capital y los servicios que resultan de una intervención para el desarrollo ; puede incluir también los cambios resultantes de la intervención que son pertinentes para el logro de los efectos directos.
How to measure outputs?
Physical outputs are the easiest to measure. As they are completed the quality should be checked before they are marked as complete. Some products such are training as a bit less easily measured but a course register or feedback evaluation can be used to test that the output was complete.
This is the new thing that you can do as a result of the output. In this case, you have space for all your family and you have a suitable environment for working from home.
In a real project, it could be a new capacity or capability. Something that you can do now that you could not do before the project for example such as speak a foreign language or programme a computer.
It is outcomes that enable people to change lives. Outputs are only useful if they help you to achieve the outcomes. For example, if you have 2 children and your partner is pregnant with another child then a 3-bed house will not achieve your outcome of one room for each child.
Other words for outcome include capacity, capability, competence, behaviour.
The likely or achieved short-term and medium-term effects of an intervention’s outputs.
Réalisation (Effet direct)
Ce que l’action doit accomplir ou a accompli à court ou à moyen terme.
Representa el conjunto de resultados a corto y mediano plazo probables o logrados por los productos de una intervención
This brings up another issue. The achievement of the outcome is often outside of the control of the project. If your employer will not let you work from home, then having a lovely home office won’t help you to achieve your outcome. For this reason it’s important when planning project to understand the outcome before thinking about the outputs, activities and inputs.
How to measure outcomes?
Outcomes are much more difficult to measure than outputs. Some outcomes are binary. You have a bedroom for your children. Others are more difficult such as your effectiveness whilst working from home. How can this be measured? Can you speak a foreign language? As a tourist or to explain complex concepts?
Often the outcome can not be measured for some time after the end of the project. For this reason, it’s very important to define the outcome measures before the project starts.
The impact is the benefits that come from the outcome. So you have a lovely new home, what improvement does this make to your life.
You get a promotion because you can concentrate on your work in your new home office. Your children do better at school because they can do their homework without interruption. Your health improves because you have less stress and your relationships with your partner improve.
Impacts take time to emerge. They are often dependent on other factors outside the control of the project, but they provide the ultimate justification for the time and effort invested in the project. Without an impact, the project is a waste of time.
The word for impact is benefits.
The OECD define impacts as:
Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.
Effets à long terme, positifs et négatifs, primaires et secondaires, induits par une action de développement, directement ou non, intentionnellement ou non.
Efectos de largo plazo positivos y negativos, primarios y secundarios, producidos directa o indirectamente por una intervención para el desarrollo, intencionalmente o no.
How to Measure Impact?
As with outcome, impact can only be measured some time after the end of the project. Typically this impact assessment will be done 6 to 12 months after the project is completed. Although for some longer term project the evaluation may be many years after the project is completed.
However, it is important to establish measures of impact as part of the project. This needs to be done early for two reasons. First, the measuring system may need to be built into the design of the project. Second, the impact helps to define the priorities for the project.
So why is this so confusing for international project teams?
So far this all seems very simple, so how can it cause confusion. There are two causes of confusion
- Inconsistences language across different models
- Translation issues
lets explore these in a bit more detail. Consider several different
|Results Based Management
|The overall goal
The goal is the impact that this project,
if combined with others, will achieve.
|Ultimate outcome –
The highest-level change to which an organization, policy, program, or project contributes through
the achievement of one or more intermediate outcomes.
A positive and measurable impact of change.
The measurable improvement resulting from an outcome perceived as an advantage by one or more
The project purpose is why the project
is being done. It describes the desired
|Outcome or result –
Results are the same as outcomes. An outcome is a describable or measurable change that is derived
from an initiative’s outputs or lower-level outcomes. Outcomes are qualified as immediate,
intermediate, or ultimate; outputs contribute to immediate outcomes; immediate outcomes contribute
to intermediate outcomes; and intermediate outcomes contribute to ultimate outcomes.
The changed circumstances or behaviour that results from the use of an output and leads to realisation of benefits.
The result of change, normally affecting real-world behaviour and/or circumstances.
Outputs are what the project is to
Direct products or services stemming from the activities of an organization, policy, program or project.
|Output The tangible or intangible product typically delivered by a project. Used interchangeably with deliverable and product.
A specialist product that is handed over to a user(s).
Any unique and verifiable product, result, or capability to perform a service that is produced to complete a process, phase, or project.
Activities define how you will do the
Actions taken or work performed through which inputs are mobilized to produce outputs.
A task, job, operation or process consuming time and possibly other resources.
The set of information relevant to the creation of one or more products.
A distinct, scheduled portion of work performed during the course of a project.
The financial, human, material and information resources used to produce outputs through activities in
order to accomplish outcomes.
All the labour and non-labour items required to undertake the scope of work to the required quality.
|Not defined although resource mentioned in several ways such as Resource Breakdown Structure
Although several definitions seem to exist from different organisations
Results Based Management: https://www.international.gc.ca/world-monde/assets/pdfs/funding-financement/rbm_glossary-eng.pdf
APM Glossary: https://www.apm.org.uk/resources/glossary/
The results is confusion even if you are a native English speaker. Introduce this to a multinational team all with different backgrounds an you get significant confusion. This confusion matters because it is core to all the planning, progress reporting an evaluation of the project.
A Proposed Solution
A proposed solution is to go back to first principles, and use simple language.
As humans, we take raw material, skills and time (inputs) ….
we apply work and effort (activity)…
to produce things (outputs)
that enable us to do something new (outcome)
and this gives us a reward (impact)
Conceptually these five levels are clear and distinct. We then need to think about how we can measure and evaluate how successful we are at each level and what are the linkages between them.