Multicriteria Decision Analysis Explained
Some decisions cannot be reduced to money. A delivery route, a supplier, a site, a technology: each option is better on some dimensions and worse on others, the dimensions are not comparable in any natural unit, and several people with different priorities have to live with the result. Multicriteria decision analysis is the structured way of handling that, and its reputation suffers because it is so often produced after a decision to justify it.
Section 5 of the PMBOK® Guide Eighth Edition includes it among the analysis techniques available, and the first thing to be clear about is what it does. It does not choose. It makes the basis of a choice explicit, which is a more useful thing and a less comfortable one.
What the method is for
Making the basis visible. When the criteria and their weights are written down, the decision stops being a matter of judgement that somebody has to be trusted on and becomes something a challenger can examine. That is valuable in any organisation and close to essential in one that has to answer for its choices publicly.
Moving the argument to where it belongs. People rarely disagree about scores; they disagree about what matters. Forcing the weights to be set first converts a disorderly argument about options into a structured one about priorities, and once the weights are agreed, a considerable part of the disagreement has already been resolved.
Four ways it goes wrong
Weights set after the scores. The most common distortion and the most damaging. Once people can see which option wins under which weighting, the weights become an instrument, and everybody in the room knows it. The discipline is to agree and record them before any option is scored.
False precision. Scores carried to two decimal places, weights expressed to the nearest per cent, on judgements made by five people in an hour. The arithmetic is exact and the inputs are not, and the presentation invites a confidence the analysis cannot support. Round the numbers to something honest.
Criteria that overlap. Whole-life cost, operating cost and energy efficiency in the same list will count the same advantage three times, and the option that is cheap to run will win before anybody has considered anything else. Criteria should be tested against each other for independence before the scoring starts.
Fatal flaws averaged away. A weighted total can carry an option through a condition it fails outright: it does not meet a regulatory requirement, or it cannot be delivered inside the mandated date. Those belong as threshold criteria applied before the scoring, and anything that fails one leaves the comparison entirely.
A delivery route and an argument about weights
A government department was choosing how to deliver a new service to the public: build in-house, extend an existing departmental platform, or buy a managed service. All three were viable and each had strong advocates, and two previous meetings had produced nothing but restated positions.
The analysis was run in the order that makes it useful. Criteria first, agreed in a session with the senior responsible owner present: time to first service, whole-life cost over eight years, in-house capability retained, flexibility to change the service later, and security assurance effort. Then weights, argued for about forty minutes and agreed before any option was discussed, with the reasoning recorded against each.
The forty minutes were the substance of the decision. The commercial adviser wanted whole-life cost weighted highest; the delivery director wanted time to first service; the argument that settled it was that the service had a legislative deadline, which made time a threshold rather than a criterion. That single reframing removed one option from the comparison entirely, because it could not meet the date, and it was lifted out of the row before scoring began.
The remaining two were scored in half an hour, and the totals came out close. That closeness was the useful output: it meant the choice was genuinely finely balanced and could be made on a factor outside the model, which was the department's existing relationship with a supplier. The decision was recorded with the weights, the scores, the threshold and the tie-breaking reason attached.
Nine months later the choice was challenged by a new minister's office. The paper answered the challenge in a day, because every part of the reasoning was already written down, including the part that was a judgement.
Running one honestly
Weights before scores, always, and recorded with their reasoning. If the weights cannot be agreed in advance, that disagreement is the real decision and it belongs in front of whoever can settle it.
Test the sensitivity. Change the two largest weights by a reasonable amount and see whether the answer moves. If it does, say so: the honest report is that the decision depends on this particular view of priorities, which is information the approver needs. If it does not, the result is robust and worth saying so too.
For a PMP® candidate, the thing to carry is that the technique structures a decision without making it, so a scenario offering a scored comparison is asking where the weights came from. A response that accepts a total without the criteria behind it has accepted somebody's priorities unexamined. Option appraisals where the arithmetic is sound and the weighting is the real argument are a regular exercise in a structured PMP exam preparation course.
Before the next option comparison, run the weights session separately and a week earlier. It feels like an extra meeting and it is where the decision actually happens, and holding it before anybody has seen a score is what makes the result defensible afterwards.
Interested in going further?
A scored matrix is either a record of how a decision was made or a decoration applied afterwards, and the difference is entirely in the sequence. Omega's PMP® Exam Preparation works through option appraisal as a governed process.
Decision analysis techniques of this kind are covered in the PMBOK® Guide Eighth Edition.
Ad · Amazon affiliate link.
References
A180: Facilitation Techniques for Difficult Project Conversations
A162: Decision Trees Without the Headache
A183: Cost-Benefit Analysis: Why the Cheapest Option Is Not Always Best
A161: Expected Monetary Value Explained
A163: Monte Carlo Analysis: What It Tells You and What It Doesn’t
PMP and PMBOK are registered marks of the Project Management Institute, Inc.
