Business

Software Vendor Evaluation Matrix: Weights, Scores and a Filled-In Example

November 11, 2026 · 6 min read · Etzal Group Team

Three proposals are on the table. The finance head likes the cheapest. The IT manager likes the one with the modern architecture. The owner liked the people from the third. The meeting ends without a decision, or with the decision of whoever spoke last. A vendor evaluation matrix does not make the choice for you. It makes everyone judge the same things, in the same way, and shows where the real disagreement lies.

Before the proposals arrive

Build the matrix first. Criteria chosen after reading proposals tend to favour the vendor someone already prefers.

Step 1: choose the criteria

For a custom software project, these six cover most of what matters:

  1. Understanding of the problem. Did they describe your situation accurately, in their own words? Did their questions show insight?
  2. Solution and approach. Does the proposed design fit the problem, the users and the systems you already have?
  3. Team. Who will actually do the work, with what experience, and for how much of their time?
  4. Delivery method and plan. Phases, milestones, testing, how changes are handled and how risk is managed.
  5. Total cost of ownership. Build price plus three years of hosting, licences, maintenance and likely changes.
  6. Support, handover and ownership. Warranty, response times, documentation, source code ownership and exit terms.

Add a seventh if your situation needs it, such as security and compliance for regulated data.

Step 2: assign weights

Weights express what matters most to this project. They must add up to 100. An example:

CriterionWeight
Understanding of the problem20
Solution and approach20
Team20
Delivery method and plan15
Total cost of ownership15
Support, handover and ownership10

Agree the weights as a group before scoring. The discussion is valuable in itself. If finance wants cost at 40 and operations wants team at 40, you have found the real disagreement, and it is better settled now than disguised as an argument about vendors.

Step 3: define the scale

A number means nothing unless everyone reads it the same way. Write a short definition for each level:

  • 5. Clearly exceeds what we need, with evidence.
  • 4. Meets what we need fully.
  • 3. Meets most of it, with gaps we can live with.
  • 2. Significant gaps or unanswered questions.
  • 1. Does not meet the need.

For important criteria, write what evidence earns a 5. For "Team": named senior people with similar projects, whom you have spoken to, committed for a stated share of time.

Step 4: score individually

Each evaluator scores each vendor alone, with a one-line reason per score. Then meet and compare.

Where two evaluators differ by two points or more, discuss it. One of them usually noticed something the other missed. Agree a score after the discussion, and do not simply average, because the average hides the reason.

A filled-in example

CriterionWeightVendor AVendor BVendor C
Understanding of the problem20435
Solution and approach20454
Team20344
Delivery method and plan15434
Total cost of ownership15523
Support, handover and ownership10345
**Weighted total (out of 500)****385****355****415**

The total for each vendor is the sum of score times weight. Vendor A: 80 + 80 + 60 + 60 + 75 + 30 = 385. Vendor B: 60 + 100 + 80 + 45 + 30 + 40 = 355. Vendor C: 100 + 80 + 80 + 60 + 45 + 50 = 415.

Vendor A is cheapest and scores lowest on the team that would build it. Vendor B has the best design and the highest cost. Vendor C understood the problem best and offers the strongest handover terms. The matrix does not hide these trade-offs. It lays them out.

Step 5: test the result

A total is easy to believe too quickly. Run four checks.

Sensitivity. Change the weights reasonably, for example cost from 15 to 25. Does the winner change? If a small shift flips the result, the vendors are close and the decision should rest on something else, such as references.

Minimums. Set a floor on critical criteria. A vendor scoring 2 on team should not win, whatever the total. Decide these floors in advance.

References. Speak to at least two past clients of the top two vendors. Ask what went wrong and how it was handled.

The people. Meet the team that will do the work, not only the sales lead. If the named people cannot attend, ask why.

Comparing price fairly

Before scoring cost, make the prices comparable:

  • Add what each vendor excluded: migration, training, hosting, licences.
  • Use the same period, three years is common.
  • Include your own team's time where a vendor expects more of it.
  • Check the day rate for changes. A low build price with a high change rate becomes expensive.

Only then score. Otherwise the cheapest proposal is simply the least complete.

Common mistakes

  • Building the matrix after reading the proposals.
  • Too many criteria, so nothing is decisive.
  • No written definitions, so a 4 means different things to different people.
  • Averaging away disagreement.
  • Letting presentation skill inflate every score. Score the presentation separately if at all.
  • Ignoring a low score on a critical criterion because the total looks good.
  • Treating the total as the decision. It is an input. A person still decides and signs.

Keep the record

File the matrix, the reasons and the reference notes with the contract. When someone asks a year later why this vendor was chosen, the answer is on one page. It also helps the vendors who lost: a short, factual explanation is a courtesy, and it keeps them willing to bid next time.

We are often on the receiving end of matrices like this one, and we would rather be scored by a clear one. If you are preparing a selection and want a second pair of eyes on the criteria, contact us, or see how we scope projects in Enterprise Software.

Summary

Choose five to eight criteria and agree weights before proposals arrive. Define what each score means. Score individually, discuss the differences, and compute the weighted total. Then test it: vary the weights, apply minimums, call references and meet the delivery team. Decide with the matrix in front of you, and keep it on file.

Frequently asked questions

How many criteria should a vendor evaluation matrix have?

Five to eight. Fewer and important differences are hidden inside one score. More than eight and each weight becomes so small that nothing decides the outcome. Group related points under one criterion and describe them in its definition.

Should price have the highest weight?

Not usually. For custom software, a weight of 20 to 30 percent for total cost of ownership is common. A project that fails or needs rework costs far more than the price difference between proposals. Weighting price above everything selects the vendor that left the most out.

What if the highest score goes to a vendor the team does not trust?

Take the doubt seriously and find its cause. Often a criterion is missing from the matrix, such as how well the vendor understood the business, or reference checks have not been done. Add what is missing, rescore every vendor on it, and see whether the result changes.

Author

Etzal Group Team

Etzal Group Team, helping businesses grow with technology. Contact us.

Related service

Free consultation

Not sure where to start? We will map the right system for your business, free.