Sensitivity Analysis

The sensitivity analysis helps answer the question, "How sensitive is my solution to changes in weights?"

We run sensitivity analyses because determining weights can be such a subjective process, and we’d like to have some sort of quantitatively warm fuzzy to back us up.

After it finds the best course of action, SquidMat goes through each evaluation criterion (i.e., each data column) and finds all the weights where the solution changes. The program tells you when a different course of action wins and the weight at which that occurs.

Here's the sensitivity analysis from the sample data file (sample.SQM).

Keep in mind that the actual weights are Quality = 3, Cost = 1.5, and Time = 1 and that Mama Mia's was found to be the best course of action.

SquidMat's sensitivity analysis begins with the first evaluation criterion, Quality, and changes its weight to 1. It finds that at a Quality weight of 1, Subway is the best course of action. SquidMat then calculates the weight at which Subway loses to another course of action. In this case, it happens at a weight of 2.2192. At that weight, China Buffet wins. Then SquidMat finds that at a Quality weight of 2.7063. Mama Mia's becomes the best course of action. Mama Mia's holds that honor for any Quality weight value at or above 2.7063, which is the range within which the actual weight of 3 lies.

Note that since Mama Mia's scores highest on the Quality criterion, it would be expected to win at higher weights for Quality. Conversely, Mama Mia's scores lowest on the less-is-better (LIB) criteria, Cost and Time. So we would expect that if it wins (and we already know that it does, since it's the overall winner) it will win at lower weights for Cost and Time.

And that's precisely what the sensitivity analysis reports. At a Cost weight of 1, Mama Mia's takes the lead and holds that lead through 1.5 (the actual Cost weight) and gives up its lead at a Cost weight of 1.7243, where China Buffet prevails. Subway, the cheapest of the three courses of action, predictably takes the lead shortly thereafter at a Cost weight of 2.6928. A similar pattern occurs for Time.

Notice that the sensitivity analysis doesn't report Mama Mia's as the winner at a weight of 1 for either Cost or Time, even though it is the winner. This is because the analysis is programmed to report changes. Since Mama Mia's is already the winner, the sensitivity analysis will not report it. It will report when the winning course of action regains its winning position after another course of action has taken it away. For example, at a Quality weight of 1, Mama Mia's loses to Subway. Mama Mia's eventually gets it back at a weight of 2.7063. Again, the emphasis is on weight values where the winning course of action would change.