Beware of some common pitfalls

Common measurement pitfalls

Understanding some common pitfalls can help you collect and interpret data:

Collecting information that does not help you understand current problems

Irrelevant indicators do not add value. Once you have identified the measures you want to collect, reflect on the scope and ask:

• How does this data support good decisions?
• Am I focused on the most important challenges or knowledge gaps?
• Are these the ‘best’ measures to meet my needs.

Relying only on a single measure or indicator 

It may not tell the whole story. Supplementary measures and descriptive data can also help you to understand results.

Assuming problems have simple causes

Often, several factors will contribute to an issue, including non-work-related factors. See if you can find things that have the most influence on the problem or situation.

People changing behaviours because they are being measured

People often change their behaviour when they know they are being observed or they have an incentive to achieve a specified goal. So, it is important to be clear about what data can, and cannot, explain.

Measurement creating unintended consequences 

Sometimes, goals can create the opposite outcomes (e.g. goals to reduce conflict can increase stress if people are concerned they can no longer report conflict).

Misinterpreting results

Sometimes measurement can make it look like a situation is new or getting worse. However, sometimes the problem was always there but people did not feel able to report it. Encouraging people to speak up about their concerns may generate more disclosures than you expected. So, it may look worse before it gets better.

Be careful benchmarking your performance

Comparing your results over time, or with an industry benchmark or average can be useful. But if you are using benchmarks, make sure the comparisons are meaningful. Comparisons are not useful if your workplace’s context, operations, risk profile, culture or issues are very different from the benchmark or industry average. There is also no point in comparing your business with an industry benchmark if that measure is not useful for making decisions for your business.

Beware of survey fatigue 

Every time you use a survey, you ask participants to invest time and energy in responding. Here are some tips on avoiding survey fatigue:

• Keep surveys short and to the point—e.g. ask direct, unambiguous questions, limit the number of questions that require written answers.
• Use language that is appropriate for the respondent.
• Use surveys only as required.

Most importantly, use the information you collect. If you do not have time, resources or willingness to act on results, it may be best not to collect the information in the first place.

You can’t measure everything. Focus on the information and data that tells you most about your organisation.   

Tips for avoiding common pitfalls

• Keep data collection manageable. Do not try to measure everything all at once.
• Keep communication lines open about what and why you are collecting data. This can improve people’s willingness to participate in the process and improve the completeness and quality of data collected.
• Keep good records. It makes it easier to look at trends over time. But remember, record keeping needs to be confidential and secure.
• Remember understanding what a number means and why it is the result at this time is more important than the actual number itself. It may be more helpful to show how the data fits into your overall approach, rather than what it means in isolation.