Although nonprofit leaders recognize the importance of measurement, they often find the process of data analysis and decision-making to be daunting. To the contrary, our 7 Steps to Data Driven Decision Making mirror the scientific process that we all learned in middle school.

The key is to set up the “experiment” appropriately: clearly define the issue you are investigating, develop clear hypotheses about what may be going on, test your hypotheses and then draw conclusions from the analysis.

Over the last few months, this series has described how to frame the issue, develop hypotheses, collect data and analyze data. In this installment, we will share our tips for Step 5: Data Interpretation.

Data Interpretation

Data interpretation is straightforward for those who set up their “experiment” correctly in steps one through four. Use your data analysis from Step 5 to test which of your initial hypotheses are true or false. Let’s look at a few examples.

Examples of Data Interpretation

Case Study 1: Foundation

Sarah is a Program Manager for a capacity-building foundation that wants to increase volunteer retention. She prioritized hypotheses to test and collected relevant data from an existing internal database. She then analyzed her data and identified strong correlations between variables. Remember that a correlation coefficient, which represents the extent to which two variables are related, is generally considered negligible or weak when between .0 and .4, moderate when between .4 and .7 and strong when between .7 and 1

Sarah’s first hypothesis stated that those grantees who invest more time and money into training and support for their volunteers will have higher retention rates. When Sarah ran a correlation test on training dollars and retention, she found a correlation coefficient of .85, indicating that these two variables had a high positive correlation. This means that the nonprofit grantees who invested more money into their volunteer training programs were, in fact, more likely to have higher retention rates. The data supported her first hypothesis.

On the other hand, Sarah’s next hypothesis, that nonprofits with greater impact would have higher retention rates, only had a correlation coefficient of .05. Thus, Sarah concluded that an organization’s results and impact were not strongly linked with retention rates as she had hypothesized they would be. She rejected her second hypothesis.

Case study 2: Private school

John, the principle of an independent school, wants to investigate the perceptions and performance of his middle school math program. He gathered his data from parent surveys, test scores, staff evaluation forms and training documents. John examined summary statistics like average, median and standard deviation to understand his data. He also decided to use t-tests to determine the statistical significance of his observations. See last month's installment, Step 4: Data Analysis, for a more detailed look at John's analytic process.

John’s parent survey revealed that satisfaction scores for parents of students in the middle school grades were 0.4 points below those in the elementary school grades. Although the difference appeared small, John's t-test revealed that it was a statistically significant difference. John learned that middle school parents had more concerns because they wondered whether their children were sufficiently prepared as they approached high school. John’s first hypothesis was true—parents did not understand what their children were learning in their math courses.

John also learned that while his elementary school math scores had improved over the last three years, his middle school scores had declined over the same period. He compared his middle school math scores with other similar schools and found out that his school’s scores fell below those of its peers.

Interpret your Data in Context

There are several ways to contextualize data, which will help you interpret your findings. For example, you can look at changes in your data over time, examine differences between demographic groups, or compare your results to the performance of peer organizations. Here are some ways to contextualize your data:

1. Data can be tracked over several years to compare the longitudinal history and see trends over time.
   
2. Survey data can be cut by demographic groups (zip code, income, gender, etc. ) to understand how the average scores are impacted when looking at the data aggregated in different ways. Sometimes it makes sense to compare data for parallel survey questions. John did this when he looked at average scores for each of the subject areas (Exhibit 1).
3. If you have access to data from peer organizations, comparing your data to theirs can help you understand where your institution stands relative to an external comparison.

It is critically important to compare your results to an external benchmark as you interpret your data. John, for instance, may have interpreted his students’ math scores differently if he saw that his scores were declining, but noticed that they still remained far above the average for similar schools. Sarah didn’t have peer group information, but used her organization’s longitudinal history to put her results into context.

Putting your data into context will allow you to understand the implications for your organization and enable you to make strong, data-driven decisions.

Check out next month’s newsletter, where we dive into Step 6: Decision Making.

Want to beef up your data interpretation skills? You can find nearly unlimited resources online, including videos, free google books, articles and blogs. If you would prefer some expert guidance, consider signing up your organization for one of Measuring Success’ Building Data Competency trainings. We can easily customize one of these workshops to meet your unique needs. Email us if you would like more information.