Why your 5-point company culture survey is outdated

Culture is intangible they tell me, something that you can feel and maybe even smell but hard to measure and put a number to. But if culture is truly foundational to the long-term success of any company, can we afford to settle for these vague terms and moving goal posts?

Most leaders in big or small organizations today understand the importance of culture in elevating a company from good to great. The difference lies in the way they articulate it and the methods they employ to measure it.

Benchmark surveys, such as Gallup’s Q12, provide a framework for measuring culture within an organization and also for comparing it across companies. I’ve deployed them in previous organizations and found them quite insightful. But the need to generalize has the downside of losing out on the specifics of an individual company. After all, every company is different in its own unique way, and this uniqueness is often the reason for their success.

A company’s culture will continue to be the driving force for its success. We at OrangeScape are aware of this, particularly since we’re scaling up. So, we conducted our most recent biannual survey with a twist, steering away from the traditional five-point scale format (strongly disagree to strongly agree).

The primary reason for such a change was to get a better understanding and analysis of the team culture. Using the insights from the floor helped make the response options as specific and close to reality as possible. We felt this would give us an edge when devising action plans based on the survey results.

The second (equally important) reason is the science behind using traditional surveys. In a thought-provoking piece, entrepreneur and author Alan Klement shines a spotlight on the fallacy that putting a number on something makes it a quantitative variable—it does not. Rather, assigning a number on something is a categorical description of quantity. This distinction becomes very important when we summarize results, as we can’t treat them as interval data when there is no way to tell if the interval between points on the scale are equally spaced.

To help you better understand the difference, here are both versions of our culture survey. The traditional five-point scale format is displayed first (orange), followed by the new approach we took (green). You can access the full survey here.

Before and after

Rationale:

In addition to knowing the level of one’s job clarity, we also wanted to understand the channel through which employees got it. In cases where there were multiple methods, this gives a clearer picture of which method was more effective for specific teams and/or roles.

Rationale:

Knowing the amount of feedback alone was insufficient. We also sought insight into:

Questions dropped

Questions added