"Ugh, not another survey"

When we ask teams to participate in the Columinity, we sometimes hear, “Ugh, another survey”. We fully support this sentiment if you consider how surveys are usually organized. As a participant, you don’t get feedback after you’re done. Nothing happens with the results, and your answers are clearly not anonymous.

And that's unfortunate. A well-designed survey is a very time-effective way to develop actionable insights from data. It can shed light on areas that you can’t otherwise measure. In return for several minutes of your time, an honestly answered survey can help your team identify important improvement areas that benefit you.

This post addresses common questions and concerns from teams about Columinity. It is ideal to read if you are skeptical about it, don’t see the point, or when your team members are.

Why measure soft factors?

Some teams don’t see the value of a survey in addition to other things you can measure about teams, like job satisfaction, cycle time, code quality, and general flow metrics. While these are all valuable, they can’t shed light on the psychosocial parts of teamwork. Because teams are groups of people, there are always a lot of psychosocial processes going on in and around teams that influence their work. How much conflict is there? Do people feel safe enough? Do members understand the value of their work? Do teams feel supported by management? 

These may feel “softer” than specific business metrics (like cycle time and code quality) because we’re asking members for their perspectives. But there is great value in learning how people perceive the work done in and by their team. You would undoubtedly be concerned when most members feel their work lacks purpose when there is a sense that stakeholders aren’t available, or when there is little attention to quality. Even when those perceptions do not match what is actually happening in a team, those perceptions are real for those who express them.

Ideally, you use surveys in addition to harder metrics. Each will provide a different perspective on your work as a team and different starting points for improvement.

Why are there duplicate questions?

Observant teams notice that Columinity has some questions that are very similar. For example, one question reads “This team is able to choose the way to go about its work” while another reads “This team is free to choose the method(s) to use in carrying out work”. That is quite similar, right? So this might feel like a mistake or a waste of time.

But it's not a mistake. The questions are similar, but they’re not the same. We include both to increase the accuracy of the measurement. One of the challenges of surveys is that you’re effectively measuring how people perceive the work in their team. This is subjective. Since we can’t look directly into someone’s brain, we need to ask well-worded questions to “triangulate” their perception indirectly. Because we must rely on language, we must contend that people vary in how they understand certain words. This may be due to personal differences, cultural differences, and language comprehension skills in general. In the examples above, the part “this team is able to” may be interpreted by one person as “if we try really hard, then yes” but as “this is a walk in the park for us, so yes”. One question also taps more into how teams do their work, whereas the other taps more into the methods that teams select to do their job.

In designing our surveys, we aim for 3 or more questions per topic that tap into related but not identical interpretations of what we’re after. We remove questions that are mostly entered in a consistent pattern because this means they are interpreted identically, and there is no point in asking the same thing twice. We remove questions that show very inconsistent patterns across participants because this means they are interpreted very differently. We keep questions that are interpreted consistently enough. This results in a set of questions with high measurement reliability. While individual questions may be answered differently by participants because they understand the questions a little differently, we can combine their answers to get a reliable measure of the topic we’re after.

So, while some questions may appear to be duplicated, they are not. We purposefully add them to make the measurement more reliable and, thus, more useful for your team to drive improvement.

How can I trust if my answers are anonymous?

Privacy and anonymity are very understandable concerns when participating in a work-related survey. How do you know that your answers won’t end up with your manager or someone from HR, who may retaliate for answers they don’t like? Unfortunately, this concern is warranted for many well-known survey platforms. It is especially relevant for surveys set up by people in your organization using tools like SurveyMonkey, Qualtrics, and Google Forms. Some online team diagnostic tools also provide individual-level answers to management, although they are not usually transparent about this to participants. Unfortunately, if you can’t trust that your answers are anonymous, you won’t give your honest perspective on the situation. And this means that everyone loses.

We designed Columinity with privacy in mind. You can trust that we won’t provide individual-level answers to your managers, HR department, or any other organization or party. Our API doesn’t even allow it at the code level. In fact, this is explicitly stated in our terms of use, so we are legally (and ethically) required to honor it.

We have implemented several mechanisms to protect your privacy to a much larger extent than other tools out there. A detailed overview is available here, but the summary is:

• We only report team-level averages. You are the only person who can see your own scores in Columinity.
• You can participate fully anonymously if you want, making it impossible for even the one person at Columinity with access to the production database to see your answers.
• The Team Report for your team only shows results when at least three people (team members, stakeholders, or supporters) participated in the same segment.
• Your team can set a password for their Team Report to restrict access.
• Your team may enable “Low-Trust Mode” in the Team Report to anonymize your team's name in the reporting.
• We categorically decline questions from organizations that request individual-level data. The only data they can access is team-level results.

Ultimately, it remains a matter of trust. However, we hope the mechanisms outlined show how seriously we take your privacy. We can’t give your team and organization a reliable measurement if you can’t trust that we won’t pass your answers on to others. So it is also in our interest, and that of your organization (even though some need to recognize this first), that you can feel confident that answers are anonymous and private.

Note that we may use data from Columinity for scientific research. The datasets don't contain any information that identifies you, your team, or your organization.

What will be done with the results?

We created the Columinity to help you and your team improve. Unlike most tools, where the report is first shared with management, we first share it with your team. It is up to you to share it further.

So, what happens with the results is entirely up to you and your team. The Team Report provides an overview of results in the various areas of our evidence-based model. This gives a sense of where things are going well and where improvement could make your experience as a team better. If you’ve taken the survey before with your team, we will also show which areas your team improved and where more work is needed. If you don’t know where to start or how to improve, we also provide tons of actionable feedback to help you get started. In other words; you and your team are fully in control and it's up to you to decide what works best, with our tool as your companion.

Obviously, there will be many areas where you will need help from other teams and management to improve. The Teams Dashboard, which comes with a subscription, provides excellent opportunities to identify patterns across teams and signal that you need help from management. Instead of the typical top-down management style encouraged by many diagnostic tools, we give your team full control and allow you to signal for help when you need it. To do so, mark any improvement action you identify as “Impeded”. It will then show up in the Teams Dashboard.

Closing Words

Participating in a Columinity scan for your team might take between 5 and 15 minutes. But sometimes, taking a step back and reflecting on how you work as a team is essential. Our evidence-based model is based on insights from organizational and social psychology. The improvements you make based on it will likely make your work more enjoyable and fulfilling and the purpose of your work clearer. We don’t know about you, but that is a nice return on investment for those minutes. Give it an honest try and see where it takes you!