Part 2 · Article 6 of 14
How many raters do you need for a 360 survey?
In short
- Invite 5 to 8 raters per rater group (peers, direct reports, others), so that at least 3 in each group reply. Most people have 1 manager, and that rating isn't anonymous.
- Fewer than 3 replies breaks confidentiality, because individual answers can be worked out.
- More raters also make results more accurate, because raters in the same group often disagree.
- For engagement surveys, invite the whole team. For client surveys, invite as many clients as you can.
Org360 defines a confidentiality threshold as the minimum number of replies a survey needs before its results can be shown. Org360 reports open after 3 replies in Standard mode and after 5 in Highest mode.
What is a rater group?
A rater group is a set of raters who share the same working relationship with the person being reviewed. The usual groups in a 360 are the person themselves (self), their manager, their peers, their direct reports and others, such as clients or partners. Each group sees a different side of how someone works, so a good 360 includes several.
Why does fewer than 3 replies break confidentiality?
- With 1 reply, the score is that person's answer.
- With 2 replies, each rater can subtract their own answer from the average and see exactly what the other rater said.
- With 3 or more replies, individual answers blend into an average that can't be taken apart.
That's why 360 providers set a minimum. The Center for Creative Leadership's Skillscope 360 needs at least 3 peers and 3 direct reports before it shows those groups, and recommends inviting 5 of each. DecisionWise recommends 8 to 15 raters in total, with at least 3 per group. The Hogan 360 needs at least 5 rater responses in total, and merges any group with fewer than 2 into a single "Other" group.
Why do more raters make a 360 more accurate?
Raters in the same group often see a person quite differently. A 1998 study of 360 ratings for 153 managers, by Gary Greguras and Chet Robie in the Journal of Applied Psychology, found little agreement between raters within the same group, and concluded that 360s need more raters than organizations usually use to be reliable.
More raters smooth out one person's very good or very bad day. It's one more reason to keep a survey short: 12 questions make it reasonable to ask 15 colleagues.
Who should you invite?
- People who've worked with you closely for long enough to see how you work, typically some months.
- A mix of groups, so you hear from people above, beside and below you.
- People who'll be honest, including people you've disagreed with. A 360 filled only with friends tells you what you already know.
- Enough per group to cover no-shows. If a group has only 1 or 2 people, merge it with another group, so those raters stay anonymous.
How many respondents does an engagement or client survey need?
- Engagement surveys: invite everyone in the team or organization. A response rate above about two thirds gives a fair picture. For a team smaller than 5, combine it with a neighboring team, so the small team's answers stay confidential.
- Client surveys: invite every client you can. With fewer than 10 replies, read the scores as a signal and pay most attention to the comments.
What the research says
- Raters disagree, so you need several. Greguras and Robie (1998) studied 360 ratings for 153 managers and found little agreement between raters in the same group. They concluded that "more raters are needed than currently used" to reach acceptable reliability.
- Thresholds are standard practice. CCL, DecisionWise and Hogan all set a minimum number of raters per group before showing results, typically 3, to protect anonymity.
Key readings: Greguras and Robie (1998); DecisionWise, 360-degree feedback best practices. Full references are in the research reading list.
Org360's view
Org360 recommends inviting 5 to 8 raters per group and expecting at least 3 replies in each. That protects confidentiality, gives a reliable average, and stays manageable because each survey takes under 10 minutes. The person running the 360 chooses the raters, because they know who sees their work (article 2).
Frequently asked questions
What is the minimum number of raters for a 360?
At least 3 replies per rater group, apart from the manager. Org360 reports open after 3 replies in Standard mode and 5 in Highest mode.
How many raters should I invite?
5 to 8 per group, or about 10 to 20 in total, so you still reach 3 replies per group if some people don't answer.
Is the manager's feedback anonymous?
Usually not. Most people have only 1 manager, so that rating can be identified. Tell your manager that up front.
What if I only have 1 or 2 direct reports?
Merge them into another group, such as peers, so their answers stay anonymous.
Sources: Greguras and Robie (1998), Journal of Applied Psychology · CCL: Skillscope 360 FAQ · DecisionWise: 360-degree feedback best practices · Peter Berry Consultancy: Hogan 360 evaluator FAQs