This guide explains how to conduct evaluation calibration.
Calibration can only be performed by administrators with evaluation management permissions.
Creating a Calibration Evaluation
In Create Evaluation - [Evaluation Detail Settings] under Evaluation Elements and Steps, you can configure whether to enable [Calibration]. Calibration can only be conducted when calibration is activated.
Closing the Evaluation Submission Period
For evaluations that include calibration, you can find the [Calibration] menu under the [Calculate Results] menu on the evaluation management screen.
To proceed with calibration, the evaluation results must have been calculated and the evaluation submission period must be closed.
What types of ratings can be adjusted through calibration?
The ratings that can be adjusted through calibration vary depending on the rating calculation settings (ratings to calculate / calculation method). Depending on the settings, it is divided into (1) Adjust both element + overall ratings or (2) Adjust overall rating only.
1. When both element ratings and overall ratings can be calibrated
If any of the following conditions apply, you can adjust both element ratings and overall ratings during calibration.
The ratings to calculate are set to 'Element ratings, Overall rating', and
The overall rating calculation method is one of the following
When the overall rating is calculated by combining competency ratings and performance ratings using a matrix
When the overall rating is calculated by combining competency ratings and performance ratings using a matrix
The overall evaluation rating calculation method in the per-step evaluation form settings is set to [Matrix combination] of evaluation elements
Or, after turning off the overall evaluation rating in the per-step evaluation form settings
In the rating calculation step, the overall rating calculation method is set to [Matrix combination of evaluation element ratings]
When the overall rating is calculated by applying weighted scores to each element rating
When the overall rating is calculated by applying weighted scores to each element rating
In the rating calculation step, the overall rating calculation method is set to [Weighted calculation of evaluation elements]
When ratings are not used in per-step evaluation forms, but both element ratings and overall rating are configured to be calculated in the rating calculation step
When ratings are not used in per-step evaluation forms, but both element ratings and overall rating are configured to be calculated in the rating calculation step
Rating not used setting in per-step evaluation form settings
In the rating calculation step, the ratings to calculate are set to produce both element ratings and overall rating
2. When only the overall rating can be calibrated
In the following cases, only the overall rating can be adjusted during calibration.
When the ratings to calculate are set to 'Overall rating only'
Or even if the ratings to calculate are set to 'Element ratings, Overall rating',
when the overall rating calculation method is a method that does not support element rating adjustment (e.g., step-by-step weighted calculation, reflecting the last rating from the lower step, etc.)
→ In this case, even if element ratings are calculated together, the calibration adjustment target is limited to the overall rating.
Conducting Calibration
Click [Calibration] - [New Rating Adjustment Session] at the bottom of the evaluation management screen.
The calibration session name can be set to a maximum of 100 characters.
Multiple calibration sessions can be added to a single evaluation.
After adding a calibration session, click [Select Now] to add adjustment targets and proceed with the adjustment right away.
However, if you try to add a member to the 'Calibration Round 1' session as an adjustment target, and that member is already included as an adjustment target in another ongoing calibration session such as 'Calibration Round 2', they cannot be added. To add the same member as an adjustment target, please finalize the ongoing calibration session that already includes that member.
After reviewing the added adjustment targets, you can adjust ratings or leave adjustment comments directly on the screen.
You can also check the checkbox on the left side of the adjustment targets to adjust ratings in bulk.
When adjusting ratings, you can see how much the rating differs from the pre-adjustment rating. Also, if you adjust an element rating, the 'Overall rating' will be automatically recalculated.
Clicking the calibration [Add via Excel] button allows you to download the rating adjustment form as an Excel file.
After entering adjustment targets and adjustment details in the template and uploading it, the changes are reflected immediately.
Click the [Select Now] button to add adjustment targets on the screen, then click the Excel icon in the upper right to download the rating adjustment form (Excel). Simply enter the adjusted rating values and adjustment comments in the Excel template and upload it, and the adjustments will be reflected immediately.
The [Adjustment Comments] entered via Excel or the product are published in the evaluation report. Therefore, please describe objectively the background of why the final rating was determined this way.
Calibration adjustment comments can be up to 10,000 characters.
Once you have completed all rating adjustments and adjustment comments, click [Finalize] in the upper right to close the ongoing calibration session.
Finalized sessions cannot be deleted.
If the same target is included in multiple calibration sessions, the adjusted rating from the session that was most recently finalized is reflected as the final rating.
In other words, if you adjust and finalize again in a new session after a calibration session has been finalized, the rating result finalized in the previous session will be overwritten by the most recently finalized session's rating.
While the rating is ultimately finalized as one, the process by which that rating was derived and the reasons for adjustment are recorded step by step in the adjustment comments. For example, if calibration was conducted over multiple rounds, the Round 1 adjustment reason and Round 2 adjustment reason accumulate so you can review the background behind the final rating determination.
For adjustment comments left during calibration, you can configure when generating the evaluation report whether to make the [Adjustment Comments] visible in the report.
However, since adjustment comments may include the context of rating adjustments and internal discussion content, it is not recommended to expose them in the member report. Instead, we recommend configuring it so that adjustment comments for member ratings can only be viewed in the manager's report. This is because managers need to fully understand the background behind rating decisions to conduct more accurate and consistent interviews and communication with members.
After generating the report with adjustment comments included, click on the generated report.
You can review the adjustment comments left during calibration. Click the adjustment comments text input field to edit the adjustment comments that will appear in the report.
FAQ
What is calibration?
What is calibration?
Calibration is a process for correcting evaluation results and serves two purposes.
To reduce evaluator bias such as leniency bias and central tendency bias, thereby ensuring objectivity
To adjust ratings in line with the relative evaluation distribution ratio, with compensation linkage in mind
It is an important process in which evaluators gather to clarify the standards for talent and performance that the organization pursues and to support growth aligned with those standards — and it is indeed a process practiced by many companies including Google, Amazon, and Meta.
Did you experience any of the following inconveniences when conducting rating adjustments after the evaluation period?
Human error: The possibility of important evaluation rating data being incorrectly recorded during copy-and-paste processes
Integrated decision-making: Various pieces of information that need to be referenced for decision-making are scattered, making it difficult to make decisions
History management: Despite being sensitive evaluation data, it is difficult to manage the history of calibration reasons and pre/post ratings
Security: The possibility of spreadsheets being incorrectly shared or sensitive data being leaked
All the inconveniences you experienced while managing calibration in Excel can be resolved through the flex evaluation feature.
Can I check which administrator conducted a calibration session and when?
Can I check which administrator conducted a calibration session and when?
While calibration is in progress or a session has been finalized, hover your mouse over the 'Last adjustment date' within the session to see which administrator made adjustments and when.
Can I modify a finalized calibration?
Can I modify a finalized calibration?
A finalized calibration session cannot be modified. If you need to modify the rating of an adjustment target, please add another calibration session, add the target whose rating needs to be corrected, and conduct calibration again!