The significance of inter-rater reliability (IRR) come into front when we need evaluations from human raters. In such cases, it is crucial to ensure that the ratings provided by different raters are consistent and reliable. IRR measures the degree of agreement among raters, which is essential for validating the quality of the data collected. There are several metrics used to measure IRR, each with its own advantages and limitations. In this section, we will discuss some of the commonly used IRR metrics and their applications