SHORT 500 WORD ESSAY

TECHNICAL NOTE

PATHOLOGY/BIOLOGY

Marta Nawrocka,1; Katarzyna Fraztczak, 1 M.Sc.; and Szymon Matuszewski,1 Ph.D.

Inter-Rater Reliability of Total Body Score—A Scale for Quantification of Corpse Decomposition

ABSTRACT: The degree of body decomposition can be quantified using Total Body Score (TBS), a scale frequently used in taphonomic or entomological studies of decomposition. Here, the inter-rater reliability of the scale is analyzed. The study was made on 120 laymen, which were trained in the use of the scale. Participants scored decomposition of pig carcasses from photographs. It was found that the scale, when used by different people, gives homogeneous results irrespective of the user qualifications (the Krippendorff’s alfa for all participants was 0.818). The study also indicated that carcasses in advanced decomposition receive significantly less accurate scores. Moreover, it was found that scores for cadavers in mosaic decomposition (i.e., representing signs of at least two stages of decomposition) are less accurate. These results demonstrate that the scale may be regarded as inter-rater reliable. Some propositions for refinement of the scale were also discussed.

KEYWORDS: forensic science, forensic taphonomy, corpse decomposition, Total Body Score, reliability, laymen

The degree of body decomposition can be quantified using the Total Body Score (TBS). The scale is a part of the method for the estimation of postmortem interval (PMI) from decomposed human remains (1). Moreover, it was often applied during decomposition studies in forensic taphonomy or entomology (2– 13). Its bases were created by Galloway et al. (14), who distin- guished five stages of decomposition (called major categories of decomposition) and several secondary categories of decomposi- tion within. These major and secondary categories were used by Megyesi et al. (1) in the first scale for cadaver decomposition called the TBS. The study of Megyesi et al. (1) was aimed at the development of the method for PMI estimation from TBS and temperature. For this purpose, authors divided a cadaver into three areas (head with neck, trunk, and limbs), for which they developed separate lists of secondary categories of decomposi- tion. The sequence of categories in the scale reflected a natural sequence of decomposition, allowing it to assign points to speci- fic categories and eventually to score decomposition. The method includes determination of the degree of decomposition in each of the prespecified areas, assignment of points according to the scale, and finally addition of points for the TBS calcula- tion. To apply the scale in studies on decomposition of animal car-

casses, it was modified several times. Adlam and Simmons (2) adapted it to study the effect of repeated physical disturbance on soft tissue decomposition of rabbit carcasses. Myburgh and col- leagues (10) adjusted the scale to pig carcasses. Although the scale is relatively new, it was used by many researchers in their

studies (2–13). The scale was, however, not subjected to the rigid scientific scrutiny. To determine the usefulness of a new method for the broad

forensic application, it is necessary, among other things, to test whether the method is reliable. In this respect a key attribute a new method should possess is high agreement of results achieved by different users. This kind of reliability is called the inter-rater reliability, and it pertains to precision of results within a group of users (15,16). It depends on the language in which the method is expressed, variables (qualitative or quantitative) it incorporates, and measurements involved. Because the TBS scale uses qualitative categories of decomposition, it may be expected that different people may differently recognize and apply these categories. Consequently, scores of different users may vary and the scale itself may have low inter-rater reliability. This effect may be particularly profound with indicators relating to cadaver tissue colors, extent of maggot activity, percentage of bone exposure, extent of flesh desiccation, and dryness of bones. The aim of this study was to determine the degree of inter-

rater reliability of the TBS scale. It was decided to examine the reliability of the scale as used by trained laymen. Although the scale is most frequently used by anthropologists in their research on decomposition (2–12), it is also applied by forensic entomol- ogists (13). The scale is suitable for a broad forensic use and, using laymen, we wanted to test its reliability when applied by nonprofessionals. With this study design, we tried to demonstrate that the scale is reliable when used by different raters and to substantiate its broad use by forensic community.

Materials and Methods

The experiment was made using photographs of pig carcasses in different states of decomposition. Tables with the scale points

1Laboratory of Criminalistics, Adam Mickiewicz University, �Sw, Marcin 90, 61-809 Pozna�n, Poland.

Received 10 Nov. 2014; and in revised form 22 Feb. 2015; accepted 17 April 2015.

798 © 2016 American Academy of Forensic Sciences

J Forensic Sci, May 2016, Vol. 61, No. 3 doi: 10.1111/1556-4029.13105

Available online at: onlinelibrary.wiley.com

were created for each of the predefined areas (head with neck, trunk, and limbs, Tables 1-3). The scale used was a compilation of earlier scales (1,2,10). It included categories of decomposition and associated indicators of decomposition along with relevant points (Tables 1-3). Major categories of decomposition sensu Galloway et al. (14) are named throughout this manuscript with the term “stage of decomposition.” Four such stages were distin- guished (i.e., fresh stage, early decomposition stage, advanced decomposition stage, and skeletonization stage). Twelve photographs were selected of pig carcasses in various

stages of decomposition. Photographs represented two levels of difficulty in scoring TBS. The level of difficulty was judged by the authors of this article and then six easy and six difficult pho- tographs were chosen to be included in the study. Difficulty was evaluated according to the number of indicators available to be judged from the photograph and the number of stages suggested by these indicators. Easy photographs showed carcasses entirely in a single stage of decomposition, that is head and neck, the whole trunk, and all limbs in just one stage of decomposition (Fig. 1). Difficult photographs displayed carcasses representing signs of at least two stages of decomposition (Fig. 2). Subjects were not aware of these categories of photographs. The study was conducted using 120 persons: 40 students each

of law, biology, and medicine. Students represented different faculties which were geographically separated, as we wanted to have broad representation of laymen and at the same time to minimize chances of confounding contacts between participants. The first part of the study took place using a group of medical students, on 23rd of May 2014, at the Medical University in Bydgoszcz. Next part took place on 3rd of June 2014 among students of law at the Faculty of Law and Administration of Adam Mickiewicz University in Pozna�n. The last part took place among students of biology at the Faculty of Biology of Adam Mickiewicz University on 6th of June 2014. Participants were

between 22 and 26 years of age, and there was equal contribu- tion of sexes. Each participant received tables with TBS scale (Tables 1-3),

12 photographs, and a result card. All groups had undergone training, which included presentation of the complete decompo- sition process of a pig carcass along with the discussion of all visible indicators and the scale. Afterward, participants were asked to familiarize themselves with other materials and then to quantify decomposition of carcasses in the photographs by using the scale. Additionally, brief notes were put next to the pho- tographs to inform participants of the features which were pre- sent but were not visible in the photographs such as the presence of maggots internally (see caption to Fig. 2). These fea- tures were determined during pig carcass experiment along with

TABLE 1––Stages of decomposition, secondary categories of decomposition, and corresponding scores (points) for the head and neck.

Stage of Decomposition* Points Secondary Categories of Decomposition

Fresh 1 Fresh, no discoloration Early decomposition

2 Pink-white appearance with skin slippage and some hair loss, maggots visible

3 Gray to green discoloration, some flesh still relatively fresh, maggots visible

4 Discoloration, brownish, drying of nose and ears, heavy maggot activity

5 Purging of decompositional fluids out of eyes, ears, nose, and mouth, some bloating may be present

6 Skin brown to black Advanced decomposition

7 Caving in of the flesh and tissues of eyes and throat

8 Moist decomposition, bone exposure <50% scored area

9 Mummification (desiccation), bone exposure <50% scored area

Skeletonization 10 Bone exposure >50% scored area with greasy substances and decomposed tissue

11 Bone exposure >50% scored area with desiccated or mummified tissue

12 Bones largely dry, but retaining some grease 13 Dry bones

*Stages of decomposition are major categories of decomposition sensu Galloway et al. (14).

TABLE 2––Stages of decomposition, secondary categories of decomposition, and corresponding scores (points) for the trunk.

Stage of Decomposition* Points Secondary Categories of Decomposition

Fresh 1 Fresh, no discoloration Early decomposition

2 Pink-white appearance with skin slippage and marbling present, few maggots

3 Gray to green discoloration, some flesh still relatively fresh, maggots visible

4 Bloating with green discoloration and purging of decompositional fluids, heavy maggot activity

5 Postbloating following release of the abdominal gases, with discoloration changing from green to black, heavy maggot activity

Advanced decomposition

6 Decomposition of tissue producing sagging of flesh, caving in of the abdominal cavity

7 Moist decomposition, bone exposure <50% scored area

8 Mummification (desiccation), bone exposure <50% scored area

Skeletonization 9 Bones with decomposed tissue, sometimes with body fluids and grease still present

10 Bones with desiccated or mummified tissue covering <50% scored area

11 Bones largely dry, but retaining some grease 12 Dry bones

*Stages of decomposition are major categories of decomposition sensu Galloway et al. (14).

TABLE 3––Stages of decomposition, secondary categories of decomposition, and corresponding scores (points) for the limbs.

Stage of Decomposition* Points Secondary Categories of Decomposition

Fresh 1 Fresh, no discoloration Early decomposition

2 Pink-white appearance with skin slippage, few maggots

3 Gray to green discoloration, some flesh still relatively fresh and marbling present

4 Discoloration, brownish, drying of extremities 5 Skin brown to black

Advanced decomposition

6 Moist decomposition, bone exposure <50% scored area

7 Mummification (desiccation), bone exposure <50% scored area

Skeletonization 8 Bone exposure >50% scored area, some decomposed tissue and body fluids remaining

9 Bones largely dry, but retaining some grease 10 Dry bones

*Stages of decomposition are major categories of decomposition sensu Galloway et al. (14).

NAWROCKA ET AL. . INTER-RATER RELIABILITY OF TOTAL BODY SCORE 799

the photographs. We predicted that scoring TBS from pho- tographs will be less accurate than from an examination of an actual cadaver. To reduce these inaccuracies, it was decided to provide participants with information, which would usually be available upon the examination of the cadaver. Two photographs were selected for the fresh stage and the skeletonization stage, three for the early decomposition stage, and five for the advanced decomposition stage. Following training, students were given 40 min to score photographs. Results were tested with the Krippendorff’s alpha statistic,

which is used to estimate the degree of agreement between scores that different people assign to the same objects (17). It usually takes values between 0 and 1: the former indicating the absence of agreement (i.e., when scores are statistically unrelated to the objects) and the latter indicating perfect agreement (i.e., when scores are perfectly related to the objects) (17,18). The closer a is to 1, the more reliable the scale. It may also take val- ues below 0, which indicates systematic disagreement among raters. The Krippendorff’s a is considered as the standard relia- bility measure (17). The statistic for ratio data was calculated separately for each group of participants and also over the whole group of participants (the ReCal software, dfreelon.org, [19]). To

analyze error rates in TBS, scores of participants were compared with the best scores as derived by consensus of the authors. Sig- nificance of differences in the error of TBS between easy and difficult photographs was evaluated with the Wilcoxon signed- rank test. Significance of differences in the error of TBS between carcasses from different stages of decomposition was tested with the Friedman rank test. Absolute errors were used in both tests. Multiple pairwise comparisons were made using the Dunn test. Nonparametric tests were chosen, because it was pre- dicted that assumptions for parametric tests will not be met. Analyses were made using Statistica 10 and Statistica Medical Set (StatSoft Inc. 2011) at 5% level of significance, which is widely accepted in scientific studies.

Results and Discussion

Krippendorff’s a for all participants was 0.818, while in par- ticular groups, it was slightly higher (a = 0.845, 0.848, and 0.840, respectively, for students of medicine, law, and biology). These results indicate that the TBS scale, when used by different people, gives homogeneous results irrespective of the user quali- fications. Although the study was made on a group of laymen who had undergone only a basic training, the scores of partici- pants were concordant. It is probable that the agreement of scores given by experts would be even higher. Similarly, if par- ticipants would examine real cadavers instead of photographs, we suspect that scores of different raters would be more similar. Accordingly, the use of the scale can be considered to ensure a high degree of agreement between results of different raters. Current results thus indicate that inter-rater reliability of the TBS scale is of an acceptable level, and from this point of view, the scale may be safely used to quantify decomposition. The values of a as recorded in this study were, however, only slightly above the limit suggested by Krippendorff (18) for reliable data, scales, or methods (a ≥ 0.8). Therefore, the scale is not a perfect one and some refinement will be needed. Significant differences in the error rate of TBS were found

between carcasses in different stages of decomposition (Fried- man ANOVA, v2 = 88.2, p < 0.01). Scores for carcasses in the advanced decomposition stage were significantly less accurate than scores for carcasses in the other stages (Fig. 3). The aver- age error rates for most photographs ranged between 2 and 3.5 points (Table 4). Scores were usually overestimated for carcasses in the first stages of decomposition, whereas they were underes- timated for carcasses in the later stages (Table 4). These results indicate that scoring decomposition is similarly difficult in dif- ferent stages of decay and only cadavers in the advanced decom- position may pose larger problems in this respect. Cadavers at the beginning of advanced decay usually represent a mosaic of features characteristic of active and advanced decomposition (20). Participants probably found it difficult to score such car- casses, and this may explain poorer accuracy of scores allocated to carcasses in the advanced decay. Moreover, indicators related to the advanced decomposition stage seem to be more challeng- ing. A poststudy survey of participant opinion revealed that for all areas of the body, the same indicators were the hardest in use, namely “moist decomposition” and “bone exposure <50% of the scored area.” These results suggest that participants had difficulties with the consistent evaluation of decomposition in percentage terms. From the broader perspective, this finding sup- ports the view that percentages as used to score decomposition may lead to inconsistencies among raters. Accordingly, methods involving percentages, as, for example, the method of Vass (21)

FIG. 1––Example of easy photograph. The whole carcass is in the fresh stage of decomposition. Only one indicator is available to the rater, that is lack of discoloration (fresh appearance of a carcass).

FIG. 2––Example of difficult photograph. Within the head and neck area, some parts are still in early decomposition, others in advanced decomposi- tion, similarly within the trunk and limb areas. Several indicators are avail- able to the rater, that is maggot activity, tissue discoloration, bone exposure, caving in of the flesh, and others. A brief note was attached to this photo- graph. It contained information “maggots feeding in the head and fore groin area.”

800 JOURNAL OF FORENSIC SCIENCES

for estimating the extent of soft tissue decomposition, may lack inter-rater reliability. It was also found that the error rate for easy photographs was

lower than the error rate for difficult photographs (Wilcoxon signed-rank test, z = 7.92, p < 0.01, Fig. 4). This result indicates that cadavers in a mosaic decomposition (i.e., representing signs of at least two stages of decomposition) pose larger problems while scoring decay compared to cadavers entirely in a single stage. This is probably the consequence of larger number of indicators available to a rater and as a result, larger probability that inconsistent scores will be suggested by particular indica- tors. It seems, however, that more emphasis on these issues while teaching the TBS scale may improve performance of the raters. Moreover, current results suggest that some indicators as pro-

posed by Galloway et al. (14) and adopted by Megyesi et al. (1) and followers (2–10) may not warrant sufficient consistency in scores of different users. Apart from bone exposure, several indi- cators specified with vague phrases (e.g., “heavy maggot activ- ity” or “some flesh relatively fresh”) may result in

inconsistencies. The variation in scores may also result from indicators involving evaluation of colors, as corpses often repre- sent a mosaic of different colors. We think that refinement of such weak indicators would improve the performance of the scale. As for the extent of bone exposure, we think that its esti- mate in the percentage terms should be replaced with a more qualitative measure, involving determination of particular bones which are exposed (e.g., the refined scale for the trunk would contain the following indicators: “exposure of ribs and sternum,” “exposure of thoracic vertebrae,” and “exposure of lumbar verte- brae”). Vague phrases could be similarly refined. Extremely vague ones (e.g., “some flesh relatively fresh”) should be, how- ever, dropped. Another point for refinement is indicators involv- ing evaluation of tissue colors. Probably, they may be reduced to just one or two, without sacrificing the resolution of the scale. This study shows that method of quantifying the degree of

cadaver decomposition with the use of the TBS scale may be regarded as reasonably reliable. The agreement of scores as given by different raters was, however, far from perfect, which implies that there is a need for improvement of the scale.

FIG. 3––Median and quartiles of absolute error for scores given by partic- ipants to pig carcasses in different stages of decomposition. Different letters denote significant differences in pairwise comparisons (Dunn test; absolute differences between mean ranks were significant at 5% level of significance, if they were larger than 0.439).

TABLE 4––Accuracy of scores given by participants to pig carcasses in different degrees of decomposition.

Carcass The Best TBS*

Errors of TBS Given by Participants

Mean† Cases of Underestimation Maximal Underestimation Cases of Overestimation Maximal Overestimation

1 23 3.0 42 �14 70 9 2 14 2.2 88 �9 13 4 3 22 5.5 72 �18 46 13 4 3 3.4 0 – 104 10 5 18 3.0 72 �8 33 11 6 19 3.1 89 �8 24 12 7 3 1.1 0 – 46 9 8 10 2.4 29 �6 74 10 9 16 2.5 60 �7 44 9 10 19 3.4 29 �5 79 11 11 28 2.8 47 �14 57 7 12 35 1.3 47 �19 0 –

*The Total Body Score derived by consensus of the authors of this article. †Mean was calculated from absolute values of errors.

FIG. 4––Median and quartiles of absolute error for scores given by partic- ipants to pig carcasses differing according to the level of difficulty in scoring decomposition.

NAWROCKA ET AL. . INTER-RATER RELIABILITY OF TOTAL BODY SCORE 801

Refinement of vague or difficult indicators and elaboration of new indicators would serve this purpose.

Acknowledgments

We would like to thank all participants for their invaluable contribution to this study. We are also grateful to anonymous reviewers for their comments and suggestions, which helped to improve the manuscript.

References

1. Megyesi MS, Nawrocki SP, Haskell NH. Using accumulated degree-days to estimate the postmortem interval from decomposed human remains. J Forensic Sci 2005;50(3):618–26.

2. Adlam RE, Simmons T. The effect of repeated physical disturbance on soft tissue decomposition–are taphonomic studies an accurate reflection of decomposition? J Forensic Sci 2007;52(5):1007–14.

3. Simmons T, Adlam RE, Moffatt C. Debugging decomposition data–com- parative taphonomic studies and the influence of insects and carcass size on decomposition rate. J Forensic Sci 2010;55(1):8–13.

4. Cross P, Simmons T. The influence of penetrative trauma on the rate of decomposition. J Forensic Sci 2010;55(2):295–301.

5. Simmons T, Cross PA, Adlam RE, Moffatt C. The influence of insects on decomposition rate in buried and surface remains. J Forensic Sci 2010;55(4):889–92.

6. Bachmann J, Simmons T. The influence of preburial insect access on the decomposition rate. J Forensic Sci 2010;55(4):893–900.

7. Connie L, Parks MA. A study of the human decomposition sequence in central texas. J Forensic Sci 2011;56(1):19–22.

8. Spicka A, Johnson R, Bushing J, Higley LG, Carter DO. Carcass mass can influence rate of decomposition and release of ninhydrin-reactive nitrogen into gravesoil. Forensic Sci Int 2011;209(1–3):80–5.

9. Gruenthal A, Moffatt C, Simmons T. Differential decomposition patterns in charred versus un-charred remains. J Forensic Sci 2012;57 (1):12–8.

10. Myburgh J, L’Abbe EN, Steyn M, Becker PJ. Estimating the postmortem interval (PMI) using accumulated degree-days (ADD) in a temperate region of South Africa. Forensic Sci Int 2013;3:165e1–6.

11. Sutherland A, Myburgh J, Steyn M, Becker PJ. The effect of body size on the rate of decomposition in a temperate region of South Africa. Forensic Sci Int 2013;231(1–3):257–62.

12. Meyer J, Anderson B, Carter DO. Seasonal variation of carcass decom- position and gravesoil chemistry in a cold (Dfa) climate. J Forensic Sci 2013;58(5):1175–82.

13. Matuszewski S, Konwerski S, Fraztczak K, Szafałowicz M. Effect of body mass and clothing on decomposition of pig carcasses. Int J Legal Med 2014;128:1039–48.

14. Galloway A, Birkby WH, Jones AM, Henry TE, Parks BO. Decay rates of human remains in an arid environment. J Forensic Sci 1989;34(3):607–16.

15. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med 2005;13(5):360–3.

16. Gwet KL. Computing inter-rater reliability and its variance in the pres- ence of high agreement. Math Stat Psychol 2008;61(1):29–48.

17. Hayes AF, Krippendorff K. Answering the call for a standard reliability measure for coding data. Commun Methods Meas 2007;1(1):77–89.

18. Krippendorff K. Reliability in content analysis: some common miscon- ceptions and recommendations. Hum Commun Res 2004;30(3):411–33.

19. Freelon D. ReCal OIR: ordinal, interval, and ratio intercoder reliability as a web service. Int J Internet Sci 2013;8(1):10–6.

20. Matuszewski S, Bajerlein D, Konwerski S, Szpila K. Insect succession and carrion decomposition in selected forests of Central Europe. Part 1: pattern and rate of decomposition. Forensic Sci Int 2010;194:85–93.

21. Vass A. The elusive universal post-mortem interval formula. Forensic Sci Int 2011;204:34–40.

Additional information and reprint requests: Szymon Matuszewski, Ph.D. Laboratory of Criminalistics Adam Mickiewicz University �Sw. Marcin 90 61-809 Pozna�n Poland E-mail: [email protected]

802 JOURNAL OF FORENSIC SCIENCES

Copyright of Journal of Forensic Sciences (Wiley-Blackwell) is the property of Wiley- Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.