psyc 545 DB replies

#1 Bryan

As defined in Tobin et al. (2021), test fairness is "the extent to which a test is used in an impartial, just, and equitable way. To define this in my own words, I need to assess the theory behind test fairness. Terms such as "impartial, just, and equitable" have varying definitions depending on the assessor. A great example of this was given in our text. During the Cold War, the USSR used testing to suppress anyone who opposed the communist regime (Tobin et al., 2021). Individuals who did not pass the test were either placed in prison or sentenced to institutionalization. At face value, this would appear unfair. I would argue it to be unfair, cruel, and tyrannical. However, those who were in power saw it as a just and equitable way to ensure the sustainability of their government. We can find many arguments against this test and unfair usage, but the opinion is subjective based upon biases, moral compass, and religious ideology. 

Religious ideologies provide a certain level of protection in arguments for test fairness. As stated in Tobin et al. (2021), when a person's biases and morals are deeply rooted in religion, scripture could suggest all humans are created equally. Genesis 1:27 ( King James Bible, 1769/2017) states, "So God created man in his own image, in the image of God he created him; male and female he created them.". To take that a step farther, look no further than the Declaration of Independence which states, "We hold these truths to be self-evident, that all men are created  equal, etc.". Scientifically, this is invalid, but many interpret this scripture and document to suggests that all humans are created equal. I would argue all humans are created with "opportunity", but the psychological and physical composition of humans is not equal. If a test creator does not acknowledge inequalities in human development and ability, test fairness will be difficult to achieve. 

I have a personal experience with test fairness that shaped my life. When I was twenty-one years old, I applied for police officer. Part of the hiring process is to complete a psychological evaluation. I did not "pass" this evaluation and was determined to be a risk. The test results indicated rebellious tendencies against authority. As someone who had never been arrested and followed rules, this shook me to the core. What did they see in me that I could not? Apparently, they were wrong, or something changed in me after that test. I never gave up on my goals, and I became a police officer in 2006 and have not looked back. 

With this test in mind, how many other people have been deterred by this type of testing and chose to give up on their goals? If I had the ability, I would research the pass/fail rate of police officer applicants who have taken psychological evaluations. I would like to assess how many were justly kept out of policing, and how many unjustly were allowed into policing. I would also like to assess the subjective input that goes into these evaluations. Also, I believe that age could play a role in the fairness of this assessment. We know through scientific research the frontal lobe is not fully developed until around age twenty-four. How does this affect the outcome of these exams when a person is of legal age to become and officer at age twenty-one. Lastly, a psychological evaluation requires human input. While doctor's take an oath of fairness, personal biases may affect their findings. 

On page 9 of the research by Crocker, valid points were made in reference to the consequences of validating achievement testing. However, I believe standardized testing should be used in classrooms to assess education evolution. I also agree it is important for educators to understand the consequences of this validation testing. As noted in Crocker (2005), educators must understand the consequences of their opinions influencing the outcome of these tests. I have observed this frustration placed on social media for parents to read. This ends up in the student homes and eventually their negativity gains support from parents and students. I think this contaminates the validity of the testing. A good example of this is my son. He has heard his teachers express their disappointment in testing mandates. His mother is an advocate against them, and his performance on this testing has gradually gotten worse as he has gotten older. He doesn't see the value in standardized testing. This in itself creates invalidation as the test is not properly assessing his educational accomplishments. 

So, what is test fairness? Test fairness is the willingness to create testing with the understanding that many subjective elements will influence results and reviews. This approach must be fair, open minded, free from personal bias, and subjective to critical thinking input. 

 

References:

Crocker, L. (2005). Teaching for the test: validity, fairness, and moral action.  Educational Measurement: Issues and Practice,  22(3), 5–11.  https://doi.org/10.1111/j.1745-3992.2003.tb00132.xLinks to an external site.

King James Bible. (2017). King James Bible Online. https://www.kingjamesbibleonline.org/ (Original work published 1769)

Tobin, R. M., Schneider, W. J., & Cohen, R. J. (2021).  Psychological testing and assessment: An Introduction to Tests and Measurement. McGraw Hill LLC.

#2 Maximilian

1. Test fairness is the validation and reliability that a test is measuring what it intends to measure and able to capture similar results over the long term. A test is determined to be fair when all people who are to take the test are provided questions that are not biased or intended to deem someone as something they are not or favoring a specific group of people that the test taker may not be a part of. Fairness of a test depends on many aspects of the test construction including sensitivity to cultural factors, face validity, language, length, integrity, mental state prior to taking the test, impressions, preferences, preparedness, and more (Cohen et al., 2022). There are many factors that can impact test fairness and the differences may be as subtle as having one or two words in a line that completely derail the questions intent. Therefore, we must be diligent in creating tests that are fair, targeting only what we intend to measure, and capture the true essence of the criteria we are searching for. I have taken many assessments but fairly recently I have been giving people the Brief Cope Scale. This scale is comprised of 28 questions identifying the coping mechanisms a person uses in their life and often gets the subjective nod of approval following the debrief of results for most of my clients. To measure whether or not this test is fair would be to have people of different backgrounds, languages, age, gender, sexuality, etc take the brief cope scale. I would be interested in comparing the coping mechanisms of elite military personnel versus conventional military personnel. Following this, similar questions would be asked to people that know the test subject well (family, friends, coworkers), and those people would take the brief cope scale based on what they believe to be true of the test taker they know well. Finally, prior to the Brief Cope Scale taking place, the test taker will be asked what they believe their top coping mechanisms are on a list that is provided to them and to which extent they use these coping mechanisms. Based on their answers and the triangulation of information from those that know them, the test can be deemed appropriate or inappropriate to measure coping mechanisms. I believe that the elite personnel will have more solution oriented coping mechanisms compared to the conventional military service members. As stated in the English Standard Version, "Open your mouth, judge righteously, defend the rights of the poor and needy."

2. Crocker (2003) provides a thought provoking case for why consequential validity studies are necessary. By giving or requiring a test, we may unconsciously create a likelihood for people to feel the need to teach to the test so that a positive change is viewed. There are a myriad of effects that take place when a test is given. Therefore, if consequential validation studies take place prior to investing substantial resources in studies then we will be better off for it (Crocker, 2003). This will most certainly be a worthwhile endeavor but with the expectation that the tests may cause more harm than good. Additionally, there are differences between psychological and educational contexts as what is measured is different and so are the outcomes. In education, comprehension and understanding of concepts and logic are challenged and tested while in psychological work, someone's health, well being, and psychological properties are emphasized. The impacts for both assessments are different and therefore should be worthwhile but still treated differently.

 

References:

Cohen, R. J., Schneider, W. J., & Tobin, R. M. (2022).  Psychological testing and assessment: An introduction to tests and measurement. McGraw Hill.

Crocker, L. (2003), Teaching for the Test: Validity, Fairness, and Moral Action. Educational Measurement: Issues and Practice, 22: 5-11.  https://doi.org/10.1111/j.1745-3992.2003.tb00132.xLinks to an external site.

English Standard Version Bible. (2001). ESV Online. https://www.openbible.info/topics/truth