Standards and Bilingual and ESL Assessment

Examples from classmates responding

1)

1) The ACCESS test is a standardized test, whereas assessing students' language proficiency through language samples or interviews is not. A standardized test is not going to have the same authenticity as a sample or interview might. An interview allows for reciprocity, so the teacher/interviewer can give wait time for responses and provide feedback. Additionally, a standardized test, like the ACCESS, requires a certain level of background knowledge. If a student has not encountered a "basket," which is one of the words used in the Reading assessment, then they may be unable to answer a question correctly due to not having the prior knowledge in either their L1 or L2. In other words, they may be able to read or listen, but if they don't have the knowledge of certain vocabulary, they will be limited. An interviewer can shift and adjust as needed. Finally, the level of engagement is different with a standardized assessment, like the ACCESS test, versus an interview or language sample. Students who struggle with focusing may miss Speaking questions on the ACCESS that will not be repeated. A face-to-face interview is more likely to maintain engagement and participation.

2) First, the SOLOM Matrix is not entirely reliable. Unfortunately, there could be discrepancies between assessors. What one teacher may score as a 3 in fluency could be interpreted as a 4 by another, especially when it can be difficult to measure a student's "understanding." Fortunately, though, it has a strong level of authenticity because it allows for observation of both informal and formal language tasks. Tasks can be identified to allow the assessment to be interactive, but this leaves a responsibility to the assessor. This is similar to construct validity. It is the assessor's responsibility to look at the student's strengths and opportunities for improvement in order to make the assessment meaningful. It certainly has the power to provide for next steps for the teacher. I believe the overall impact is positive, as it is not a high-stakes, standardized test and can take place informally. When assessment is informal, it takes the pressure off of students and they can perform more authentically instead of in response to test questions that may or may not be designed for students like them. Ultimately, the teacher must consider the student, their age level, and cognitive ability and design tasks that can be meaningfully assessed in order for the SOLOM to be useful.

2) After looking through the ACCESS tests at the high school level, the key difference I have found when comparing them to language samples or interviews is the freedom that students would have when it comes to samples and interviews. For example, the ACCESS tests have very prescribed content that cannot change based on the individual and their experiences, but samples and interviews are going to have a lot more to do with that individual’s actual experiences and life.  This standardized form of assessment could prove itself to unjustly evaluate students based on their personal or cultural background.

 

The SOLOM matrix has a variety of strengths and weaknesses that are evident.  The biggest strength in my opinion is how a student’s oral language proficiency can be broken down into five main categories- comprehension, fluency, vocabulary, pronunciation, and grammar.  Breaking oral language into these categories can allow the teacher to compartmentalize each of these facets and clearly rate them based on the performance indicators in each box of the matrix. Another strength of the SOLOM matrix is the ease at which one can use the matrix-- simply identifying the performance indicator that most closely describes what they heard.  Lastly, this type of assessment tool is able to be extremely precise-- meaning that the results would be consistent. If this tool is used every time, it should be quite logical to track improvement throughout a student’s learning career.  The only glaring weakness that I can see in the SOLOM matrix is the idea that the total score can give a final level by which students can be classified, but, unless the entire matrix with the sub categorical ratings is included, it would be impossible to track improvement in any of the five areas of oral language performance.

The ACCESS test most definitely does not give ELLs the freedom to "show what they know", so I am in complete agreement with you on that point. I did not think of the ACCESS test in terms of not doing each student a favor in regards to their cultural or personal background. I do think that is another huge difference. During the standardized type tests, there is no room, like you said, for clarification or for familiarity for some students. Where during an interview, the interviewer has the ability to adapt and change on the go in the questions that are being asked from the given responses.

For the SOLOM matrix, I love the strength that you mentioned of how one can use the matrix with such ease. I went through an example of the matrix on the website, and did not even stop and think of how user friendly it was and somewhat easy to figure out when going through it for the first time. It is very explicit within the five categories of what you should be looking for. The only weakness I worry about, that I mentioned in my post, is the reliability of it. I know it is pretty straightforward, but I feel like there is always that gap and room for misinterpretation between scorers.

Question #1

The ACCESS Test differs from testing students using language samples and interviews in that it is a computer adaptable test. ACCESS uses visuals and scenarios that may be unfamiliar to the ELLs.  Although context is given it is not authentic context.  Assessing students through interviews allow the teacher/test administrator the opportunity to choose contextual conversations based on the student’s proficiency level to see if growth has occurred.  After taking the ACCESS practice test, I empathize with students at my school (Cass Tech).  The students often have to be tracked down to test and do not take the test seriously. If I am administering the test, I tell them that they are testing year after year because they have not demonstrated proficiency on the test.  Most of the ELLs were born in the U.S. and can read, write, speak, and listen proficiently. They feel belittled taking the test year after year.  These students are often in honors, advanced placement, and international baccalaureate classes. Due to funds being tied to testing, we do not promote students opting out of the test, but we do let them know that opting out is an option.  It would be beneficial if the student would only have to retest the portion of the test, they did not score at the level 6.

Question #2

The strengths of the SOLOM matrix are:  1.  The assessment task is authentic because it captures the student’s natural language sample. 2.  Construct validity can be obtained by the teacher focusing on meaningful and appropriate open-ended questions. 3.  It is interactive because the student is using language and skills are being assessed.  4.  Another strength of the SOLOM matrix is that it is adaptable to multiple languages.

The weakness of the SOLOM matrix are:  1.  Reliability – a teacher may assess a student at a higher or lower ability level based upon familiarity with the student. The results may not be consistent.  2.  It is not practical.  This form of testing is very time consuming.  3.  This test matrix only assess oral language.  You will need another test for reading and writing.

One of the main differences between the ACCESS test and language samples or interviews is in the formality of it all. The ACCESS test is more formal and standardized, trying to be a one-size fits all for every student, but that is pretty much what standardized tests are not. They do not give any room for personality or for students to show or express their utmost potential. On the other hand, language samples or interviews allow for variation in the tracking of proficiency for the students. I watched a video on Enrique, from the Purdue College of Education website, during an interview he was having. The interview was very informal and asked questions that Enrique would be able to elaborate on. Even though he did not answer directly to what question was being asked every time, his abilities were still able to be accessed. I think that creating the informality of certain assessments is beneficial to the confidence of the student as well as giving them the freedom to fully and explicitly present what they know and what they are capable of. When reading the chapter by Solano-Flores and Trumbell, they state that testing ELLs in only one language will only give you a partial picture of their academic achievement. This is another huge difference between the ACCESS test and an interview. In an interview, they allow students the freedom to fluently change between languages in the case that they are in need of vocabulary help.

The SOLOM matrix has many strengths to it that go hand in hand with the usefulness framework that was mentioned in our class presentation as well as mentioned by McKay. One strength is that it is valid. I think because the matrix is broken down into five categories, this allows for the scorer to see and/or listen for exactly what they need to. The matrix is measuring exactly what it needs to be. If the SOLOM matrix is being used through an interview, I think a strength could be the authenticity of it. Through the video I watched, it was obvious that the questions being asked were relating to the interviewees life and that they had a general knowledge of the questions being asked. One weakness of the SOLOM matrix is the reliability of it. I find it difficult to be able to proctor assessments and assess them the same as colleagues. From working in the school, I can already tell that there are certain aspects that I might find more valuable in learning a language than do others. This could also be the case when assessing ELLs. I think it would be beneficial to have more criteria on what is being assessed to make the assessments more reliable. That is one of the main weaknesses of the matrix. Other than that, I think the matrix is a good way to assess where students are at and what areas of support they may need from that point on.