case study.

Running head: JOURNAL CRITIQUE 1

Journal Critique - Assessing School Effectiveness

JOURNAL CRITIQUE 2

Summary of the article

The program being evaluated by the authors was a Collegiate Learning Assessment

(CLA) program. This article was based on the criticism regarding the students taking the

Collegiate Learning Assessment (CLA) program which is used in the measurement of the value

added in the learning institutions through carrying out a test on the capability of the freshmen

and seniors to the logical thinking and clear writing. The methods used by the Collegiate

Learning Assessment to help in the determination of the value added were outlined. Authors of

this article provided some of the criticism being provided by individuals especially when it

comes to the computation of the value added. The responses to the concerns raised were also

provided.

According to this article, CLA program currently involves the use of the linear regression

model and in this case, the school is considered to be the unit of assessment whereas the school

means SAT score is the sole explanatory variable. Authors of this article responded to the

criticism by indicating that students who are participating in the CLA program tend to be

identical to their fellow student based on the measurements which were observed by the authors.

Also, the senior participants are similar to the participating freshmen and immediately when

there is a control for SAT scores, CLA tasks performance is unrelated to the aspects of the task,

student academic major, the demographics of the learners, and student' features like size. The

disapprovals of the CLA procedure are not being reinforced by data. In addition, the problems

which are created by the confounding, as well as the selection bias, are affecting different

categories of research programs. CLA is lucky due to its ability to address the challenges faced.

JOURNAL CRITIQUE 3

Evaluation goals

The goal of Collegiate Learning Assessment (CLA) program which was under

examination by the authors of the article is to offer colleges as well as universities with some

information concerning how the level of improvement which has been between the freshmen and

the senior years. The information is also on whether there is more or reduced enhancement as

compared to the expectation with regard to the progress being made by the students in other

learning institutions. The information being provided is aimed at supplementing as opposed to

the replacement on the mechanism through which schools are assessing the outcome of

education. Since CLA outcome meant for the internal use, CLA program is not publishing

personal or the school level of data.

CLA program is also measuring the value added by carrying out a comparison of the

freshmen and the senior tests which is used for the assessment of the skills to be gained by the

students. The skills include those which are used by the student in solving the problems, making

analytic reasoning, skills for the critical thinking, and skills meant for writing. Therefore,

through this goal, it has drawn focus and there are a number of concerns which have been raised

concerning the technique being used by the CLA program in the estimation of the value added.

The skills that CLA is focusing on are used in a broad series of educational majors and are

always appreciated by managers in many organizations. The measurement of the value added by

an educational program is based on looking at the contribution which has been made by each

learning institution towards the learning processes of the students in the areas where testing is

being done.

JOURNAL CRITIQUE 4

Theoretical concepts

There is no evidence of the theoretical concept which has been used by the authors to

connect the program to the goals or outcome.

Research Methods

The method used by the authors of this article was first based on giving a description of

the techniques used by the CLA to compute value added. This was followed by giving a

summary on some of the criticism regarding the question of which students are taking the CLA

test. According to the author of this article, it was revealed from the methods used that samples

are always not random and this, therefore, implies that there is the existence of concern in

relation to the selection bias.

Regarding the methods of giving a summary on the criticism in relation to the questions

with the students who are taking CLA test, the authors of this article get the opportunity of

illustrating how the students who are taking CLA tests tend to be identical to their classmates

with regard to the SAT scores and other background features. This method also enabled the

authors of this article to prove that participating seniors tend to be very identical to the

participating freshman.

Further evidence from the provision of the summary regarding the question on the

student taking CLA tests is that as soon as there is presence of SAT score controls, the CLA

tasks performance is not linked to the areas of the tasks, the learners academic major, the

JOURNAL CRITIQUE 5

demographic features of the learners, and the characteristic like size of the learning institution.

Therefore, based on the criticism procedure adopted by the authors, it was also evident that

reproaches of the CLA procedure are not completely reinforced by the presence of data. Another

method used by the authors is giving an outline of the research which has been carried out by the

CLA, for example, the development of the new techniques used in the computation of value

added.

Conclusion

The conclusion of the authors of this article is that learners who are participating in the

CLA program tend to be identical to their fellow students with regard to the dimension examined

by the authors. The variations in the background features between the participants and the non-

participants are always smaller due to the greater variation between the seniors and the freshmen

within the CLA scores. The addition of different categories of variables to the regression

calculation which consist of the SAT scores is unlikely to enhance the accuracy when it comes to

the prediction of the CLA scores.

Even though there is a possibility of having differences in selection bias, the adequate

data is an indication that biases are too minute to matter. There is a lack of feasibility in the

random selection of the participants. Longitudinal designs appear not to be perfect when

compared to the present cross-sectional designs. In addition to the concerns related to the

selection bias, there are other concerns which have been made by the critics of the CLA and

these concerns are related to the maturation, matric sampling of the measurements, and the

stacking of the deck which appears to be a lacking basis. Therefore, problems are being created

by the confounding selection bias which is affecting several types of research programs.

JOURNAL CRITIQUE 6

Evaluation Concepts

The concept which is applicable to the article is the process evaluation. This is a concept

which is aimed at determining whether program activities have been implemented based on the

previous intention. As for the case of the article, the goal was to give a summary of the criticisms

with regard to the question surrounding the students taking CLA test. Collegiate Learning

Assessment (CLA) program is used for the measurement of the value added in the learning

institutions through performing a test of the logical thinking and the clear writing of the colleges

and university students. CLA is also used in measuring the skills of the students when it comes to

tackling problems, making logical intellectual, critical thoughts and writing. These are the major

areas which are thought to be appropriate to a wider series of educational areas and are valued by

most businesses.

Despite the goals and objectives of the CLA program in providing students with the

necessary skills needed for the professional gains, there are criticisms which have been raised

regarding the students who are taking the CLA tests. Authors of this article looked at the role of

CLA programs such as computing of the added values and evaluating some of the critics which

have been provided by the individuals who do not support this program or questioning the

effectiveness of the CLA program. Authors of this article offer some of the responses to the

opponents by indicating that learners who are taking CLA examinations are identical to their

colleagues on the SAT scores and related backgrounds features. Authors of this article also

reveal that immediately there is a control for SAT scores, the outcomes in the CLA test seem to

be dissimilar to the task content and student academic major among others.

JOURNAL CRITIQUE 7

References

Klein, S., Santa, M. C., Berkeley, C. A., Shavelson, R., Stanford, C. A., Bolus, R., et al. (2008).

Assessing school effectiveness. Evaluation Review, 32(6), 511-525.