This 82 page report discusses issues that impact on the validity of the Language Arts "Dimensions of Learning" standard for grading writing portfolios. The Dimensions of Learning standard was set forth by the California Learning Assessment System/Educational Testing Service Research and Development Portfolio Assessment Project.

This report was published in August of 1995. During the time this study took place the CLAS/ETS standards were in an embryonic stage of development. However the methodologies used in this study and the overall approach to the issue of authentic/valid/reliable/consistent large scale assessment of student portfolios is still extremely relevant. This report is a valuable contribution to the areas of how portfolio writing, student selection of pieces to include, rubrics, peer review, student revision and editing, use of resources, etc. effect the assessment process and in turn effect what is actually taught in the classroom. The report attempts to synthesize, compare and contrast the (1) Actual teaching/writing curriculum and approaches in four classrooms in grades 2,4,7-8 and 8 (2) Individual student responses to questions from investigators about their writing and the writing process that are designed to "elicit" the dimensions rubric and (3) Analysis of the student's portfolios.

The study takes a close look at the two fundamental parts of the Dimensions of Learning standard as applied to writing, namely: (1) Writing for Multiple Purposes, Genres, and Audiences and (2) Writing with Resources, Processes, and Reflection. Pages 4-8 provide the complete rubric description. It then analyzes these two parts of the rubric from the point of view of: (1) Opportunities to Learn (what is actually taught in the classroom) and (2) Student's Understandings/Choices for Portfolios, Resources, Processes and Reflection. The report then presents an overall summary and conclusion.

The conclusion stresses the need for sharing of information between teachers, assessment professional and students. According to the report's authors, "Too often the voices of assessment experts, teachers and children are separated- expressing themselves in distant contexts and rarely meeting in conversation…The purpose of this report is to bring these voices together in dialogue that will inform all concerned…to find points of analytical congruence that will help writers learn their craft." The goal is standards that are feasible, productive and developmentally appropriate. There is a need to balance "the vision of student choice as a desirable goal for students with what is needed to (a) benefit their growth as writers and (b) to ensure portfolio raters are provided appropriate evidence." One of the more interesting conclusions of the report was that "Using portfolio scores to measure individual student achievement seems particularly problematic, in that portfolio writing contains writing composed in complex social contexts with highly variable support from teachers, peers and parents." However, the report concludes or implies that a coherent assessment system is possible. The report concludes that teachers and students would benefit from knowing in advance how raters read and assess portfolios. Case studies of individual rater assessments and "think-alouds" may be useful vehicles. Teachers must themselves provide their own think-alouds to model for their students the process of portfolio selection, peer review of writing, etc.

What I found more interesting than the study's conclusions were the recorded observations of teachers and students in the classroom as they attempted to teach and learn writing as a process and adapt to the new methods of portfolio assessment. Of particular interest was the feedback from students based on researchers' questions designed to assess student understanding and how that understanding related to classroom instruction.

Student Feedback

The report's authors tended to recommend a balance between the classical and romantic approaches to teaching composition in the classroom. However, though all teachers to some degree exhibited "balance" they clearly leaned to one or the other and this was reflected in student's responses and progress. The report's authors felt that whether the instructional approach was classical or romantic would affect the CLAS/ETS assessment, as raters themselves would be teachers who might favor one approach over the other.

Conclusion

The CLAS/ETS assessment for writing and accompanying rubric has a strong impact on what is actually taught in the classroom. Teachers will inevitably help to "prepare" their students to do well on these assessments. Obviously the reverse is true as well -the CLAS/ETS rubrics reflect what is being taught in the classroom. However there is significant variation in what is actually being taught to students and this will effect the validity of the actual test results in assessing individual student performance. In fact, since writing is contextual in nature and the role of the teacher is so pivotal -portfolio assessments are not a truly reliable evaluation of individual student performance as much as an evaluator of teacher /student /class /societal influences on student's writing.

Finally, it is clear from student feedback that they are having great difficulty in achieving the objectives of the CLAS "Dimensions in Learning" rubric for writing. Teachers need to carefully assess whether or not they, as teachers, are merely "wrapping" tried and true "classical" approaches in a "prettier" wrapper or whether they are actually teaching writing as a process for constructing meaning and developing "higher order" thinking skills. Further, a careful assessment must be made of whether and how teaching writing as a process is actually benefiting students. That is, are students able to comprehend the rubric's objectives and goals and become forceful, independent, versatile writers who use writing to construct meaning for themselves and their audience. The study suggests that teachers have a long way to go to actually achieve this. Students, to often seem either to be mimicking the process without real understanding or are attuning their effort more to teacher grades and classroom time constraints than to achieving true understanding. Are we expecting to much or are we simply not teaching well enough and in either case -can we fairly test the end results on a large scale. The study poses the right questions but offers no firm answers. Perhaps there are no answers that can be broadly applied to the diversity of California classrooms.