Questions about Assessment?
Answers from Office of Superintendent of Public Instruction (OSPI)
Q: How does Washington assess its students?
A: Our state assessment system fulfills the state’s Education Reform Law of
1993 and the requirements of the federal No Child Left Behind Act. We
have two tests: the Measurements of Student Progress (MSP) for grades 3-8, and the High School Proficiency Exam
(HSPE). The tests are shorter than the former WASL test, and online
versions will be phased in over the next few years. More information
about state testing is available on our State Testing FAQs page.
Q: What are the graduation requirements?
A: All public high school students are required to meet statewide
graduation requirements in order to earn a diploma. The Washington State
Legislature requires state testing and the Washington State Board of
Education establishes minimum credit requirements, the Culminating
Project and the High School and Beyond Plan. OSPI explains the
graduation requirements in The Graduation Toolkit. The toolkit is an online resource to help educators and families understand state graduation requirements.
Q: How do I see my child’s test booklet?
A: The option to review a student's state test booklet
is available to parents and guardians of students who were enrolled
full time or part time in Washington public schools during state
testing.
Q: What is No Child Left Behind?
A: The No Child Left Behind Act of 2001 authorizes several federal education programs that are administered by the states. The law is a reauthorization of the Elementary and Secondary Education Act.
NCLB significantly raises expectations for states, local school
districts, and schools in that all students will meet or exceed state
standards in reading and mathematics by 2013-14.
Q: What are the new state tests called?
A: The grades 3-8 tests are called the Measurements of Student Progress (MSP). The high school tests will be called the High School Proficiency Exam (HSPE). These tests are under development and will incorporate the state’s academic learning requirements.
Q: When do the new tests take effect?
A: The Measurements of Student Progress (MSP) and the High School Proficiency Exam (HSPE) will first be given in spring 2010.
Q: Why are you replacing the WASL?
A: Superintendent Randy Dorn, who came into office in January 2009,
believes the state assessment system should be a less complex and more
responsive system of measuring students’ progress. For more information,
see our June 2009 press releases (June 11 and June 18).
Q: What is different about these tests compared to the WASL?
A: The tests are shorter and online versions will be phased in over the next few years.
Q: Doesn’t shortening the tests make them easier?
A: No. The entire testing system will be much more efficient, but the
tests, still based on our state learning standards, will continue to be
valid and reliable.
Q: Does my son or daughter still need to pass a state test to graduate from high school?
A: Yes. No matter what the state test is called, the graduation requirements
that go along with it have not changed. If a student passed one or more
content areas on the WASL, he/she would not have to take the HSPE in
that specific subject. For example, a high school student in 2009 passes
the reading WASL, but not the writing WASL. That student would need to
take the writing HSPE but not the reading.
Q:. What subjects are covered on our state tests?
A: The MSP and HSPE are based on the state’s learning standards contained in the Essential Academic Learning Requirements (EALRs). Students are tested in:
- Reading: Grades 3, 4, 5, 6, 7, 8 and 10
- Writing: Grades 4, 7 and 10
- Math: Grades 3, 4, 5, 6, 7, 8 and 10
- Science: Grades 5, 8 and 10
Q: What types of questions appear on state tests?
A: The MSP and HSPE tests are much shorter than the WASL and include
multiple-choice and short-answer questions. Four-point essay questions
have been eliminated on reading, math and science tests. This change
allows students to show they are able to solve the problems, while not
being scored on their writing ability on the math, reading and science
tests.
Q: What makes our state tests different from other standardized tests?
A: The MSP and HSPE are unlike more familiar standardized tests, which
measure students' performance against other students. Our state tests
measures students' performance against a set of learning standards, not
against their peers. Think of the MSP/HSPE like the test you take to
earn a driver’s license. It doesn’t matter what other drivers score,
only what you scored and that you have the driving skills and knowledge
of traffic laws to “meet the standard” and get a license.
Q: Who writes our state testing questions?
A: Washington educators helped build our state tests and continue to
refine them. They review every question for content quality and its
relation to the state’s learning standards.
Q: What steps are taken to ensure that our state testing questions do not contain cultural bias?
A: Every state testing question goes through extensive analysis by a
Bias and Cultural Fairness Committee of specially trained educators and
community members before inclusion. Before any question is placed on a
state test, each question goes through review to ensure there is no
cultural bias in the exam. Each question also is given a trial run, or
is piloted, with students to determine whether the question poses
special difficulty for students from different backgrounds.
Q: Who scores our state tests?
A: Hundreds of people score our state tests. All scorers are monitored closely by Data Recognition Corporation
(DRC), the state’s testing operations contractor. State testing scorers
must have, at a minimum, a four-year college degree. All scorers are
monitored daily to ensure their scores meet criteria of accuracy and
consistency set by Washington educators. Scorers who are unable to score
according to these criteria are dismissed.
Q: How are student responses scored?
A: Reading, math and science: All one-point multiple-choice questions
are machine scored. Short-answer questions that are worth two are scored
by trained raters who use scoring guides defined by Washington
educators.
Writing: The same scoring guides are used every year for all of the
writing questions. Scorers are trained to set aside their own personal
opinions about what score a test should receive, so that student
responses are scored consistently and according to the criteria
determined by Washington educators and Data Recognition Corporation.
Q: How are passing scores determined?
A: This process is called “standard setting” because students must meet a certain performance standard
to pass. Setting standards for the MSP and HSPE is a thoughtful and
involved process incorporating the feedback of many people. First, a
standard-setting panel for a state testing subject and grade level is
convened. Members include teachers, parents and community members
representing Washington’s geographically and ethnically diverse
population.
The panelists for each content area review "performance-level descriptors," which are the written descriptions of what students should know and be able to do in that subject and grade. They also look at the test itself to see how many points a student should earn on the test to meet the “performance-level descriptors.” The panel’s work is done in rounds. After the first round of deliberations, panelists discuss each others’ perspectives and then conduct a second round of review. A third round is done before the panel, as a whole, makes final recommendations.
Next, an “articulation” committee is brought together to ensure the suggested standards relate sensibly to one another across the different grade levels. The articulation committee members represent standard-setting committee members from different subjects and grades. The articulation committee reviews the standard-setting committee’s recommendations and can make its own set of recommendations.
Both the recommendations from the standard-setting panel and articulation committee are forwarded to the State Board of Education for review and adoption. Once the State Board of Education decides which recommendation to adopt, that is the performance a student must achieve in order to “meet standard” or pass the MSP and HSPE.
Q: Are standards reset each year?
A: No. Once the State Board of Education
adopts a set of standards for a state test, the state carries that
expected level of performance from year to year. Each year a new edition
of the test is developed. Most of the questions on the test are new,
but some appeared in subsequent years. The repeated items are “anchor”
items that are used to link the performance on one year’s edition of our
state test to earlier editions of the test. This uses a procedure
called “equating.” By equating the current year’s state test to previous
state tests given in previous years, the performance standard can be
maintained over time.
Q: What is a good score on our state tests?
A: A student’s performance on the reading, math and science MSP and HSPE is reported using “scale scores.”
Scale scores are three-digit numbers that are used to place the student
into one of four levels: Advanced (Level 4), Proficient (Level 3),
Basic (Level 2) and Below Basic (Level 1). A scale score of 400 is
assigned to a student who has just barely met the state standard; this
score is at the lower end of Level 3. Students scoring in Level 4 are
said to have exceeded the state standard. Students with scores in Level 1
or Level 2 have not met standard.
Students generally have to achieve a score that represents approximately 60 to 65 percent of the points possible on each test to pass. That score or above means they have met the required standard for proficiency in that particular subject.
Q: What steps are taken to make sure that the scoring of open-ended items is valid and reliable?
A: The Office of Superintendent of Public Instruction
(OSPI) uses rigorous methods to ensure that the scoring process yields
valid and reliable results. Valid scoring means that a scorer assigns
the same score to a student response as would be assigned by an expert
panel of Washington educators. Reliable scoring means that different
scorers consistently assign the same score to a student response.
The following quality control measures are used when open-ended questions are scored on the MSP and HSPE. Beginning in the 2009-10 school year, the MSP and HSPE reading, math and science no longer have long-answer, four-point questions. Only the writing tests have extended response questions:
- Item-by-item scoring: Teams of scorers are trained to score responses to a single open-ended state testing question and all responses to that question are scored by the team. Once all students’ responses to that question have been scored, the team is trained to score a different test question. This process adds to the scoring consistency, as scorers need only keep a single scoring guide in mind as they score. This approach also protects against scorer biases that could come into play if entire test booklets were scored by a single scorer.
- Double-scoring: All writing responses in high school are scored twice to verify that scoring is consistent and aligned to the scoring rubrics.
- Supervisors Reread Scored Student Work: In addition to double-scoring, each scoring supervisor rereads an average of five percent of the papers from scorers under their supervision every day. If a supervisor discovers that a scorer begins to assign scores that do not match the scoring guidelines, the supervisor consults with the scoring director and together they retrain that scorer, using the original training materials. This on-the-spot checking helps keep the scoring consistent. If a scorer has drifted from the scoring guidelines, the scores he/she has recently-assigned are removed and those papers are reinserted into the queue to be rescored. Scorers who prove unable to score consistently after retraining are dismissed from scoring.
- Blindly Inserted Validity Papers: A "validity paper" is a student response that has been pre-approved by Washington content (reading, writing, math and science) specialists as being a clear example of a score point. Multiple validity papers are blindly inserted into a scorer's assignment of student responses to be scored. Scoring supervisors receive a daily report of how well scorers' decisions matched with the pre-determined score on the validity papers. Any variation from the scoring criteria is addressed immediately.
- Protocols to Handle Unique Responses: Scorers are trained to only assign a score to student responses that are consistent with the examples provided in training. If a scorer encounters a student response that is unique, novel or otherwise unfamiliar, the scorer seeks advice from the supervisor. If the response is new to the Supervisor, the Scoring Director intervenes. At this point, the Scoring Director can decide either that the response is merely a nuance of what is already described in the scoring rubric or that the response is truly unique. If the response is a nuance, all the scorers are notified and re-trained on that particular type of response; if the response is one that has not yet been encountered, OSPI content staff intervene and determine what score should be assigned, after which scorers are re-trained.
- Communication between OSPI and the scoring contractor: OSPI representatives are on site at Data Recognition Corporation facilities during training and scoring to monitor the quality processes and to address any content questions that may surface.
Q: How are test results reported?
A: Results are reported for individual students, schools, districts and
the state according to four performance levels defined by the State
Board of Education:
- Level 4: Advanced
- Level 3: Proficient
- Level 2: Basic
- Level 1: Below Basic
Every family of a student who takes a state test will receive a score report. Each school/district decides how families will receive this report (e.g. mail or parent/teacher conference). Check with your school or district to find out how you will receive your student’s results. Students in grades 10, 11 and 12 who take a state test in the spring will get two score reports: One in early June for reading, writing and math, and a second report in September that includes the results from the science HSPE and a look at school, school district and state scores.
Q: What happens if a parent refuses to have his/her child take a state test?
A: A student in grades 3-8 who doesn’t take a state test may miss out on
having any learning issues identified sooner rather than later. These
students also may miss out on the academic help paid for by the state
and federal government; this help is often attached to an individual
student test scores. High school students who do not take and pass a
state assessment in reading and writing will not graduate.
Q: Can a student’s scored booklet be reviewed?
A: Parents/Guardians may request to review their child’s test booklet. See state guidelines and forms.
Q: Can a parent/guardian appeal a student’s score after looking at the test booklet?
A: Parents/guardians may only appeal a score on a high school assessment
that is required for graduation: WASL, HSPE, WAAS-DAW, WAAS-Portfolio
or Collection of Evidence. A score appeal results in OSPI review of
particular scoring errors, such as errors on open-ended items, incorrect
score calculations, mistakes affecting erasures, test labeling, and
lightly marked bubbles on multiple choice items. Read the state guidelines. An appeal form will be provided when the parent/guardian reviews the test.
Q: How are state test results used?
- Improvements in teaching and learning
Parents, students, and educators use the results to:- Follow student progress
- Identify strengths, weaknesses and gaps in curriculum and instruction
- Fine tune curriculum alignment with the statewide standards
- Identify students who may need additional help
- School and district accountability
Under No Child Left Behind, the state reports on the Adequate Yearly Progress of students in schools and districts based on WASL results. - Student accountability
Students are required to pass a state assessment in reading and writing as one graduation requirement for a high school diploma. Students are given multiple opportunities, if necessary, to pass the tests. Alternatives also are available for students who have tried the exam at least twice. Students also must meet local requirements for high school graduation (for example, completion of required coursework).
Q: Is a listing of school and/or district state testing scores available?
A: Yes. The state has an extensive Web site for the public to view all elements of state testing at our Washington State Report Card.
Q: What does it take for a student to do well on a state test?
A: Students do well on state tests when they come to class regularly and
do their schoolwork. It’s also important for educators to use a
curriculum that emphasizes the state academic standards and regularly
ask students to think, communicate and solve problems. "Drill-and-kill
exercises" and fill-in-the-blank "test prep" a few weeks before taking
the test aren’t effective.
More Questions? http://k12.wa.us/assessment/StateTesting/FAQ.aspx