Get Schooled

By Jeff

June 7, 2007 9:57 AM | Link to this

Bridget,

I want to see raw numbers. After all, make the passing score low enough and my pet rock can pass the test…

By em

June 7, 2007 10:08 AM | Link to this

Bridget,

Welcome back! Jeff is right. The raw numbers will tell the real story. Until the State Department of Education reveals how the scores are being scaled, the emporer will always be clothed. My EOCT scores this past spring were stellar until I look at the raw scores. A student who scored a 90 on the EOCT got 68% of the 75(not 90) questions correct. Manipulating the numbers to make them reflect improvement is the real issue the AJC should be delving into.

By Janine

June 7, 2007 10:41 AM | Link to this

Manipulating of numbers is absolutely the key to open Pandora’s CRCT box. HOw many questions in each section and how many did the students have to get right in each section to pass. Then you do the percentages. In addition, teachers spend huge amounts of time now TEACHING THE TEST QUESTIONS….THE FORMAT AS WELL AS THE CONTENT..[notice …to the test questions,which is different from teaching to the test.

There are so many practice tests given during the year with questions that are almost identical to the actual test questions. And, the students are drilled, often daily, on questions known to be on the tests.

I hope the AJC will make a comparison of scores our students make on the CRCT with those they make on a nationally standardized test such as the ITBS…Though I have heard that Cox wants to eliminate those tests that show how GA students stack up with the rest of the nation. Two years ago a 7th grade student who made the minimum passing score on the reading section of the CRCT, most often fell in a very low percentile on the ITBS and was at the ITBS 3rd grade reading equivalent.

By jim d

June 7, 2007 10:44 AM | Link to this

Welcome home sweetie,

Hope you had a great time.

As for the test results? Hell anyone with a lick of common sense won’t put much stock in them anyway.

By Lee

June 7, 2007 10:47 AM | Link to this

When you read the spin in the AJC from Cathy Cox, you would think that Ga students were just blowing this test away. Looking at the graphs, we see a different story.

Most statisticians would say that a 1-2% gain is irrelevant. Heck, most statisticians qualify their opinion with a “+/- one or two percent”.

What’s distressing to me is to see the failure rates rise dramatically once the student hits middle school.

Middle school is a huge problem area in Georgia.

Middle school is why I am now paying private school tuition.

As the old adage says; “There are three types of lies. Lies, damn lies, and statistics.”

By jim d

June 7, 2007 10:49 AM | Link to this

Oh did I mention the only reason science scores haven’t improved is because they haven’t quite figured out how to dumb science down—-YET

By Bridget Gutierrez

June 7, 2007 10:55 AM | Link to this

Hi, Jeff and em! Could you be more specific about what “raw numbers” you’d like to see?

Are you referring to the cut scores, which show how many questions a student needs to answer in order to pass the test? (If so, I’ve already been compiling those and am looking at writing a story about them.)

Are you referring to the actual scale scores that show how students fared on the exams (the numbers that look something like 100, 300 or 600, etc.)?

Or are you talking about something else entirely?

By Janine

June 7, 2007 11:27 AM | Link to this

Lee has a point about middle schools being a problem. I think at least part of the problem is that all of these young adolescents who have typical adolescent/immaturity problems are thrown together, reinforcing the bad stuff for each other. I would like to go back to K-7 and 8-12.

By Jeff

June 7, 2007 11:32 AM | Link to this

Bridget:

This is what I want to see:

ASSuming that the test is divided into sections, I want to see the following:

For each section: Number of questions, average number of correct answers, average number of incorrect answers, average number of blanks, scale factor

For the test: weights of the various sections, scale factor

On EACH test: I want the score to be reported as NumCorrect / TotalQuestions = Score

If the State then wants to say that 30% is the “passing” score, so be it. At least they’re honest about it at that point…

By Janine

June 7, 2007 11:40 AM | Link to this

Jeff…your suggestion is great: Simple, everyone can understand the major parts, everyone can draw conclusions as to what’s really happening here. However, the State Dept. will present scores that way, WHEN PIGS FLY!

By suanne

June 7, 2007 12:02 PM | Link to this

Are the test questions this year exactly the same as last year’s test questions? Or at least the same style? If not then I don’t know how we can truly compare. If cut scores are the same as last year then the next question is if they made the test easier. I’ve always wanted to try an experiment to see how many students could pass this test at the beginning of the year. With cut scores so low I suspect it would be a large number. Seriously, why no pre-testing? This could surely better direct teaching for the year.

By Jeff

June 7, 2007 12:05 PM | Link to this

Janine,

eh, Bridget asked me what I WANTED, not what I thought I’d GET…

My entire political stance is becoming very much in line with this statement:

Keep It Simple, Jack@$$ (Would be Keep It Simple, Stupid, but I’m pretty mad at ALL the elected officials at this point and I’m to the point of saying that if you’re an incumbent, I AINT voting for ya.)

By Janine

June 7, 2007 12:13 PM | Link to this

Pre testing and post testing is a basic to actually have evidence of improvement. In the middle and late 1970’s, Dekalb had a Reading Ctr. in Clarkston to help students who had difficulties in reading during the year and in the summers. Pre and post testing was a mandatory part of every student’s instructional plan.

By Janine

June 7, 2007 12:23 PM | Link to this

Instruction at the Dekalb Reading Ctr. was 1:1 or 1:2. If one was a teacher there, he/she had better have a super explanation if the student didn’t make significant progress. Did I mention they closed the center? Which is so-o Dekalb!

By Ernest

June 7, 2007 12:26 PM | Link to this

Welcome back, Bridget! Count me among those that like Jeff’s suggestion for how scores should be reported. It should be simply enough for the lay person to understand. Is Dana Tofig lurking today?

I might add something else for additional analysis, any ‘weights’ that particular questions had. If they got 33% of the questions correct and that reflected most of the ‘easy’ ones, it helps me with remediation strategies.

By luvs2teach

June 7, 2007 12:42 PM | Link to this

suanne and Janine - you both brought up a major issue I have with the year-to-year comparisions that we make (and it ties right back to Lee’s adage of ‘lies, damn lies, and statistics’).

First, if you are looking at a subject such as reading or math, you can look for growth from one year to the next, because they are subjects where the skills build. However, when you compare this year’s 8th grade scores to last year’s 8th grade scores, you aren’t looking at growth, because you are looking at an entirely (well, almost entirely) DIFFERENT set of kids!

I suppose you could look at how that group did as 7th graders, and then how they did as 8th graders, but the tests are different based on each grades performance standards.

Now in social studies, or my subject, science, you have different subject matter each year - For example 7th grade social studies teaches regional geography, and 8th grade teaches Georgia History. Seventh grade science is Life Science and (this year anyway) 8th grade was Earth Science. Again, if you compare this year’s 8th graders to last year’s, you are looking at differnt kids, and if you compare how well this year’s 8th graders did compared to their performance as 7th graders, you are looking at an entirely different subject!

So how valid are these comparisons?

I’m dismayed at the thought of getting rid of the ITBS - I think, at minimum, it should be given early in the gateway years of 3, 5, and 8 (or even in the year prior) so problems can be targeted. Like others, I have seen students with low, but passing, scores on the CRCT be in the lowest quartile on the ITBS.

Non-ed folks might not understand the difference in the two tests - the CRCT is criterion referenced - how well you do is compared to a standard, and you are judged alone against the standard. ITBS is norm referenced. A norm group takes the test and all others are compared to the performance of the group.

Another common misconception with the ITBS is the grade level equivalency. I often have parents who think their child should be placed in a higher level class because their child “performs at the level of a 10th grader.” Well, that’s not how the GE works. If you had a GE of 10.6 (10th grade, 6th month of school) on an 8th grade test, that doesn’t mean you should be in the 10th grade, or even that you are as smart as a 10th grader. It means that you performed as well as an average 10th grader from the norming group on the 8th grade test.

More testing info can be found here Criterion Vs Norm-Referenced Testing

That being said, I would still love to know what the cut scores are.

By catlady

June 7, 2007 1:03 PM | Link to this

We will NEVER get what the bloggers are asking for, Bridgett, but perhaps you can find out WHY they won’t give us the information. Couldn’t the FOI act pry them loose?

As Lee said, I see little statistically significant, since missing (or getting right) one question could, cumulatively, make it appear to be worse (or better). IMHO, the test is missing the kind of validity that better tests (nationally normed tests) usually get: that if you gave the test to the same child twice you would get the same results. I am pretty sure that is NOT true of this test (otherwise, how would you explain the “miracle” gains these kids will make in a few weeks of summer school?) Either we can work miracles in a few weeks of summer school (when 180 days did not do the trick), or the scores are manipulated in the summer, or the test does not have validity. Now, DOE, which is it? Are the regular teachers incompetent—if so we should just save money and send everyone to 4 weeks of summer school. Or do you manipulate the cut scores or weighting? Or is the test lacking in the kind of validity I am talking about? Pick your poison.

I truly have no idea why Ms. Cox is so proud of these scores!

And yes, I’d like to see an equivalency table linking CRCT to ITBS (realizing it would be a rough approximation). The comparisions I have unofficially done are NOT something to brag about; in fact, they should horrify folks! (ie, kids who barely pass the CRCT in reading rank nearly 2 years below grade level on the ITBS). So if our “big improvement” via the GPS is working, why can’t our students do reading like the average student in other states? Are the GPS standards THAT LOW?

This is a story that is dying to be busted open! Welcome back, Bridgett, and start bustin’!

By catlady

June 7, 2007 1:15 PM | Link to this

One other thing: I am willing to bet in the next year or two the standard scores will be changed again, rendering “comparison impossible”, as it was supposedly from moving reading from 300 to 800 for the passing score.

How do we do on NAEP scores? I think near the bottom. Let’s cross-reference the NAEP scores of those who took it with the CRCT scores.

As jim d so succinctly says, “smoke and mirrors”. I’d use a stronger term than that.

By Middleschoolteacher

June 7, 2007 1:38 PM | Link to this

Last year at my school in Gwinnett, we asked these same questions about cut scores. We were given an Excel sheet that contained all cutscores for 2006, grades 1-8. I teach seventh life science which had 60 questions in 2006. According to the sheet, students had to answer 30 correctly to achieve minumum and 50 to achieve exceeds. Other subjects had different amounts.

By Ernest

June 7, 2007 1:39 PM | Link to this

luvs2teach, thanks for the info on Criterion Vs Norm-Referenced Testing. I know many that get confused and/or misinterpret the scores. I thought I understood the ITBS to be nationally normed meaning I could compare scores across states. To me, knowing how we do nationally impacts our ability to both attract and retain business, along with the obvious of ensuring we are properly educating our children. Since each state may come up with their own assessment requirements, it makes state to state comparisons subjective at best.

I know Janine is in DeKalb. I’m sure she is interested like myself in knowing whether implementing the Springboard program from the College Board had a positive impact on middle school scores. There were ‘high hopes’ when this investment was made so I hope we are at the point where we can realize some type of ROI.

By sweetea

June 7, 2007 1:50 PM | Link to this

The whole thing is BS. The areas where my daughter aced the ITBS she just “met standards” on the CRCT. The areas where she was lowest on the ITBS, she kicked tail (near perfect or perfect score) on the CRCT.

I think that the CRCT is a waste of time honestly. It tells me very little that I am interested in and just stresses my kid out. It punishes students and schools based on their performance for an hour on one day of the year.

I found out today that her teacher went over her individual scores with her before school was out and gave her a hard time about her reading score. My kid was in 4th grade and consistently tested as being on a 7th grade reading level. She consistently reads books way above the 4th grade level and understands them well. Explain to me why she only “met” standards” in reading?????? Her lexile has been near 1000 all year - the CRCT shows it at 825. She was 825 at the beginning of third grade!

Also, everyone who knows my daughter knows that she has trouble with spelling, grammar and composition. However, she aced (perfect score) the language arts CRCT.

To me the ITBS is far more meaningful than the CRCT. I want to know what my kids knows in comparison to the rest of the nation.. i.e. the people that she will be competing against for college slots. It turns out that she knows more than the vast majority of the kids in the nation in all subjects… so I am pleased. Also, her ITBS scores have always coincided well with what I know to be her strengths and weaknesses. The CRCT is always just a crapshoot.

I could care less whether she knows what’s on some little GA podunk checklist from the GA DOE…. Her GA “criterion” knowledge won’t do much for her on the SAT and when she gets the heck out of this ridiculous state when she reaches the age of majority.

My question is how do you opt out of the CRCT? In many states there is a mechanism, what is it in GA. I have tried to find it. I’d much rather may daughter spend CRCT week reading a good book than filling in bubbles that tell me nothing.

Second question… when are the results by school and district coming out? I’m looking forward to the hand wringing that always accompanies that event! When are they due out?

By catlady

June 7, 2007 1:54 PM | Link to this

Anyone know how often the ITBS scores are re-normed?

By Janine

June 7, 2007 1:58 PM | Link to this

Bridget…I hope you and everyone here will look at the 12:42 post by luvs2teach. The issue of comparing the scores of two entirely different groups of students is and always has been bordering on the ridiculous.

I post on Jim Wooten’s {THinking Right} blog occasionally when education is his topic. He met with Margaret Spelling[s]?,I think she is Sec. of Ed., and asked several of us to e mail him questions we would like to ask. That was mine… why NCLB [or any educator] would believe that comparing the 2006 8th grade scores with those of the 2007 8th graders would give valid information about student improvement. Just one example: The 2007 8th grade may have had 50% limited English speakers [those counted in the regular student population] and the 2006 class may have had 20%. What reasonable person/entity would not see that as an invalid comparison for the basis of a school’s/teacher’s accountablily.

He said he asked her. She said she was aware of that little issue and they were “working on that”. Forget the reason she gave for not “working on that” BEFORE rolling out NCLB.

By Janine

June 7, 2007 2:05 PM | Link to this

Don’t know about ITBS norming,catlady…but , when I had an absent student, I used to look over the ITBS questions while the kids were taking the test…did the same for CRCT. NOT EVEN CLOSE to the same level of difficulty or consistency of difficulty among subjects. In my experience, a child’s ITBS scores were a more credible assessment of a student’s abilities.

By catlady

June 7, 2007 2:12 PM | Link to this

Sweetea, I believe the district and school results have been “out” for three weeks or so, but are not released to the public yet. They have to be “gone over” (spun) first. As far as I know, the only way you get out of the CRCT is to withdraw from school or have a terrible accident or illness that keeps you out during the testing window. Your child to be pretty sick or hurt, because they will send a teacher to your house to give it. The schools are evaluated on their response rates, to keep them from sending the sp ed kids home for a few days.

Please remember re: your child’s results on both tests, that they are both rough measures of a child’s performance on those questions ON THAT PARTICULAR DAY. Nothing more.

By News on the CRCT

June 7, 2007 2:15 PM | Link to this

This just in: In addition to the “student success,” it’s just been confirmed that Terry Shiavo also passed the CRCT.

By Janine

June 7, 2007 2:17 PM | Link to this

sweetea@1:50 “The whole thing is BS.” Thanks!! Now that… is a credible assessment!

By catlady

June 7, 2007 2:26 PM | Link to this

Janine, I agree about the ITBS. The question could be, why are we giving third graders (in late March) a reading test that says they “meet standards” when on a nationally normed reading test their score says they are reading as we would expect kids near the end of first grade would read if they were given the same test? Or, to put it another way, why would a child meeting standards for third grade in Georgia also score so that 81% of third grade students nationally would surpass the score? The secret is in the construction of the test and the weighting thereof.

There is too much riding on these tests that have little or nothing to do with the children, IMHO.

By luvs2teach

June 7, 2007 2:30 PM | Link to this

I am cutting and pasting some pertinent info from the University of Iowa’s site about the ITBS. The link is Iowa Test of Basic Skills

Italics and bolding will be mine :-)

Types of Scores

Raw Score (RS) The number of questions a student gets right on a test is the student’s raw score (assuming each question is worth one point). By itself, a raw score has little or no meaning. The meaning depends on how many questions are on the test and how hard or easy the questions are.

Percent Correct (PC) When the raw score is divided by the total number of questions and the result is multiplied by 100, the percent-correct score is obtained. Like raw scores, percent-correct scores have little meaning by themselves. They tell what percent of the questions a student got right on a test, but unless we know something about the overall difficulty of the test, this information is not very helpful. Percent-correct scores are sometimes incorrectly interpreted as percentile ranks, which are described below. The two are quite different.

Grade Equivalent (GE) The grade equivalent is a number that describes a student’s location on an achievement continuum. The continuum is a number line that describes the lowest level of knowledge or skill on one end (lowest numbers) and the highest level of development on the other end (highest numbers). The GE is a decimal number that describes performance in terms of grade level and months. For example, if a sixth-grade student obtains a GE of 8.4 on the Vocabulary test, his score is like the one a typical student finishing the fourth month of eighth grade would likely get on the Vocabulary test. The GE of a given raw score on any test indicates the grade level at which the typical student makes this raw score.

Grade equivalents are particularly useful and convenient for measuring individual growth from one year to the next and for estimating a student’s developmental status in terms of grade level. But GEs have been criticized because they are sometimes misused or are thought to be easily misinterpreted… Grade equivalents are particularly suited to estimating a student’s developmental status or year-to-year growth. They are particularly ill-suited to identifying a student’s standing within a group or to diagnosing areas of relative strength and weakness.

Developmental Standard Score (SS) Like the grade equivalent (GE), the developmental standard score is also a number that describes a student’s location on an achievement continuum. The scale used with the ITBS and ITED was established by assigning a score of 200 to the median performance of students in the spring of grade 4 and 250 to the median performance of students in the spring of grade 8.

The main drawback to interpreting developmental standard scores is that they have no built-in meaning. Unlike grade equivalents, for example, which build grade level into the score, developmental standard scores are unfamiliar to most educators, parents, and students. To interpret the SS, the values associated with typical performance in each grade must be used as reference points.

The main advantage of the developmental standard score scale is that it mirrors reality better than the grade-equivalent scale. That is, it shows that year-to-year growth is usually not as great at the upper grades as it is at the lower grades. (Recall that the grade-equivalent scale shows equal average annual growth — 10 months — between any pair of grades.) Despite this advantage, the developmental standard scores are much more difficult to interpret than grade equivalents. Consequently, when teachers and counselors wish to estimate a student’s annual growth or current developmental level, grade equivalents are the scores of choice.

Percentile Rank (PR) A student’s percentile rank is a score that tells the percent of students in a particular group that got lower raw scores on a test than the student did. It shows the student’s relative position or rank in a group of students who are in the same grade and who were tested at the same time of year (fall, midyear, or spring) as the student. Thus, for example, if Toni earned a percentile rank of 72 on the Language test, it means that she scored higher than 72 percent of the students in the group with which she is being compared. Of course, it also means that 28 percent of the group scored higher than Toni. Percentile ranks range from 1 to 99.

A student’s percentile rank can vary depending on which group is used to determine the ranking. A student is simultaneously a member of many different groups: all students in her classroom, her building, her school district, her state, and the nation. Different sets of percentile ranks are available with the Iowa Tests of Basic Skills to permit schools to make the most relevant comparisons involving their students.

Types of Score Interpretation An achievement test is built to help determine how much skill or knowledge students have in a certain area. We use such tests to find out whether students know as much as we expect they should, or whether they know particular things we regard as important. By itself, the raw score from an achievement test does not indicate how much a student knows or how much skill she or he has. More information is needed to decide “how much.” The test score must be compared or referenced to something in order to bring meaning to it. That “something” typically is (a) the scores other students have obtained on the test or (b) a series of detailed descriptions that tell what students at each score point know or which skills they have successfully demonstrated. These two ways of referencing a score to obtain meaning are commonly called norm-referenced and criterion-referenced score interpretations.

Norm-Referenced Interpretation Standardized achievement batteries like the ITBS and ITED are designed mainly to provide for norm-referenced interpretations of the scores obtained from them. For this reason they are commonly called norm-referenced tests. However, the scores also permit criterion-referenced interpretations, as do the scores from most other tests. Thus, norm-referenced tests are devised to enhance norm-referenced interpretations, but they also permit criterion-referenced interpretation.

A norm-referenced interpretation involves comparing a student’s score with the scores other students obtained on the same test. How much a student knows is determined by the student’s standing or rank within the reference group. High standing is interpreted to mean the student knows a lot or is highly skilled, and low standing means the opposite. Obviously, the overall competence of the norm group affects the interpretation significantly. Ranking high in an unskilled group may represent lower absolute achievement than ranking low in an exceptional high performing group.

Most of the scores on ITBS and ITED score reports are based on norm-referencing, i.e., comparing with a norm group. In the case of percentile ranks, stanines, and normal curve equivalents, the comparison is with a single group of students in a certain grade who tested at a certain time of year. These are called status scores because they show a student’s position or rank within a specified group. However, in the case of grade equivalents and developmental standard scores, the comparison is with a series of reference groups. For example, the performances of students from third grade, fourth grade, fifth grade, and sixth grade are linked together to form a developmental continuum. (In reality, the scale is formed with grade groups from kindergarten up through the end of high school.) These are called developmental scores because they show the students’ positions on a developmental scale. Thus, status scores depend on a single group for making comparisons and developmental scores depend on multiple groups that can be linked to form a growth scale.

An achievement battery like the ITBS or ITED is a collection of tests in several subject areas, all of which have been standardized with the same group of students. That is, the norms for all tests have been obtained from a single group of students at each grade level. This unique aspect of the achievement battery makes it possible to use the scores to determine skill areas of relative strength and weakness for individual students or class groups, and to estimate year-to-year growth. The use of a battery of tests having a common norm group enables educators to make statements such as “Suzette is better in mathematics than in reading” or “Danan has shown less growth in language skills than the typical student in his grade.” If norms were not available, there would be no basis for statements like these.

Norms also allow students to be compared with other students and schools to be compared with other schools. If making these comparisons were the sole reason for using a standardized achievement battery, then the time, effort, and cost associated with testing would have to be questioned. However, such comparisons do give educators the opportunity to look at the achievement levels of students in relation to a nationally representative student group. Thus, teachers and administrators get an “external” look at the performance of their students, one that is independent of the school’s own assessments of student learning. As long as our population continues to be highly mobile and students compete nationally rather than locally for educational and economic opportunities, student and school comparisons with a national norm group should be of interest to students, parents, and educators.

A common misunderstanding about the use of norms has to do with the effect of testing at different times of the year. For example, it is widely believed that students who are tested in the spring of fourth grade will score higher than those who are tested in the fall of fourth grade with the same test. In terms of grade-equivalent scores, this is true because students should have moved higher on the developmental continuum from fall to spring. But in terms of percentile ranks, this belief is false. If students have made typical progress from fall to spring of grade 4, their standing among fourth-grade students should be the same at both times of the year. (The student whose percentile rank in reading is 60 in the fall is likely to have the same percentile rank when given the same test in the spring.) The reason for this, of course, is that separate norms for fourth grade are available for the fall and the spring. Obviously, the percentile ranks would be as different as the grade equivalents if the norms for fourth grade were for the entire year, regardless of the time of testing. Those who believe students should be tested only in the spring because their scores will “look better” are misinformed about the nature of norms and their role in score interpretation.

Scores from a norm-referenced test do not tell what students know and what they do not know. They tell only how a given student’s knowledge or skill compares with that of others in the norm group. Only after reviewing a detailed content outline of the test or inspecting the actual items is it possible to make interpretations about what a student knows. This caveat is not unique to norm-referenced interpretations, however. In order to use a test score to determine what a student knows, we must examine the test tasks presented to the student and then infer or generalize about what he or she knows.

Criterion-Referenced Interpretation A criterion-referenced interpretation involves comparing a student’s score with a subjective standard of performance rather than with the performance of a norm group. Deciding whether a student has mastered a skill or demonstrated minimum acceptable performance involves a criterion-referenced interpretation. Usually percent-correct scores are used and the teacher determines the score needed for mastery or for passing.

Even though the tests in the ITBS and ITED batteries were not developed primarily for criterion-referenced purposes, it is still appropriate to use the scores in those ways. Before doing so, however, the user must establish some performance standards (criterion levels) against which comparisons can be made. For example, how many math estimation questions does a student need to answer correctly before we regard his/her performance as acceptable or “proficient?” This can be decided by examining the test questions on estimation and making a judgment about how many the minimally prepared student should be able to get right. The percent of estimation questions identified in this way becomes the criterion score to which each student’s percent-correct score should be compared.

When making a criterion-referenced interpretation, it is critical that the content area covered by the test — the domain — be described in detail. It is also important that the test questions for that domain cover the important areas of the domain. In addition, there should be enough questions on the topic to provide the students ample opportunity to show what they know and to minimize the influence of errors in their scores.

Most of the tests in batteries like the ITBS or ITED cover such a wide range of content or skills that good criterion-referenced interpretations are difficult to make with the test scores. However, in most tests the separate skills are defined carefully, and there are enough questions measuring them to make good criterion-referenced interpretations of the skill scores possible. For example, the Reference Materials test covers too many discrete topics to permit useful criterion-referenced interpretations with scores from the whole test. But such skills as alphabetizing, using a dictionary, or using a table of contents are defined thoroughly enough so that criterion-referenced interpretations of scores from them are quite appropriate. However, in an area like Mathematics Concepts at Level 12, some of the skill scores may not be suitable for making good criterion-referenced interpretations. Each of the six skills in that test is a broad content area which is further defined by two to four subskills. Furthermore, some skills, such as measurement, each have only three questions to cover a broad topic. That is generally too few for making sound judgments about mastery.

The percent-correct score is the type used most widely for making criterion-referenced interpretations. Criterion scores that define various levels of performance on the tests are generally percent-correct scores arrived at through teacher analysis and judgment. Several score reports available from Iowa Testing Programs include percent-correct skill scores that can be used to make criterion-referenced interpretations: Primary Reading Profile, Class Item Response Record, Group Item Analysis, Individual Performance Profile, and Group Performance Profile.

Wow hope that helps clear things up, LOL

My interpretation would be that, in looking a a child’s progress, the CRCT is like an x-ray, and the ITBS is like an MRI.

By catlady

June 7, 2007 2:30 PM | Link to this

Sweetea, and others, I am “comforted” by the fact that, at our school at least, about 30% of the students get no social studies and or science instruction at all—they are sent to “extra” instruction in reading and or math while their classmates are studying the GPS in science and social studies. (There is no other time in the day). So when they don’t know about the state bird of Georgia (how important!) it is no surprise to me.

By jim d

June 7, 2007 2:39 PM | Link to this

In terms we simple folk can understand.

Cut scores and comparisons of different groups from different years to show improvement is PFM, (thats Pure F’n Magic)

By catlady

June 7, 2007 2:44 PM | Link to this

Sweetea, and others, I am “comforted” by the fact that, at our school at least, about 30% of the students get no social studies and or science instruction at all—they are sent to “extra” instruction in reading and or math while their classmates are studying the GPS in science and social studies. (There is no other time in the day). So when they don’t know about the state bird of Georgia (how important!) it is no surprise to me.

By WhatWillBridgetDo?

June 7, 2007 2:45 PM | Link to this

Bridget,

Are you an “education reporter” or did you just sleep in a Holiday Inn Express last night? Seriously.

Go sit in some classrooms with kids who passed the CRCT and watch them read. Then ask them some questions about what they read. You might get that much needed reality check, if you’re willing to ask yourself “How did they pass this test when they can’t read”

When your done, explain for us why the AJC didn’t follow through on the discipline story where APS officials FALSIFIED FEDERAL DOCUMENTS.

And this time try to come up with something less lame than “the APS beat reporter is on leave” because you look COMPLETELY STUPID, when you turn right around and do a puff piece on Carver.

By catlady

June 7, 2007 2:53 PM | Link to this

jim d, more like Wishful Thinking or Blowing Smoke.

By Janine

June 7, 2007 3:05 PM | Link to this

jimd’s analysis:

“In terms we simple folk can understand…Cut scores and comparisons of different groups from different years to show improvement is PFM, (thats Pure F’n Magic)”

Let’s jump on it, Bridget! WHat a story! Has anyone from the AJC EVER EVEN ASKED Dana Tofig to explain?

By Martina

June 7, 2007 4:27 PM | Link to this

Another reason why science scores are so low - our 3rd grade Georgia Performance Standards are Georgia Habitats - plants and animals of the different regions. Do we have any materials to make sure we are teaching standard information? NO! We had to develop our own curriculum, so no wonder scores are all across the board!

By thomas

June 7, 2007 5:04 PM | Link to this

Here’s the deal for the CRCT:

If the child does not pass the reading portion of the CRCT, they are illiterate morons.

If the child “passes” the CRCT (800-849), they know their ABCs.

If the child “exceeds” in the reading portion, they can reasonably “read”.

This is the case regardless of QCCs, GPS’, etc. The CRCT has always been hokum. A retarded person could pass the social studies section of the CRCT. I had a fourth grader who came to my class in December, fresh in the country from Mexico. He was a very marginal student. This godawful bastard got a 317 on the SS section and failed the heck out of every other section.

The “cut scores” were actually raised over the last two years— THANKS TO THE AJC!!! If it were not for the AJC’s prodding and reporting, they probably would have never raised the cut scores from 40%.

By Janine

June 7, 2007 5:55 PM | Link to this

OH..THomas..you just reminded me…we had students [and not just a few] in my middle schools who spoke so little English…who couldn’t string 5 words together to make a sentence, who passed the reading and English sections of the CRCT….NOw what does that tell you???? That’s another thing the AJC could pursue if they were inclined to do so.
HOW TELLING IS THAT??

By Larry

June 7, 2007 5:59 PM | Link to this

I have long believed that scaled scores for high-states tests are used to manipulate results. It’s pretty obvious that’s what happened with the Texas TASS, but I was never able to document this locally. GCPS and I fought along a fine legal line when I attempted to verify the Gateway scaling factors after the tests were printed, but before the raw scores were known. I’m pretty good at ferreting out such things, but GCPS was better at defense and I was never able to prove raw scores influence the scaling factors.

The obvious red flag with CRCT scores is the fact they use two different scaling factors on the same test, depending on whether or not the assessment relates to a GPS course. This violates the very reason for having scaled scores, which is to allow comparisons between tests with a different level of difficulty.

Lacking proof, I can’t publicly state scaling factors are used for score manipulation; just that their purpose is other than the official explanation.

Incidentally, the number of questions, raw score and much of what Jeff requested IS on the individual CRCT test results issued to parents.

By Middleschoolteacher

June 7, 2007 6:39 PM | Link to this

Just a question, Bridget, does the ajc have plans to post the top 50 schools on the CRCT as it did last year? Most of the bloggers here probably wouldn’t have much confidence in the scores, but I thought it was interesting to look at them.

By Lisa B.

June 7, 2007 7:46 PM | Link to this

Thomas,

The S.S. part of the CRCT, as you know, hasn’t changed to GPS yet. The test will be much more ‘rigorous’ when GPS is implemented. Ha!

By high school teacher

June 8, 2007 9:15 AM | Link to this

Bridget, I just have to ask:

did you get the inspiration for the story title from The Jerk? “The new phone books are here! The new phone books are here! I’m somebody!”

Sorry, summertime just makes me silly. :)

By Bridget Gutierrez

June 8, 2007 4:06 PM | Link to this

WWBD: I’ve addressed these issues with you before. I’m a professional journalist paid for reporting facts. I have no incentive or reason to lie or make up false excuses.

Middleschoolteacher: I’m not sure if we’ll run the lists again this year. We’ll have to wait to get the data and see what we can do with it.

high school teacher: I think I was actually channeling Paul Revere: “The Redcoats are coming! The Redcoats are coming!”

By WhatWouldBridgetDo?

June 8, 2007 5:53 PM | Link to this

Quote from Bridget: “I have no incentive or reason to lie or make up false excuses.” Uh, how about keeping your job?

What other reason would “professional journalists” not follow through with follow up stories as explosive as cheating scandals or falsifying documents on discipline?

Please don’t insult our intelligence by saying the AJC doesn’t have its own political agenda, and that that agenda doesn’t influence what reporters can and cannot cover…it doesn’t help much with your credibility…you’d be better off telling us where reporters go to blog so we can get the real story