What you read…

Fish climb a tree

Free to download and share. Click to enlarge.

In terms of tests which leave a lot to be desired, the reading comprehension test ranks near the top.

Some of us are far removed from the fill-in-the-bubble types of reading comprehension tests which are at the basest levels of academia. We do not get a pass for venerability. In fact, there are quite a few people banking (in real currency) on our failing such tests.

Classic

If you look at the questions designed to gauge reading comprehension, you will instantly notice something peculiar: You do not have to comprehend what you read to answer the questions.

(Pause whilst everyone scratches their heads.)

Frankly, most reading comprehension tests are actually memory tests.

  • Did Sally put the peanut butter away before she ate her sandwich?
  • How many fish did Jethro catch in the concrete pond?
  • What year was the microchip invented?
  • Who ate the last pie of cake?
  • Where is the remote?

fill in the bubblePeople who do not understand the testing language have a good shot at being able to choose the correct recall answer by matching the words in the text to the answer choices. Recall is not comprehension. Without ever comprehending why the microchip was invented, the reader has a better than 50% chance to get the correct answer.

For the majority of the questions, the answer was plainly stated in the reading, can be deduced or can be induced. The last two of those now define comprehension. Critical thinking is not comprehension. Those questions test the reader’s knowledge beyond the scope of the test, specifically the reader’s ability to recall states and activities not contained in the text.

Many people read texts who have no real world experience about what they read. Do they comprehend it any less? Is it impossible to understand a new concept?

Real World

In order to get information to pass freely in our society, the media (with a large emphasis on “news” media) condenses key points into headlines. In ages gone by, headlines were meant to attract attention and entice the reader to actually read the story. Today, headlines are the majority of the content on offer.

extra newspaperIn support of the previous statement, look into the number of hoaxes which are still readily available on social media after Snopes posts the truth. Further, look at all the scam and “gotcha” sites whose posts have hundreds of thousands of shares. The people who want their friends and family to know about this disaster or that tragedy or the newest discovery have not taken the time to even read what they have shared.

Alternatively, headlines are disingenuous. Aside: How disheartening is it for the writer to see a well-researched piece capped with a completely misleading (or intentionally false) headline?

What does this have to do with comprehension? Headline writers the world over count on your ability to understand less than 15 words. The 150-150,000 words following the headline are inconsequential because the headline made you buy the (paper, magazine, book) or click on the link. How is this a test of comprehension?

If you understood what the headline had to offer and (read, bought, clicked), you passed the first test. The second test is trickier.

The content supported the headline: You share, give glowing reviews, refer friends, buy more as gifts, look at other merchandise. You turn your comprehension into money.

The content did not support the headline: You bury the results. You deny comprehension altogether.

How skewed are those statistics? The only ones who report comprehension are the ones who agree. Sounds too much like the thief being responsible for reporting his crimes.

Why? and How?

The only real questions which test comprehension (understanding) of any text are ones which begin with how or why. Unless framed in a discussion or essay, these answers cannot readily be assessed and simply cannot be garnered with multiple choice. It would require comprehension of both the main text and the available answers. It would present two avenues for error and result in lower scores.

Loquacious

Loquacious

Does discussion actually test our comprehension? How often have discussions be nothing beyond a polling of agreement? They fail to explain the text in favor of counting those who hold the same opinion regardless of facts.

Comprehension is complete when a text is transformed into the words of the reader. Only in a few instances can a reader understand a concept without being able to put it into the vernacular.

We eschew this type of testing because it must be graded by a person who can comprehend the answer instead of by a machine which can detect markings on a paper. It calls to mind a grad student who could not understand the correct usage of the word nonplussed. Without checking to be sure the word was used correctly, he marked the paper off based on his misunderstanding of the meaning. While the student who is more articulate than the teacher is less common, the dumping of this sort of task to those who may not be is commonplace.

Further, comprehension tested via essay calls into question writing ability. In your lifetime, you have known at least three people who could write well enough to cover their illiteracy and known at least three who could read without the ability to write a complete sentence correctly.


Is there a viable way to test for reading comprehension in a standardized fashion?

Hashtags: #readingcomprehension #standardizedtests

Thank you for sharing The M3 Blog with hashtags.

© Red Dwyer 2014
Re-Blogging of this or any other post on The M3 Blog
is expressly forbidden.
Copyright and Privacy Policy available in The Office.
Previous Post
Leave a comment

12 Comments

  1. I do not believe there is a way to test for comprehension using multiple choice. This is a flawed process, always has been. Multiple choice, whether for reading comprehension or anything else relies upon the test taker not to guess at the answer, not to rush through the test. Humans, by their nature are lazy and incapable of concentrated focus. They will skim questions, especially where the question is longer than three sentences (one paragraph) and they will guess at the answer.

    Where the choice is one of two, 50% opportunity to be right.

    Where the choice is one of four, 25% opportunity to be right. They might take some time to read to eliminate the obvious wrong answer, but still ultimately guess.

    In my humble opinion? The only legitimate test is essay testing, this though would force engagement by humans to evaluate answers.
    Valentine Logar recently posted..Bow Your Head and WeepMy Profile

    Reply
    • We agree. Essays are a test of both reading comprehension and writing skills, as well as a test of the grader’s ability to discern the test-taker’s intent.

      Reply
  2. This level of questions is called the reading for specific information questions. Comprehension tests up the bar by also adding true or false, fact or opinion, getting the main idea, making predictions from the text and reaching conclusions from the text, arranging in chronological order. These higher level reading skills require the student to formulate answers not pick answers. Multiple choice answers can be tricky. The rule is one choice ridiculous, one choice correct and two choices could be right and all of these or none of these. A problem with the maybe could be kinda right is that they are based upon the academic opinion of the test creator which in many instances I disagree with and could provide citations to contradict them in that they are right answers which kills the students chances due to opinion test creator bias. Tests are culturally biased. Average kids in Miami don’t know what autumn is. They know what fall means though. None would know what a creek, ravine, rill or lea is. So the geographic region of students locale may help or hinder test success because of particular vocabulary is or not used in particular regions. Kids do not know what a wrench is because in Caribbean English a wrench is a spinner. They expect kids to use context clues. But since all they know is the mall, sports and popular music they have no context upon which to draw. I had to learn over two hundred Latin and Greek roots in the eighth grade. That stuff has not been taught in over 40 years. In the past much television programming was based on war, westerns, famous people, historical eras but these kids today have only sitcoms and music and could not tell you if Napoleon had tanks and airplanes. VALENTINE LOGAR is correct in that the essay type question is the best measure of comprehension. In my 33 years of high school teaching only the college bound could do this despite the earnest efforts of language arts teachers throughout all the grade levels. Many of my students never could get it in that looking at the words is not reading the words and never reached a level to engage in the text. I told them if pictures were not formulating in their minds as they read, they were not reading. They looked at me like I was nuts. They could do questions like the ones beginning this post where they can find specific information. 50% of Miami students,foreign born, English is the secondary language(inadequate proficiency) with such as Spanish, Haitian Creole or Caribbean English the first language.

    Reply
    • I like to point out the colloquial impossibility of standardized testing. Were we a standardized society, we would not be democratic. The idea of standardized testing is absurd to me. Always shall be.

      Reply
  3. I never liked the true or false or pick from 1 to 4 etc. This is a percentage game and if you have no idea, you’ll cross your fingers, and maybe get lucky. When I went to school, I had to explain what I meant in answer to a question. What better way for examination? Good way to scope the class to see where the misunderstandings were strongest. These days, my grandchildren are doing true or false and pick one.

    Reply
    • Saddest part for me is a technique my child has morphed into real life. They are taught to answer even if they think it is wrong, for the possibility they get the answer correct, which inherently eschews “I don’t know” as an answer. Two emergent skills are frightening at this juncture:

      1. A lie is an answer. Although she knows the lie is a lie, there is the possibility it will answer the question correctly because the answer is beyond her ability.
      2. The false confidence supplied by never “not knowing” an answer means she is headstrong in her belief as long as she answers a question, she has completed the task.

      It disturbs me immensely teachers (of all people beyond parents) forego hearing “I don’t know.” To my, very simple albeit experienced, mind, IDK is an opportunity to share learning with someone who has taken a huge leap of faith IDK will not get them bullied.

      I could go on, but that will need to be another post.
      xxx

      Reply
  4. Whatever testing method is easiest and quickest is the one that’s going to be used, despite its inaccuracies. Much of school is like that. You don’t have to comprehend, you have to regurgitate. I remember one person I knew who had something like a 98% average in all subjects because he had a super memory. But ask him to apply the knowledge to something that was not in the textbook, and he simply couldn’t.

    Reply
    • The problem is we are teaching rote answers for critical thinking problems. It is the fallacy of the standardized test. You cannot have a canned response which calls for reader’s to call on experiences beyond the scope of the text. I am entirely tired of the “easy way”, but then again, Mantra and I already said that.

      Reply
  5. It’s strange how some students are classed as super brainy on paper but when given a simple task in practice will fail miserably. It all seems kind of pointless but then it is all about theory and practice. I know a few people that are hopeless when it comes down to filling out a form of some kind, but when faced with real life situations excel where others fall by the wayside.

    I don’t particularly like the A to D answers on a paper examination, as the results can be so varied, of course someone with the ability of remembering things, likened to a photographic memory for instance can always be an ‘A’ grade student but how will that same person achieve in practical things unless having been shown, and even then it is not necessarily a given that he or she will be able to function when theory is put into practice.

    I am probably missing the whole point of this posting with my waffling but hey your posts certainly offer a lot of scope my dear friend and that is what makes a fantastic read, it gives us food for thought.

    Have a lovely evening Red…

    Andro xxxx

    Reply
    • I have known dean’s list students who could not come in out of the rain. I have also know students who never graduated who were phenomenal successes. In the end, what one knows must be gauged by someone interested in the truth about the student’s grasp of the knowledge. I do not believe machines have the ability to do so in light of administrators’ inability to produce tests to gauge such grasp. Waffle on. It is entirely the point of the comment box. xxxx

      Reply
  1. Clear Air | The M3 Blog
  2. Three Day Post | The M3 Blog

Leave a Reply to Red of M3 Cancel reply

Your email address will not be published.

CommentLuv badge

This site uses Akismet to reduce spam. Learn how your comment data is processed.