Example of Bad Test Specifications

                                    TEST OF FRENCH FOR POSTGRADUATE STUDIES

                                                    Specifications for the Reading Test

General Statement of Purpose

The Test of French for Postgraduate Studies is a test battery designed to assess the French language proficiency of students who do not have French as their first language and who hope to undertake postgraduate study at universities and colleges where French is the medium of instruction. 

The aim of the battery is to select students who have sufficient French to be able to benefit from an advanced course of academic study, and to identify those linguistic areas in which they might need help.

The focus of the test battery is on French for Academic Purposes.

The Test Battery

The battery consists of four tests:

                                  Reading                             60 minutes

                                  Writing                              60 minutes

                                  Listening                            30 minutes

                                  Speaking                           15 minutes.

Separate scores are reported for the four tests. There is a different set of specifications for each of the four tests.

Reading Test

Time allowed: One hour

Test focus: The level of reading required for this test should be in the region of levels 5 to 7 of the English Speaking Union (ESU) Yardstick Scale

Candidates will have to demonstrate their ability to read textbooks, learned articles and other sources of information relevant to academic education. Candidates will be expected to show that they can use the following reading skills:

a) skimming

b) scanning

c) getting the gist

d)distinguishing the main ideas from supporting details

e) distinguishing statement from example

f) deducing ideas and information

g) deducing the use of unfamiliar words from context.

h) understanding relations within the sentence

i) understanding relations across sentences and paragraphs.

Source of texts:Academic 

Academic books, papers, views, newspaper articles relating to the academic subjects. The texts should not be highly discipline- specific, and should not disadvantage students who are not familiar with the topics. 

There should be four reading passages, each of which should be based on a different academic discipline. Two of the texts should be from the life and physical sciences, and two from the social sciences. As far as possible the four texts should exemplify different genres. For example, one text might consist of a review, a description of some results and a discussion.

Test tasks:

Each test question should sample one or more of the reading abilities listed above. Test writer should try to achieve a balance so that one or two skills are not over tested at the expense of the others.

Item types:

The Reading Test should contain between 40 and 50 items- approximately 12 items for each reading passage. Each reading passage and its items will form one sub- test. Each item will be worth one mark. Items may be open- ended, but they must objectively markable. Item writers should provide a comprehensive answer key with their draft test.

Item writers should use a variety of item types. These may include the following:

identifying appropriate headings

matching

labelling or completing diagrams, tables, charts, etc.

copying words from the text

information transfer

short answer questions

gap filling

sorting events or procedures into order

Rubrics: 

There is a standard introduction to the Reading Test which appears on the front each Reading Test question paper. Item writers, however, should provide their own instructions and an example for each set of questions. The language of the instructions should be not higher than Level 4 of the ESU Yardstick Scale

Reference:
Buck, G. 1991. Expert Estimates of test item characteristics. Paper presented at the Language Testing Research Colloquium, Princeton.

WHY IS THIS TEST SPECIFICATIONS CONSIDERED "BAD"?

- Lack of  weighting

- Lack of number of sections

- No sample papers

- No sample of students' performance on tasks

  

As a conclusion, this test specifications is a bad test specification because it does not contain all the 21 ITEMS that should be included in it.