ML20151D628
| ML20151D628 | |
| Person / Time | |
|---|---|
| Site: | Shoreham File:Long Island Lighting Company icon.png |
| Issue date: | 06/10/1988 |
| From: | LONG ISLAND LIGHTING CO. |
| To: | |
| Shared Package | |
| ML20151D407 | List:
|
| References | |
| 3.04, NUDOCS 8807250184 | |
| Download: ML20151D628 (10) | |
Text
OTP 3.04 REV I
M~LCO OFFICE OF TRAINING g,,,,,;,, 3,, ggg.
~'~
7 PROCEDURES TITLE:
DEVELOPING WRITTEN TESTS 1.0 PURPOSE The purpose of this procedure is to assign the responsibilities and direct the actions necessary to develop and maintain tests for use in evaluating performance based training as well as to enter test items into an exam item bank.
2.0 SCOPE This procedure presents a suggested step-by-step method that can be used to prepare written exam items and formulate an exam when not using an exam bank.
The method te be used for entering the test items into a exam item bank is also presented.
3.0 DEFINITIONS Technically qualified - A person whose accomplishments and position make him worthy of judging subject matter relevancy, accuracy, and currentness.
4.0 DISCUSSION Tests based on objectives (known as criterion-referenced) are developed at the Office of Training to student mastery of the approved curriculum.
measure Tests serve two purposes at the Office of Training:
for measuring performance of objectives.
o a means for diagnosing and correcting problems or o
a means weaknesses during the instructional process.
8807250184 880 15 PDP ADOCK 05000322
- 9 e
i 1
j d
i g,,,s.
.~
- - ~ -
t APPROVED BY/DATE
'gg/
(
/
There are written and performance (JPM or hands-on) tests.
This procedure will address only how to develop written tests.
Written tests measure competency in cognitive skills.
(Please refer to procedure 3.02, "Developing Job performance Measures," for developing performance tests.)
The following five types of written test items make up the tests discussed in this procedure:
o multiple choice, o
modified true/Calse, o
matching,
o completion (short answer, fill-in),
o essay.
Multiple choice items should be used when the instructor wants the student to react appropriately with minimal prompt, to discriminate, to make generalizations, or to interpret new information.
Modified true/ false items are useful for determining knowledge.
We use postulates everyday as we do our jobs; therefore, those test items are excellent to test knowledge that may be taken for granted in the work place.
We should be careful not to give credit for a false item unless the student writes whr.t the statement should say to make it true.
Modified true/ false items eliminate guessing.
Matching test items should be used when the instcuctor wishes to measure the student's ability to identify associations and relationships.
A completion item can be a question, a statement, or an association to complete.
The instructor is looking for students to possest either knowledge when responding with word or phrase answers or ability to solve l
problems when responding with numerical answers.
Essay questions--the most difficult to score--allow the instructor to test for organization, presentation, logic of arguments, application of methods and procedures, or evaluation of concepts or ideas.
OTP 3.04 Rev. 1 page 2 of 10
i 5.0 RESPONSIBILITY AND AUTHORITY 5.1 Lead instructors are responsible for ensuring test items are developed for programs under their Control.
5.2 The tralning administrative section supervisor (TAS) is responsible for entering all submitted test items into the r* levant test item bank, and for maintaining the appropriate exam item bank records.
5.3 Training section supervisors are responsible for approval of exam items and exams.
5.4 Training instructors and instructional technologists are responsible for the development of exam items in accordance with this procedure.
5.5 All individuals involved in the preparatier and handling of exam items during the formulation of an examination are responsible for maintaining the confidentiality of both exam items and the resultant exam.
5.6 The TAS is responsible for advising office personnel of the need for safeguarding examinations being typed or copied, including throwaway pages which are defectise.
Disc..cded exam items are to bv shredded.
6.0 REFERENCE DOCUMENTS 6.1 Institute of Nuclear Power Operations INPO 85-006, "Principles of Training System Development "
February 1985 6.2 Nitko, Anthony J. et. al.,
Education,a1 Tests and Measurement an Introduction.
Hartcourt Brace Jovanovich, Inc. N.Y.
1983.
OTP 3.04 Rev. 1 Page 3 of 10
t
\\
4
.(
7.0 ACTIONS 7.1 Formulating Exam Items and Exams 7.1.1 The following actions are provided as a suggested method for developing exam items and exams.
Other methods may be used at the discretion of the training section supervisor.
A consistent method must be used to ensure that exam items reflect lesson objectives; the exam measures mastery of course goals; exam items selected are appropriate for valid evaluation of the students mastery of the lesson objectives.
7.1.2 Exam Construction Guidelines o
List the objectives wh.ah you want this exam to evaluate.
These objectives should be found in your course or lesson plan.
l l
o The lessen should be reviewed to l
assure that it addresses each exam
- j item included on the exam.
o Review each objective in terms of expected student outcome.
o Specify the type of exam item for evaluating each objective, and the percent of the grade this objective will comprise.
This translates to the percent of the grade for the test each question is worth.
When writing essay type items, please refer to 7.5.1 to 7.5.4.
{
i OTP 3.04 Rev. I j
Page 4 of 10
o Construct the evaluation items you have specified for the different objectives.
You might need to construct more than one item for each objective.
Keep in mind that it is also best to use no more than three or four different types of exam items (multiple choice, essay) on a single exam.
It is recommended you attempt to use no more than two different types of exam items and no more than three different sections, if you are teaching a course you have not taught before.
It is important to include the exact weight for each section on the exam, o
Assemble all like items together and try to organize each section of the exam from easy to more difficult, in order of operation, or in logical sequence.
When completed, review for trivial items, items that depend on the correct response or give the correct response to another item, and negative responses that tend to trick rather than measure ability.
Now, you are ready to write directions for each of the different sections or exam items you have decided to use, o
Make sure the directions include what the student is to do, how it is to be done, and where the response is to be recorded, o
Let a peer instructor review the test if never used before.
This step helps to identify exam items that may be ambiguous.
Then let his comments and the exam sit for a period of time (a day or two) in a secure place.
Now reread it for revision or editing.
o Construct a key for scoring, that clearly identifies the correct answer or answers.
OTP 3.04 Rev. 1 Page 5 of to
e.
(
o Analyze the results, after the exam has been administered, to determine how good the exam is and to start a bank for collecting exams to use in the future.
7.2 If the instructor plans on constructing an exam for an existing program, which has an exam bank he/she shall use OTp 5.02, "Formulating Tests From Question and Answer Bank and Administering Tests" to formulate the test.
7.3 If the instructor decides to creato new exam questions for an exam bank, he should develop those questions using the guidelinec for the construction of exam items giver. in section 7.1 of this procedure, and then proceed to enter them into the exam bank starting with section 7.5 of this procedure.
7.4 When an exam bank does not exist for a course and the instructor is creating both new exam items and a new exam he/she should formulate the exam using the method in 7.1 or an alternative method.
7.4.1 Exam items should be prepared using the Exam Item Form (appendix 8.1) and by following steps 7.5.1 through 7.5.8 of this procedure.
7.4.2 Upon approval of exam items the exam is constructed by the instructor and forwarded to the TAS for processing using the word processing request form.
7.4.3 The approved exam items, form (s), and a master copy of the exam and answer key shall be placed in the appropriate program file.
I OTp 3.04 Rev. 1 page 6 of 10
1 i
7.5 Creating Exam Items for Inclusion into an Existing Exam Bank 7.5.1 Obtain copies of the "Exam Item Question Form" (Appendix 8.1).
7.5.2 Complete the blocks for the Program, Duty area, LP# and Objectives on the "Exam Item Form."
There is to be only one question per form.
List the objectives to be tested by their identifying number.
Do not list more than five objectives.
Note:
If the exam question entered g) on the "Exam Item Form" is Non-Objective based, identify the objective by inputting a 9999.
7.5.3 Write the question in the space provided.
Include the figure numbers of any necessary figures with the
.;u *:s t i ot s, if figures are on file with the administration section.
7.5.4 In the space provided, write the answer required for full credit.
Verbatim responses should not be expected.
Acceptable alternative or additional information may be included parentheti-cally.
If a number of points are required to be addressed by a question, appropriate point values are to be assigned.
7.5.5 Complete the "Prepared By" block, attach a copy of the relevant learning objectives and forward the package to a subject matter expert (SME) and instructional technologist (IT) respectively for their review and comment.
7.5.6 The SME is to review and comment on the question and answer for technical accuracy, sign and return the package to the originating instructor.
4 OTP 3.04 Rev. 1 Page 7 of 10
7.5.7 The IT is to review and comment on the question and answer for the appropriateness of the question to the referenced objective (s) and verification that the wording of the question elicits the answer supplied in the key.
Upon completion the package is signed and returned to the originating instructors.
7.5.8 Upon disposition of comments the instructor forwards the newly created question and answer (s) to his training section supervisor (TSS) for approval.
7.5.9
.If the exam item is approved, the "Exam Item Form" is forwarded to the administrative section for exam bank entry.
7.5.10 The training administrative section supervisor or designee enters the item into the exam bank.
The individual entering the item signs the form and files it in the hard copy file of the appropriate program file.
4 7.6 A copy of the complated exam with answer key shall be placed in the appropriate program file.
8.0 Appendices 3.1 Exam Item Form i
i i
OTP 3.04 Rev. 1 Page 8 of 10
a s
i Appendix 8.1 Page 1 of 1 EXAM ITEM FORM PROGRAM DUTY AREA LP4 l
l l
l l
l l
Objectives 1
2 3
4 5
l l
1 Question:
Point (s) l ADMIN SECTION ONLY l l
LICENSE PROGRAMS ONLY l
l l
l Applicable NRC Section ( ) l l Record Number l
l 1 2
3 4
5 6
7 8 l 1
i t
1 i
OTP 3.04 Rev. 1 Page 9 of 10
- r,.
1 i
t Answer:
k Prepared By:
Signature
/Date Approved for Technical Accuracy:
Signature (SME)
/Date i
Approved for Instructional Adequacy:
Signature (IT)
/Date Approved for
~
Signature (TSS)
/Date Entered On Signature
/Date t i l
OTP 3.04 Rev. I t
Page 10 of 10 t
L