ML13343A291

From kanterella
Revision as of 10:33, 4 November 2019 by StriderTol (talk | contribs) (Created page by program invented by StriderTol)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
PV-2013-11 ES-401-9 Draft Written Exam Comments
ML13343A291
Person / Time
Site: Palo Verde  Arizona Public Service icon.png
Issue date: 11/09/2013
From: Kelly Clayton
Operations Branch IV
To:
Arizona Public Service Co
laura hurley
References
Download: ML13343A291 (13)


Text

ES-401 PV-2013-11 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 N S E Part of the worksheet words are cutoff-need to fix this for final submittal.

2 H 3 X N Agreed to edit the justification page. Still not done on rev 5 of comments form.

S Completed and question is now Sat.

3 F 3 B S E Explain credibility of ESD as a distracter with S/G pressures stable. This affects two distracters.

This has the appearance of SRO-only to me (Selecting the appropriate procedure).

You need to emphasize information in the worksheet that supports this question as 4 H 2 X X N entry condition knowledge that an RO is required to know..ESD is not credible with SG pressure high and stable for both SGs.

PV working on new question since ESD not credible for given stem conditions.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S Completed rewrite of question and it is now Sat.

5 H 3 B S 6 H 3 B S 7 H 2 N S D does not seem credible. Question is asking for the response of the system.

U Distracter D requires operator action and stands as an outlier to the other distracters.

8 H 2 X X B This is not a controller failure or a positioner failure, but a sensor failure. KA mismatch. This is also on the op test (either audit or NRC 2013, cant remember) A controller failure would allow heaters, spray to come on / off depending on failure (ie S multiple things occur).

Rewrote the question and is now Sat.

9 H 2 N S 10 H 2 N S Obtaining CRS permission to manually protect the plant upon the failure of an automatic protective function is not credible. This affects two distracters. Also, does E

this question really test applicant knowledge on operation within in the facilitys license and amendments per the K/A?

C and D not completely credible since CRS permission is not needed for an 11 F 2 X N immediate action failure. Once you state the first part of distracters C and D then the second part doesnt tie well at all. You could have use one button for train 2 or use both buttons to ensure proper AFAS or MSIS or something like that.

S Agreed to replace verbiage of may be and use immediate and also tie the KA better to license requirement in the justification section. Question is now Sat.

C not credible distracter. When does a pump ever require going to start twice for the E

12 H 2 X B given conditions?

S Edit C to state can be started by going to start a second time. Question is now Sat.

U The applicant should have to determine that is necessary to prevent an uncontrolled 13 F 3 X B restoration of RCP seal injection during the Blackout EOP. Question should be phrased to ask why charging pumps handswitches are pulled to Lock. Having the uncontrolled restoration of RCP seal injection in the answer matrix coupled with good Page 2 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only distracters would better fit the K/A.

The KA is reason for the action, not the action as the question is written to S Will rework the question. Question is now Sat.

HANDOUT E I cant read the drawing you are providing-is this the quality of the hand-out you are 14 H 2 N providing?

S Updated camera with better picture for the exam. NRC email would not accept new, better picture due to file size. It will come with final DVD submittal.

Discussion on credibility of Distracter D is loss of PKC-M43 but I believe it is E

15 H 3 M supposed to be PKD-M44.

S Typo changed. Question is now Sat.

16 H 4 N S 17 F 3 B S Used on 2010 exam.

18 H 4 B S 19 H 3 N S HANDOUT 20 H 4 N S E To preclude the possibility of two potentially correct answers (B and D), what is the definition of Immediately? Some facilities define immediately as within a one hour space of time.

Several potential issues with this question. First, the KA is asking for reasons for aborting a startup when you lose a SR nuclear instrument channel, and the focus is on a sub-system of the startup channel. The reason should be something along the lines of two SU channels are required to prevent one channel from masking a reactor 21 H 2 X X X N criticality event or something along those lines. Secondly, the 1/m plot can be done electronically or manually, which could be appealed later. Finally, can an applicant argue your definition of immediate versus within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />?

Resample KA since PV is not required to stop SU with loss of a SR instrument so cant meet the KA. New KA , New question, D Distracter needs work.

Do we need to state that the primary reason for doing the action because A looks like it might be argued as correct unless we clarify? Page 3 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S PV made changes as recommended and question is now Sat.

E D does not seem plausible. Continue fuel movement with the given parameters in 22 F 2 X B the stem?

S Add more credible discussion to justification section. Done and Question is now Sat.

23 H 2 B S Need more discussion on why distracter B is credible. It doesnt seem plausible to E

24 H 3 X B secure ALL charging pumps under the given conditions.

S Add more info to justification section. Done and Question is now Sat.

25 F 3 N S 26 H 3 N S Editorial: Indicate steam tables on Proposed references to be provided for the 27 H 3 B S pedigree of this question. Done.

E Distracter symmetry. Distracters C and D provide reasons for their respective level but A and B do not.

Concerned with response in stem versus an initial response, also concerned with C 28 H 2 X M and D distracters because of wording-control at lower level, implying lower level than before the trip or present conditions? Distracters C and D have reasons for the S answer while A and B do not, need to fix this.

Done and Question is now Sat.

29 F 2 N S U Knowing the class is not enough knowledge for power supply KAs.

30 F 1 X N S Rewriting the question. Done and Question is now Sat.

31 F 2 B S E Justify credibility in justification section. PV reminded CE that entry condition for FR 32 H 2 N S are RO knowledge. CE agreed. Done and Question is now Sat.

33 H 2 N S E Is it really plausible that Nuclear Cooling Water Surge Tank NORMAL makeup would be done manually?

34 F 2 X N Distracters A and B not credible for normal makeup to a tank-always auto.

S Page 4 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Change question to different sources of water. Done and Question is now Sat.

E Partial cueing. Answer D has containment isolation valve in it while stem has containment isolation valve in it. Need to think of another way to ask or setup the 35 H 3 X N answer to avoid this condition.

PV added containment penetration verbiage to the stem in addition to words already S there to mask any cueing that might have occurred. Done and Question is now Sat.

E Editorial Comment: Distracter/Correct Answer mis-labeled in accompanying 36 H 3 N S background paperwork. Done and Question is now Sat.

Using the worksheet from the NUREG makes it easier to review. For this question, it is hard to determine what system 012 is without guessing from the question or going 37 F 2 N S to the outlines. You should restate the system after 012 to make the worksheet more clear. Looked up and System 012 is RPS.

TSB are SRO -only questions. The reason it is marked Edit instead of Unsat is E because with some minor changes it will be okay. A safety limit is RO knowledge.

The supporting references need to focus on the SL to make sure it is not appealable.

Also, distracters C and D are partially not credible. Three as a choice is not credible.

38 F 3 X X N You could focus the question on the temperature alone and the adjustment for burnup (possibly) as these are CFR requirements and this would be RO knowledge.

S Working on an edit to remove the three portion of the question. Done and Question is now Sat.

E Explanation for A is incorrect. You state SIAS is correct and you mean MSIS because 39 H 3 X B S SIAS is not in distracter A. Done and Question is now Sat.

40 F 3 B S 41 F 2 B S Used on 2012 exam.

42 F 2 B S Does not seem credible that Essential AFW flow path would be isolated with a E standard MSIS. This affects two distracters.

43 H 3 X N For distracters A and D it doesnt seem believable that the essential flowpath for AFW S would be isolated by an AFAS. Please explain.

Rewrite the question. Done and Question is now Sat.

44 H 2 X N E Partial cueing with level in answer D and stem level transmitters. Page 5 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S Make the edits discussed. Done and Question is now Sat.

U LOD=1 too easy.

45 H 4 B PV is reworking the question or maybe new KA.

S New KA, New question, and Question is now Sat.

46 H 3 B S 47 F 2 N S Partial KA mismatch (did not predict impact on) also with answer D this is the only E credible choice as written. I would recommend making this a 2 X 2. Using a 48 H 2 X N procedure does not mean that you have to choose between procedures in the answer choice, it means that you have procedure steps to take in the answer.

S PV is reworking the question. Done and Question is now Sat.

49 F 2 X N S 50 H 2 X M S Distracter A is a nothing happens distracter and NUREG-1021 discourages this E because could appear as a trick to an applicant. Why would the NRC exam have an answer where all these facts are given and the answer is nothing happens? This 51 H 2 X B makes the distracter not credible.

S PV will make a new A distracter as EDG amps will raise. Done and Question is now Sat.

A does not seem credible and also appears as an outlier to the other distracters and E correct answer.

Distracter A is a nothing happens distracter and NUREG-1021 discourages this 52 H 2 X B because could appear as a trick to an applicant. Why would the NRC exam have an answer where all these facts are given and the answer is nothing happens? This makes the distracter not credible.

S PV will modify A to be RU2 alarms and isolates NCW to ECW. Re-sequenced the choices and made changes as recommended and Question is now Sat.

53 H 3 B S E K/A requires ability to locate AND operate knowledge. This question does not address location and therefore needs further enhancement in this regard.

54 F 2 X N PV will modify to include location in A and B and use 90 ft in containment. Done and S Question is now Sat. Page 6 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 55 F 2 N S Using the worksheet from the NUREG makes it easier to review. For this question, it E

is impossible to determine what the system is without guessing from the question or going to the outlines. You should restate the system name and number after the KA to make the worksheet more clear. Looked up and the system is 002 (Reactor Coolant) for this question. The system ID you are using is wrong.

First, what does within the capacity of CVCS system mean? Is it defined as all three 56 H 3 X X N charging pumps running and letdown secured? What letdown lines in service with which orifices? Reference needs to support this or it could be appealed. This determines if A and B are credible but not correct for this stem and answer. Also, isnt this usually covered during the Op test scenarios?

PV will edit the stem. Also add justification for why different from this event on op test.

S PV made all recommended changes and Question is now Sat.

Knowing the generic class of bus is not enough knowledge for bus power supply KAs.

U The KA was intended to ask a specific bus power supply for the Charging pumps.

Also, the NUREG discourages adding information to the question (such as control power in this case). Just ask:

What is the power supply to the Charging pumps?

A. CCP-01 D21 57 F 2 X N CCP-02 D22 CCP-03 B23 B. CCP-01 D22 CCP-02 D21 CCP-03 B23 C. etc S

PV will rework question as recommended. Done and Question is now Sat.

58 H 3 N S E Distracter A does not seem credible with rods going in due to Tave reduction.

59 H 2 X N S Done and Question is now Sat.

E Partial KA mismatch. This question does not use procedures as designed in that you 60 F 3 X N put the procedure action in the stem to place H2 recombiners in service and this portion should be in the answer choices. Page 7 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only PV will rework question as recommended. Look up A to make sure it is not correct S Question is now Sat.

61 F 2 B S On 2010 exam.

62 F 2 N S 63 H 2 N S Two possible correct answers. 40OP-9MT02 4.6.1step 8 says that DFWCS may be placed in Single Element Control. Step 4.6.3.4 has a note to consider boron E equalization but does not require it. This affects D and B.

64 H 3 X N Procedure reference is wrong in justification section (MT01 vs MT02).

PV needs to look at it again and then talk again. After discussion this statement in S the procedure is that it may go into single element during testing but not placed in single element by operator manipulation, therefore B is not correct as written. Edit on reference done and Question is now Sat.

Answer matrix overloaded. Asking for five pieces of information to achieve the E correct answer. Only need two pieces to identify the correct answer. This affects all distracters and the correct answer. Recommend making this 2x2 style answer format.

65 H 4 X N Not balanced. All info not needed to answer question. Worksheet issue with system S ID 072 (had to go look up system 072 - ARM)

PV will rework to 2 X 2. Done and Question is now Sat.

LOD=1. When is a peer check on the control boards, under the given conditions of U this stem, not required?

66 F 1 X N Also, If peer check is not required then peer checker is moot point for distracter B and S D. Makes them not credible.

PV will rework. Done and Question is now Sat.

67 F 3 N S U Need better reason C and D are credible to part (1) of the answer.

68 F 4 X X N This is an SRO-only topic per 55.46b(6).

E PV will sample another KA. Page 8 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Question needs indep. Added to A and concurrent added to D and more words to make D more credible.

S PV made mods and questions is now Sat.

Answer D much longer than others, potential cueing to it. Make distracter B longer by E

69 F 3 X X B changing it to same as D except use 4 day instead of 14 days at the end.

S PV will make the change. Done and Question is now Sat.

70 F 2 N S E Partial K/A mismatch. K/A requires knowledge of the process for controlling equipment configuration not knowledge of controlling specific equipment 71 F 3 X B configurations. This question does not seem to address the process aspect.

S PV will rework. Done and Question is now Sat.

72 F 2 X N S 73 F 3 N S E Make distracter C 1) Normal OP (2)Abnormal OP 74 F 3 X N S PV will make the change. Done and Question is now Sat.

75 F 2 X B S 76 H 3 N S HANDOUT E You have cueing in the answer C and distracter D with use of P/T limits and not having this in the other two distracters. Your justifications for credible distracters is 77 H 3 X N missing.

S Resolve A and B put in P/T limits to correct cueing.

PV made changes as recommended and question is now Sat.

RO level question.

U RO knowledge, justification has typo, also. The SG target levels are 45-60% not 45-78 H 3 X N 65%.

S New question written (original was new question also ) and is now Sat.

E Page 8 of reference material you added a comment for loss of PKA-M41 and 79 H 3 X N distracter C but distracter C is actually the answer.

S Made change to reference and question is now Sat. Page 9 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only K/A incomplete in question as given. K/A requires applicant be able to explain and E apply. Application portion is apparent with determination of operability. Explanation part needs to be added.

80 F 2 X N KA includes explanation of the system limit, which is not tested for this question.

Partial KA mismatch.

S PV modified the question to include explanation as part of selection and question is now Sat.

81 H 3 N S HANDOUT 82 H 3 N S HANDOUT Stem states that RU-142 channels are stable. Should they be in alarm? Please E clarify to alleviate any ambiguity.

Justification for credible but not correct is missing on many questions. IS RU-141 or 83 H 3 X N 142 high? This is needed for entry to the procedure and all that is stated is stable, so technically there is no correct answer as written.

S PV modified question and it is now Sat.

84 H 3 N S Is D a credible distracter? Appears as an outlier with two FRP answers and one E

LOAF. Recommend making D also a LOAF with different application than C.

85 H 3 X N Distracter symmetry for D. Change it to a LOAF, etc.

S PV made changes and question is now Sat.

E Recommend making the first part of the answer on B the same as the first part on C. Throttling HPSI pump B stands as an outlier to the other answers. Eliminating this possibility will balance the distracters.

86 H 2 X N Distracter symmetry for B. Change it to same intro part as C, then have balanced 2 X 2.

S PV made changes and question is now Sat.

87 H 2 N S HANDOUT E Same comment as question 87. There are three not met to only one met in the answer possibilities with the met being incorrect. Add one more met to balance distracters to alleviate immediately discarding it as an outlier.

88 H 3 X N Reference states distracter C for CTPC-1 and this is the correct answer. Distracter symmetry for D. Same as other issues above. Modify D to create balanced 2 X 2.

S PV made changes and question is now Sat. Page 10 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E Two correct answers. Given stem info is >1700#

I think A is correct as written. Greater than 1700 psig (1750 > 1700) for seal 2 pressure. I also think that B could be correct, too.

89 H 3 X N Justification is wrong for distracter A and B. Fix that.

PV made changes to A and B to state that Controlled BO must be isolated when BO S flow is >9.5gpm. Question is now Sat.

U Entry level conditions for AOPs are RO level questions.

If CEDMs are air-cooled, hence HVAC procedure addresses this, then providing no cooling water in stem allows applicant to eliminate A and B (ie not credible). The 90 H 2 X X N remaining part of the question is really entry conditions for the 9ZZ17 procedure, E which is RO knowledge.

You need to tie the stem conditions with why each is inoperable in the justification S section. Still not done on rev 5 of comments form. Done and Question is now Sat.

91 H 3 N S HANDOUT 92 H 2 N S 93 F 2 N S I would add additional information to the stem to indicate the status of manning for E Units 1 and 3. The implication is that one of the other units is providing the required additional RO in order for D to be correct but it is not stated in the stem. This could 94 F 2 X N lead to an argument of two possible correct answers. I would remove the proviso from D and include the required information in the stem to alleviate any ambiguity.

S PV made the recommended changes and question is now Sat.

95 F 3 N S HANDOUT 96 H 2 B S HANDOUT I would argue that by stating actions to manage risk in the stem information and utilizing the procedure Assessment and Management of Risk When Performing U

Maintenance in Modes 1 and 2 would eliminate A and C as credible distracters.

Need to clarify in the the stem to state what is the minimum accumulated time in a 97 F 2 X X N week. The stem cues to the answer because of words used actions to manage risk and in the answer Management of Risk. Procedure.

S Get another KA or find SRO LO for this question and fix cueing issue.

Changed KA, wrote new question and it is now Sat.

98 F 3 B S Page 11 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 12 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 99 H 2 N S

  • Security Related Question** HANDOUT We need to make sure there are not too many EAL items on the exam as a whole-100 H 2 N S look at admin JPM SRO-5 and other questions on this written exam. Justification due to continued weakness in EALs and Station felt like it should be a focus area for the exam.

B= 26 (34.7%) F= 33 (44%) E= 34 RO TOTALS: M= 4 (5.3%) H= 42 (56%) U= 10 Additional Notes:

N= 45 (60%)

B= 2 (8%) F= 6 (24%) E= 9 SRO TOTALS: M= 0 H= 19 (76%) U= 3 Additional Notes:

N= 23 (92%)

GENERAL COMMENT

S:

1. Bank / Mod / New questions are indicated in their respective column field above.
2. Chief Examiner comments are indicated in blue where multiple comments exist, otherwise they are in black.
3. Average difficulty is __2.46 ___ on the RO exam and _2.52__ on the SRO exam.
4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 2 41.2 = 2 43.2 = 4 41.3 = 0 43.3 = 0 41.4 = 8 43.4 = 1 41.5 = 10 43.5 = 17 41.6 = 3 43.6 = 1 41.7 = 18 43.7 = 0 41.8 = 7 41.9 = 0 41.10 = 24 41.11 = 0 41.12 = 2 41.13 = 0 41.14 = 1 Page 12 of 13 OBDI 202 - IOLE Process

ES-401 PV-2013-11 13 Form ES-401-9

5. The answer distribution is: RO / SRO A = 18 (24%) / 3 (12%)

B = 18 (24%) / 6 (24%)

C = 15 (20%) / 9 (36%)

D = 24 (32%) / 7 (28%)

6. There are 10 questions with attachments provided. Questions 14, 19, 77, 81, 82, 87, 91, 95, 96, and 100 require hand-outs Page 13 of 13 OBDI 202 - IOLE Process