ML22265A107

Task Order No. 31310022F0087 Under Contract No. 31310022A0002

ML22265A107
Person / Time
Issue date:	09/22/2022
From:	Jeffrey Mitchell Acquisition Management Division
To:	Pacific Research & Evaluation
References
31310022A0002
Download: ML22265A107 (22)

Text

PAGE OF PAGES ORDER FOR SUPPLIES OR SERVICES IMPORTANT: Mark all packages and papers with contract and/or order numbers. 1 22

1. DATE OF ORDER 2. CONTRACT NO. (If any) 6. SHIP TO:

31310022A0002

a. NAME OF CONSIGNEE 09/22/2022

3. ORDER NO. 4. REQUISITION/REFERENCE NO.

Nuclear Regulatory Commission 31310022F0087 EDO-22-0007

5. ISSUING OFFICE (Address correspondence to) b. STREET ADDRESS U.S. NRC - HQ 11555 Rockville Pike Acquisition Management Division Mail Stop: TWFN-07B20M Washington DC 20555-0001

c. CITY d. STATE e. ZIP CODE Rockville MD 20852-2738

7. TO: f. SH P VIA

a. NAME OF CONTRACTOR PACIFIC RESEARCH & EVALUATION LLC 8. TYPE OF ORDER

b. COMPANY NAME

a. PURCHASE X b. DELIVERY

c. STREET ADDRESS REFERENCE YOUR:

3507 S. CORBETT AVENUE Except for billing instructions on the reverse, this delivery order is subject to instructions contained on this side only of this form and is Please furnish the following on the terms issued subject to the terms and and conditions specified on both sides of conditions of the above-numbered

d. CITY e. STATE f. Z P CODE this order and on the attached sheet, if contract.

PORTLAND OR 972394326 any including delivery as indicated.

9. ACCOUNTING AND APPROPRIATION DATA 10. REQUISITION NG OFFICE See Schedule Off of the Executive Director

11. BUS NESS CLASS FICATION (Check appropriate box(es)) 12. F.O B. POINT

a. SMALL X b. OTHER THAN SMALL c. DISADVANTAGED d. WOMEN-OWNED e. HUBZone

f. SERVICE-DISABLED g. WOMEN-OWNED SMALL BUS NESS (WOSB)

h. EDWOSB VETERAN-OWNED ELIGIBLE UNDER THE WOSB PROGRAM

13. PLACE OF 14. GOVERNMENT B/L NO. 15. DELIVER TO F.O B. POINT 16. DISCOUNT TERMS ON OR BEFORE (Date)

a. INSPECTION b. ACCEPTANCE Destination Destination 30

17. SCHEDULE (See reverse for Rejections)

QUANTITY UNIT QUANTITY ITEM NO. SUPPLIES OR SERVICES ORDERED UNIT PRICE AMOUNT ACCEPTED (a) (b) (c) (d) (e) (f) (g)

GSA Contract #: 47QRAA18D006Y Pacific Research & Evaluation has certified they are a small business under NAICS Code 541618 and overrules the OF347 box 11.

The U.S. Nuclear Regulatory Commission Continued ...

18. SHIPPING POINT 19. GROSS SHIPPING WEIGHT 20. INVOICE NO. 17(h)

TOTAL (Cont.

pages)

21. MAIL INVOICE TO:

a. NAME $280,839.50 SEE BILLING INSTRUCTIONS b. STREET ADDRESS ON REVERSE (or P.O. Box) 17(i)

GRAND TOTAL

c. CITY d. STATE e. Z P CODE

$280,839.50

22. UNITED STATES OF 23. NAME (Typed) 09/22/2022 AMERICA BY (Signature) JEFFREY R. MITCHELL TITLE: CONTRACTING/ORDER NG OFFICER AUTHORIZED FOR LOCAL REPRODUCTION OPTIONAL FORM 347 (Rev. 2/2012)

PREVIOUS EDITION NOT USABLE Prescribed by GSA/FAR 48 CFR 53.213(f)

ORDER FOR SUPPLIES OR SERVICES PAGE NO SCHEDULE - CONTINUATION 2 MPORTANT: Mark all packages and papers with contract and/or order numbers.

DATE OF ORDER CONTRACT NO. ORDER NO.

09/22/2022 31310022A0002 31310022F0087 ITEM NO. SUPPLIES/SERVICES QUANTITY UNIT UNIT AMOUNT QUANTITY ORDERED PRICE ACCEPTED (a) (b) (c) (d) (e) (f) (g) hereby issues BPA Call 31310022F0087 entitled, "Evaluation of the NRC Knowledge Management Program" This call is issued under BPA number47QRAA18D006Y -

31310022A0002.

Accounting Info:

2022-X0200-FEEBASED-7M-7MD001-1061-11-6-225-251A-11-6-225-1061 Period of Performance: 09/19/2022 to 10/31/2023 00001 Evaluation of the NRC Knowledge Management 280,839.50 Program [Labor Hour]

Line Item Ceiling: $280,839.50 Incrementally Funded Amount: $280,839.50 The obligated amount of award: $280,839.50.

The total for this award is shown in box 17(i).

TOTAL CARR ED FORWARD TO 1ST PAGE (ITEM 17(H)) $280,839.50 AUTHORIZED FOR LOCAL REPODUCTION OPTIONAL FORM 348 (Rev. 4/2006)

PREVIOUS EDITION NOT USABLE Prescribed by GSA FAR (48 CFR) 53.213(f)

31310022A0002/31310022F0087 2 - Section 2 ..................................................................................................................................4 2.1 NRC SPECIFIC INFORMATION .........................................................................................4 B - Continuation Pages .................................................................................................................5 B.1 BRIEF PROJECT TITLE AND WORK DESCRIPTION.......................................................5 B.2 TYPE OF CONTRACT (JULY 2020)...................................................................................5 B.3 CONSIDERATION AND OBLIGATION- LABOR-HOUR CONTRACT ...............................5 B.4 PRICE/COST SCHEDULE..................................................................................................5 C - Contract Clauses .....................................................................................................................6 C.1 PACKAGING AND MARKING ............................................................................................6 C.2 BRANDING .........................................................................................................................6 C.3 INSPECTION AND ACCEPTANCE BY THE NRC (SEP 2013) .........................................6 C.4 PLACE OF DELIVERY-REPORTS.....................................................................................6 C.5 PERIOD OF PERFORMANCE ...........................................................................................6 C.6 KEY PERSONNEL. (JAN 1993) .........................................................................................6 C.7 2052.215-71 CONTRACTING OFFICER REPRESENTATIVE AUTHORITY. (OCT 1999)7 C.8 52.244-2 SUBCONTRACTS. (JUN 2020)...........................................................................9 C.9 STATEMENT OF WORK ..................................................................................................12 D - Contract Documents, Exhibits and Attachments ...................................................................22 Page 3

31310022A0002/31310022F0087 2 - Section 2 2.1 NRC SPECIFIC INFORMATION CONTRACTOR ACCEPTANCE OF TASK ORDER 31310022F0087 Acceptance of Task Order No: 31310022F0087 should be made by having an official, authorized to bind your organization.

_______________________ Name

_______________________ Title

_______________________ Date Page 4

31310022A0002/31310022F0087 C - Contract Clauses C.1 PACKAGING AND MARKING (a) The Contractor shall package material for shipment to the NRC in such a manner that will ensure acceptance by common carrier and safe delivery at destination. Containers and closures shall comply with the Surface Transportation Board, Uniform Freight Classification Rules, or regulations of other carriers as applicable to the mode of transportation.

(b) On the front of the package, the Contractor shall clearly identify the contract number under which the product is being provided.

(c) Additional packaging and/or marking requirements are as follows: N/A.

C.2 BRANDING The Contractor is required to use the statement below in any publications, presentations, articles, products, or materials funded under this contract/order, to the extent practical, in order to provide NRC with recognition for its involvement in and contribution to the project. If the work performed is funded entirely with NRC funds, then the contractor must acknowledge that information in its documentation/presentation.

Work Supported by the U.S. Nuclear Regulatory Commission (NRC), Office of the Executive Director for Operations, under Contract/order number 31310022A0002_31310022F0087.

C.3 INSPECTION AND ACCEPTANCE BY THE NRC (SEP 2013)

Inspection and acceptance of the deliverable items to be furnished hereunder shall be made by the NRC Contracting Officers Representative (COR) at the destination, accordance with FAR 52.247 F.o.b. Destination.

Contract Deliverables:

Refer to the deliverable section in the Statement of Work.

C.4 PLACE OF DELIVERY-REPORTS The items to be furnished hereunder shall be delivered, with all charges paid by the Contractor, to:

Electronic copies to:

a. Contracting Officers Representative (COR) [See: 2052.215-71 CONTRACTING OFFICER REPRESENTATIVE AUTHORITY. (OCT 1999)]

b. Jeffrey.Mitchell@nrc.gov, Contracting Officer (CO)

C.5 PERIOD OF PERFORMANCE This contract shall commence on 09/19/2022 and will expire on 10/31/2023.

C.6 KEY PERSONNEL. (JAN 1993)

Page 6

31310022A0002/31310022F0087 (1) Technical direction to the contractor which shifts work emphasis between areas of work or tasks, authorizes travel which was unanticipated in the Schedule (i.e., travel not contemplated in the Statement of Work (SOW) or changes to specific travel identified in the SOW), fills in details, or otherwise serves to accomplish the contractual SOW.

(2) Provide advice and guidance to the contractor in the preparation of drawings, specifications, or technical portions of the work description.

(3) Review and, where required by the contract, approval of technical reports, drawings, specifications, and technical information to be delivered by the contractor to the Government under the contract.

(c) Technical direction must be within the general statement of work stated in the contract. The COR does not have the authority to and may not issue any technical direction which:

(1) Constitutes an assignment of work outside the general scope of the contract.

(2) Constitutes a change as defined in the "Changes" clause of this contract.

(3) In any way causes an increase or decrease in the total estimated contract cost, the fixed fee, if any, or the time required for contract performance.

(4) Changes any of the expressed terms, conditions, or specifications of the contract.

(5) Terminates the contract, settles any claim or dispute arising under the contract, or issues any unilateral directive whatever.

(d) All technical directions must be issued in writing by the COR or must be confirmed by the COR in writing within ten (10) working days after verbal issuance. A copy of the written direction must be furnished to the contracting officer. A copy of NRC Form 445, Request for Approval of Official Foreign Travel, which has received final approval from the NRC must be furnished to the contracting officer.

(e) The contractor shall proceed promptly with the performance of technical directions duly issued by the COR in the manner prescribed by this clause and within the COR's authority under the provisions of this clause.

(f) If, in the opinion of the contractor, any instruction or direction issued by the COR is within one of the categories as defined in paragraph (c) of this section, the contractor may not proceed but shall notify the contracting officer in writing within five (5) working days after the receipt of any instruction or direction and shall request the contracting officer to modify the contract accordingly. Upon receiving the notification from the contractor, the contracting officer shall issue an appropriate contract modification or advise the contractor in writing that, in the contracting officer's opinion, the technical direction is within the scope of this article and does not constitute a change under the "Changes" clause.

Page 8

31310022A0002/31310022F0087 (g) Any unauthorized commitment or direction issued by the COR may result in an unnecessary delay in the contractor's performance and may even result in the contractor expending funds for unallowable costs under the contract.

(h) A failure of the parties to agree upon the nature of the instruction or direction or upon the contract action to be taken with respect thereto is subject to 52.233 Disputes.

(i) In addition to providing technical direction as defined in paragraph (b) of the section, the COR shall:

(1) Monitor the contractor's technical progress, including surveillance and assessment of performance, and recommend to the contracting officer changes in requirements.

(2) Assist the contractor in the resolution of technical problems encountered during performance.

(3) Review all costs requested for reimbursement by the contractor and submit to the contracting officer recommendations for approval, disapproval, or suspension of payment for supplies and services required under this contract.

(4) Assist the contractor in obtaining the badges for the contractor personnel.

(5) Immediately notify the Security Branch, Division of Facilities and Security (SB/DFS) (via e-mail) when a contractor employee no longer requires access authorization and return of any NRC issued badge to SB/DFS within three days after their termination.

(6) Ensure that all contractor employees that require access to classified Restricted Data or National Security Information or matter, access to sensitive unclassified information (Safeguards, Official Use Only, and Proprietary information) access to sensitive IT systems or data, unescorted access to NRC controlled buildings/space, or unescorted access to protected and vital areas of nuclear power plants receive approval of SB/DFS prior to access in accordance with Management Directive and Handbook 12.3.

(7) For contracts for the design, development, maintenance or operation of Privacy Act Systems of Records, obtain from the contractor as part of closeout procedures, written certification that the contractor has returned to NRC, transferred to the successor contractor, or destroyed at the end of the contract in accordance with instructions provided by the NRC Systems Manager for Privacy Act Systems of Records, all records (electronic or paper) which were created, compiled, obtained or maintained under the contract.

(End of Clause)

C.8 52.244-2 SUBCONTRACTS. (JUN 2020)

(a) Definitions. As used in this clause-Page 9

31310022A0002/31310022F0087 Approved purchasing system means a Contractor's purchasing system that has been reviewed and approved in accordance with Part 44 of the Federal Acquisition Regulation (FAR).

Consent to subcontract means the Contracting Officer's written consent for the Contractor to enter into a particular subcontract.

Subcontract means any contract, as defined in FAR Subpart 2.1, entered into by a subcontractor to furnish supplies or services for performance of the prime contract or a subcontract. It includes, but is not limited to, purchase orders, and changes and modifications to purchase orders.

(b) When this clause is included in a fixed-price type contract, consent to subcontract is required only on unpriced contract actions (including unpriced modifications or unpriced delivery orders), and only if required in accordance with paragraph (c) or (d) of this clause.

(c) If the Contractor does not have an approved purchasing system, consent to subcontract is required for any subcontract that-(1) Is of the cost-reimbursement, time-and-materials, or labor-hour type; or (2) Is fixed-price and exceeds-(i) For a contract awarded by the Department of Defense, the Coast Guard, or the National Aeronautics and Space Administration, the greater of the simplified acquisition threshold, as defined in FAR 2.101 on the date of subcontract award, or 5 percent of the total estimated cost of the contract; or (ii) For a contract awarded by a civilian agency other than the Coast Guard and the National Aeronautics and Space Administration, either the simplified acquisition threshold, as defined in FAR 2.101 on the date of subcontract award, or 5 percent of the total estimated cost of the contract.

(d) If the Contractor has an approved purchasing system, the Contractor nevertheless shall obtain the Contracting Officer's written consent before placing the following subcontracts:

All new hires under the prime, consultants and/or subcontractors not part of the proposal.

(e)(1) The Contractor shall notify the Contracting Officer reasonably in advance of placing any subcontract or modification thereof for which consent is required under paragraph (b), (c), or (d) of this clause, including the following information:

(i) A description of the supplies or services to be subcontracted.

(ii) Identification of the type of subcontract to be used.

(iii) Identification of the proposed subcontractor.

Page 10

31310022A0002/31310022F0087 (iv) The proposed subcontract price.

(v) The subcontractor's current, complete, and accurate certified cost or pricing data and Certificate of Current Cost or Pricing Data, if required by other contract provisions.

(vi) The subcontractor's Disclosure Statement or Certificate relating to Cost Accounting Standards when such data are required by other provisions of this contract.

(vii) A negotiation memorandum reflecting-(A) The principal elements of the subcontract price negotiations; (B) The most significant considerations controlling establishment of initial or revised prices; (C) The reason certified cost or pricing data were or were not required; (D) The extent, if any, to which the Contractor did not rely on the subcontractor's certified cost or pricing data in determining the price objective and in negotiating the final price; (E) The extent to which it was recognized in the negotiation that the subcontractor's certified cost or pricing data were not accurate, complete, or current; the action taken by the Contractor and the subcontractor; and the effect of any such defective data on the total price negotiated; (F) The reasons for any significant difference between the Contractor's price objective and the price negotiated; and (G) A complete explanation of the incentive fee or profit plan when incentives are used. The explanation shall identify each critical performance element, management decisions used to quantify each incentive element, reasons for the incentives, and a summary of all trade-off possibilities considered.

(2) The Contractor is not required to notify the Contracting Officer in advance of entering into any subcontract for which consent is not required under paragraph (b), (c), or (d) of this clause.

(f) Unless the consent or approval specifically provides otherwise, neither consent by the Contracting Officer to any subcontract nor approval of the Contractor's purchasing system shall constitute a determination-(1) Of the acceptability of any subcontract terms or conditions; (2) Of the allowability of any cost under this contract; or Page 11

31310022A0002/31310022F0087 (3) To relieve the Contractor of any responsibility for performing this contract.

(g) No subcontract or modification thereof placed under this contract shall provide for payment on a cost-plus-a-percentage-of-cost basis, and any fee payable under cost-reimbursement type subcontracts shall not exceed the fee limitations in FAR 15.404-4(c)(4)(i).

(h) The Contractor shall give the Contracting Officer immediate written notice of any action or suit filed and prompt notice of any claim made against the Contractor by any subcontractor or vendor that, in the opinion of the Contractor, may result in litigation related in any way to this contract, with respect to which the Contractor may be entitled to reimbursement from the Government.

(i) The Government reserves the right to review the Contractor's purchasing system as set forth in FAR Subpart 44.3.

(j) Paragraphs (c) and (e) of this clause do not apply to the following subcontracts, which were evaluated during negotiations:

None.

(End of clause)

C.9 STATEMENT OF WORK Project

Title:

Evaluation of the NRC Knowledge Management Program

Background

The Nuclear Regulatory Commission (NRC) defines Knowledge Management (KM) as a continuous, disciplined, and timely process of identifying, collecting, and using information to better accomplish the job. The NRC practiced the capturing, preservation, sharing, and use of organizational knowledge long before the term Knowledge Management came into common use. In 2006 with the establishment of a formal KM Program, the NRC initiated a more structured and systematic approach to KM. Since then, NRC has established over 37 Communities of Practice that serve to create opportunities for learning, building capability, and sharing knowledge. The agency has also established valuable KM tools like Nuclepedia, the KM Toolkit for Supervisors, other OCHCO Toolkits, the Career Mentoring Program, Weekly KM sessions for inspectors, KNOWvember, and Design Your KM Approach.

In 2017, the KM Programs focus shifted to address how the agency can better integrate KM practices and approaches into daily operations and do so systematically. NRC developed the Knowledge Management Strategy 2018-2022 (ML20023B283) [Attachment No. 1], including a KM Program goal to Maintain NRCs regulatory and technical excellence by ensuring critical knowledge is captured, available and used by all NRC staff, present and future. The KM Strategy defined the following expected outcomes or desired state of the KM Program for 2022:

Knowledge Management becomes routine and is integrated into daily work activities and work processes.

Knowledge is captured at all levels of the organization which results in improved productivity, allowing people to do their jobs more effectively.

Page 12

31310022A0002/31310022F0087 Knowledge is consistently managed and readily accessible and used, improving the organizations ability to execute operations more efficiently.

In April 2022, NRC issued NUREG-1614, Vol. 8, "Strategic Plan, Fiscal Years 2022-2026" (ML22067A170). The Strategic Plan defines NRCs strategic goals, objectives, and key activities that will be used to achieve the agencys mission. Strategic Objective 2.2 states, "Enable the workforce to carry out the agencys mission by leveraging modern technology, innovation, and knowledge management support data-driven decisions in an evolving regulatory landscape."

Additionally, Strategy 2.3.4 states, "Improve knowledge management by identifying and capturing critical information and leveraging the agencys investment in modern information management and technology to enhance information accessibility and searchability." This further emphasizes how the KM Program is vital for NRC to accomplish its mission.

Additionally in April 2022, NRC issued NUREG-2251, Vol. 1, "Capacity Assessment for Statistics, Research, Evaluation, and Other Analysis, Fiscal Year 2022" (ML22066B054). One area it assessed was the Knowledge Management Program. It included the following finding and mitigation strategy for Knowledge Management in section 5.7.2.1:

Finding: Knowledge management tools are not utilized to their fullest extent to ensure successful capture and transfer of knowledge to staff. Survey results for each of the key agency functions show that approximately half of surveyed staff and management usually use knowledge management resources and processes (internal wiki site, videos, publications, etc.)

to capture best practices. Knowledge management will influence agency performance over the next 5 years, given that approximately 26 percent of the NRCs workforce is currently eligible to retire and approximately 44 percent will be eligible to retire within the next 5 years. High attrition over the next 5 years could negatively impact some positions identified in this assessment and will leave a critical knowledge gap.

Mitigating Strategy: The NRC should evaluate the NRCs knowledge management program to better align the efforts with expected outcomes. The evaluation should explore ways to elevate the priority and urgency of capturing critical knowledge and best practices. Attention should be focused on the positions with highest projected attrition as identified through the NRCs Strategic Workforce Planning (SWP) process (ML17109A315). The evaluation should consider methods to increase knowledge management engagement with the NRCs senior-level staff.

The evaluation should include a cost-effectiveness analysis to better understand the cost compared to the expected outcomes. To measure effectiveness, performance indicators should be established as a result of the evaluation. In addition, usage data for Nuclepedia should be thoroughly tracked and analyzed to find how to maximize the usefulness of this resource for the NRC.

As a result of this finding, and as specified in the mitigating strategy, the NRC is conducting an evaluation of the Knowledge Management program.

Objective The objective is to obtain expert technical and analytical assistance to design and conduct an evaluation of the NRCs knowledge management program to improve effectiveness.

Scope of Work Page 13

31310022A0002/31310022F0087 Provide the NRC with expert technical and analytical assistance to design and conduct an evaluation of the NRCs KM Program.

The evaluation will be conducted in four key phases.

Phase 1 focuses on gaining knowledge and understanding Phase 2 focuses on the development of a detailed and structured evaluation plan Phase 3 focuses on conducting the evidence building and evaluation activities; an Phase 4 focuses on additional technical assistance and change management support.

Phase 1: Gaining Knowledge and Understanding Phase 1 initiates with a task kick off meeting, and includes informational interviews, background research, and the development of a logic model. This phase will be led by the principal and senior evaluator and will be completed within 65 days from the initial kickoff meeting. The contractor will need to quickly get up to speed on background and contextual information, including agency and program specific requirements and processes, resources and tools available, and key stakeholders. The contractor will meet with NRC staff on a weekly basis to provide status updates and to coordinate on upcoming activities.

Kickoff Meeting The evaluation of the KM Program will begin with a kickoff meeting between the contractor and key NRC staff and the Contract Officer Representative (COR). The purpose of this meeting is gathering background information regarding specific needs, goals, desired outcomes; evaluation questions of interest; and gaining a better understanding of the evaluation timeline and constraints. The NRC has developed a series of potential evaluation questions, along with additional context, to provide a high-level understanding of the potential extent of the evaluation.

These questions have been developed based on initial discussions with staff and senior leadership and are merely a starting point for discussion.

1. To what extent is the KM Program producing worthwhile results and meeting program objectives and outcomes?

The KM Strategy defined the following expected outcomes:

a. Knowledge Management becomes routine and is integrated into daily work activities and work processes.

b. Knowledge is captured at all levels of the organization which results in improved productivity, allowing people to do their jobs more effectively.

c. Knowledge is consistently managed and readily accessible and used, improving the organizations ability to execute operations more efficiently.

NRC's internal assessment, completed as part of the Capacity Assessment, found that KM tools were not utilized to their fullest extent to ensure successful capture and transfer of knowledge to staff. Survey results for each of the key agency functions showed that only approximately half of surveyed staff and management routinely used KM resources and processes. The Capacity Assessment also stated that the KM Program needs a means to measure program effectiveness.

Page 14

31310022A0002/31310022F0087

2. Does the NRC have a common understanding across the agency of what types of knowledge we are trying to capture?

Across the NRC, offices and regions have different goals, tasks, priorities, and stakeholder interactions. Additionally, the various offices and regions have different staffing needs, different critical skills, and different challenges associated with KM. With the large scope of the KM Program, additional questions may need to be addressed to aid in answering the larger question. These questions may include:

1) To what extent is the NRC consistently managing the KM Program agencywide and integrating it into daily work activities and processes?

2) To what extent is a systematic and standardized process being used for capturing the knowledge associated with critical areas?

3) To what extent have KM approaches been integrated into operational guidance and has this produced effective knowledge transfer?

4) To what extent has the KM Program identified and mitigated knowledge gaps or challenges?

Additionally, the NRC is expected to hire a large number of new staff over the next few years.

What knowledge can the agency leverage from new employees? Are there specific areas the NRC can enhance with outside knowledge and experiences?

3. What type of knowledge is most valuable to transfer?

The NRC's internal assessment, completed as part of the Capacity Assessment, found that approximately 26 percent of the NRCs workforce is currently eligible to retire and approximately 44 percent will be eligible to retire within the next 5 years. High attrition over the next 5 years could negatively impact some positions more than others and could leave critical knowledge gaps, hindering NRC's ability to efficiently complete its mission. While the agency is aware that attention should be focused on the positions with highest projected attrition as identified through the SWP, some of these positions are already captured by the KM Program more effectively than others. Other types of knowledge, while critical for the agency, may not be effectively captured and distributed by the KM Program.

Additional questions may need to be addressed to aid in answering the larger question. These questions may include: To what extent has the KM Program identified and mitigated knowledge gaps or challenges? To what extent is the agency prioritizing knowledge capture and transfer consistent with the key objectives of the existing KM Strategy?

4. Which KM methods are effectively transferring knowledge?

The NRC KM Program has a large volume of tools that have been created and implemented over the years. These include, but are not limited to, Nuclepedia, Communities of Practice, the KM Toolkit for Supervisors, other OCHCO Toolkits, the Career Mentoring Program, Weekly KM sessions for inspectors, KNOWvember annual event, Design Your KM Approach, Job-specific training, and qualifications (focusing on On-The-Job training activities). This creates a large umbrella of information covered under the KM Program. Of these tools, which ones are being used as they are designed, and which tools are most effective at capturing and distributing useable information?

5. Are the expended KM resources commensurate with the benefits?

Page 15

31310022A0002/31310022F0087 The NRC places a high expectation on staff to capture, maintain, and transfer knowledge using a variety of methods and tools. However, while the expectation is high, time and resources have been a limiting factor when trying to meet this expectation. Some tools included in the KM Program are labor-intensive and cumbersome to input information into, while others are difficult to gather information from. A better understanding of the KM Programs cost-effectiveness or benefit may provide a better understanding of the limitations or constraints.

Informational Interviews The contractor will conduct informational interviews with key staff from NRC offices and regions involved with the KM Program to learn about ongoing activities, gather additional background information, and gain further insights into any challenges, needs, and opportunities associated with the KM Program.

Background Research The contractor will collect and systematically review relevant documents and background materials to learn more about the context surrounding the KM Program, relevant team/program/stakeholder practices and desired outcomes, and to assist in the development of a program logic model. Background research will also include learning about KM activities and outputs, as well as short-term and long-term outcomes. The contractor will also gather information about the types of stakeholders that may be involved in the data collection, and how the results of the evaluation will be used.

Logic Model Development The contractor shall develop a logic model for the KM Program to ensure that all program activities are examined in relation to desired outcomes. The logic model will be developed in collaboration with key NRC staff to ensure that all activities, outputs, and outcomes are included in evaluation planning and implementation. During development of each logic model, the contractor will review information gathered during the kickoff meeting, informational interviews, and background research to suggest refinements to activities and outcome statements as necessary and will revisit this process throughout the evaluation to maintain an updated logic model.

Phase 1 Meetings and Reporting Requirements The NRC will conduct a kickoff meeting with the contractor and meet weekly, or as needed, to the discuss the status of the evaluation and to exchange information. The contractor will take meeting notes and document any actions or decisions as an outcome of the meeting. The contractor will electronically submit meeting notes to the NRC COR and project lead within three business days of the meeting.

The contractor shall electronically submit a draft logic model for NRC review and comment to the NRC COR and project lead within 45 calendar days from the initial kickoff meeting. The NRC will provide comments to the contractor within 10 calendar days from the receipt of the draft logic model. The contractor will revise the draft logic model to address NRC comments and electronically submit a final logic model within 10 calendar days from the receipt of NRC Page 16

31310022A0002/31310022F0087 comments.

Phase 2: Designing Evidence-Building and Evaluation Activities The contractor shall use the information gathered in Phase 1 to develop a detailed and structured evaluation plan that should, at a minimum, clarify the scope and purpose of the evaluation and include a milestone and deliverable schedule, evaluation questions, approaches and methods, boundaries and limitations, and data collection needs. Phase 2 will be led by the principal and senior evaluator with support from the evaluator and a subject matter expert (Level

2) with relevant knowledge, experience, and education. The contractor will meet with NRC staff on a weekly basis to provide status updates and to coordinate on upcoming activities. The final evaluation plan shall be completed within 50 calendar days after the completion of Phase 1.

Evaluation Questions As part of the evaluation plan, the contractor will develop goals for the evaluation that will inform the development of evaluation questions. The contractor will incorporate the evaluation questions into the logic model developed in Phase 1 to directly inform the evaluation approaches and methods.

Evaluation Methodology As part of the evaluation plan, the contractor shall tailor a design specific to the evaluation questions developed for the KM Program evaluation. The evaluation plan should consider a design using both formative and summative components to understand program outcomes and provide recommendations for programmatic changes throughout the evaluation. The contractor should consider additional methods such as cost-benefit analysis, cost-effectiveness analysis, needs assessment, quality assessment, and trend analysis. Depending on the evaluation questions, the contractor should determine the statistical methods that are most appropriate for analyzing the data. Quantitative analyses should use applications such as R Studio or SPSS to perform statistical analyses. Qualitative data collected through interviews, focus groups, and open-ended survey items should be coded for themes and analyzed using applications appropriate for qualitative and mixed method evaluation.

Data Collection Methods As part of the evaluation plan, the contractor shall determine the appropriate data collection needs and methods that consider both quantitative data (e.g., Likert-type survey questions or numeric data) and qualitative data (e.g., descriptions, open-ended survey questions, focus groups, or interviews). The contractor shall identify and describe the tools to be used for the data collection but also the timing and conditions under which data will be collected. The contractor shall also include a description of how and who is collecting the data, as well as a timeline for all data collection, analysis, and reporting efforts. The contractor should use standardized or custom designed data collection tools, as appropriate, to measure relevant variables and assess progress toward project objectives. The development of any custom designed data collection tools by the contractor shall be explicitly authorized by the NRC prior to development and the data collection tools shall be developed in collaboration with the NRC.

Additionally, the contractor should minimize data collection efforts and should maximize the use of existing data whenever possible.

Page 17

31310022A0002/31310022F0087 Phase 2 Meetings and Reporting Requirements The NRC will meet with the contractor weekly, or as needed, to the discuss the status of the evaluation and to exchange information. The contractor will take meeting notes and document any actions or decisions as an outcome of the meeting. The contractor will electronically submit meeting notes to the NRC COR and project lead within three business days of the meeting.

The contractor shall electronically submit a draft evaluation plan, as discussed above, for NRC review and comment to the NRC COR and project lead within 30 calendar days from the completion of Phase 1. The NRC will electronically submit comments to the contractor within 10 calendar days from the receipt of the draft evaluation plan. The contractor will revise the draft evaluation plan to address NRC comments and electronically submit a final evaluation plan within 10 calendar days from the receipt of NRC comments.

Phase 3: Conducting Evidence-Building and Evaluation Activities The contractor will use the evaluation plan developed in Phase 2 to conduct the evaluation on the KM Program. The KM Program evaluation should be conducted in accordance with the Guiding Principles for Evaluators established by the American Evaluation Association. The contractor should conduct the evaluation using a developmental evaluation approach to ensure the processes informs and supports an innovative and adaptive process by asking evaluative questions, applying evaluation logic, and gathering and reporting evaluative data to support the program development with timely feedback. Phase 3 will be led by the principal and senior evaluator with support from the evaluator, analyst, and a subject matter expert (Level 2) with relevant knowledge, experience, and education. The final evaluation report shall be completed within 165 calendar days after the completion of Phase 2.

The contractor should use, if applicable, the four essential practices of cultural competence established by the American Evaluation Association: 1) Acknowledge the Complexity of Cultural Identity, 2) Recognize the Dynamics of Power, 3) Recognize and Eliminate Bias in Language, and 4) Employ Culturally Appropriate Methods. The contractor should be culturally responsive by establishing an understanding of the cultural contexts in which the KM Program operates to maximize program effectiveness and achieve desired outcomes. The culturally responsive approach should carefully consider the culture of the NRC and KM Program and examine impacts through a lens in which the culture of participants is a key factor.

Phase 3 Meetings and Reporting Requirements The NRC will meet with the contractor weekly, or as needed, to the discuss the status of the evaluation and to exchange information. The contractor will take meeting notes and document any actions or decisions as an outcome of the meeting. The contractor will electronically submit meeting notes to the NRC COR and project lead within three business days of the meeting.

The contractor shall electronically submit a draft evaluation report, as discussed above, for NRC review and comment to the NRC COR and project lead within 120 calendar days from the completion of Phase 2. The NRC will electronically submit comments to the contractor within 15 calendar days from the receipt of the draft evaluation report. The contractor will revise the draft evaluation report to address NRC comments and electronically submit a final evaluation Page 18

31310022A0002/31310022F0087 report within 30 calendar days from the receipt of NRC comments.

Phase 4: Technical Assistance and Change Management Support The contractor will develop a strategy to implement the recommendations included in the final evaluation report, as appropriate. The strategy should, at a minimum, include goals and objectives that align with the outcome of the evaluation, performance indicators that provide meaningful performance measurements of the KM Program, and mitigating strategies necessary to overcome any needs, opportunities, or challenges identified during the evaluation.

Phase 4 will be led by the principal and senior evaluator with support from the evaluator, and a subject matter expert (Level 2) with relevant knowledge, experience, and education. The final evaluation report shall be completed within 65 calendar days after the completion of Phase 3.

Phase 4 Meetings and Reporting Requirements The NRC will meet with the contractor weekly, or as needed, to the discuss the status of the evaluation and to exchange information. The contractor will take meeting notes and document any actions or decisions as an outcome of the meeting. The contractor will electronically submit meeting notes to the NRC COR and project lead within three business days of the meeting.

The contractor shall electronically submit a draft KM strategy, as discussed above, for NRC review and comment to the NRC COR and project lead within 45 calendar days from the completion of Phase 3. The NRC will electronically submit comments to the contractor within 10 calendar days from the receipt of the draft evaluation report. The contractor will revise the draft evaluation report to address NRC comments and electronically submit a final evaluation report within 10 calendar days from the receipt of NRC comments.

Reporting Requirements Monthly Letter Status Report (MLSR)

The contractor shall provide a MLSR which consists of a technical progress report and financial status report. This report will be used by the Government to assess the adequacy of the resources proposed by the contractor to accomplish the work contained in this SOW and provide status of contractor progress in achieving activities and producing deliverables. The report shall include order summary information, work completed during the specified period, milestone schedule information, problem resolution, travel plans, and staff hour summary. The final MLSR will also include any lessons learned and helpful insights identified during the evaluation.

Final Reports The contractor shall electronically submit all draft and final reports as discussed in each phase above.

Deliverables and Delivery Schedule Section # Deliverable Due Date Format Submit to Page 19

31310022A0002/31310022F0087 Gaining 45 calendar days

1) Draft Logic COR and Knowledge and from the initial Word Document Model Project Lead Understanding kickoff meeting Gaining 65 calendar days

2) Final Logic COR and Knowledge and from the initial Word Document Model Project Lead Understanding kickoff meeting Phase 2:

Designing 30 calendar days Evidence- 3) Draft Evaluation from the COR and Word Document Building and Plan completion of Project Lead Evaluation Phase 1 Activities Phase 2:

Designing 50 calendar days Evidence- 4) Final Evaluation from the COR and Word Document Building and Plan completion of Project Lead Evaluation Phase 1 Activities Phase 3:

Conducting 90 calendar days Evidence- 5) Draft Evaluation from the COR and Word Document Building and Report completion of Project Lead Evaluation Phase 2 Activities Phase 3:

Conducting 165 calendar days Evidence- 6) Final Evaluation from the COR and Word Document Building and Report completion of Project Lead Evaluation Phase 2 Activities Technical 45 calendar days Assistance and from the COR and Change 7) Draft Strategy Word Document completion of Project Lead Management Phase 3 Support Technical 65 calendar days Assistance and from the COR and Change 8) Final Strategy Word Document completion of Project Lead Management Phase 3 Support Applicable Documents and Standards NUREG-2251, Vol. 8, "Strategic Plan, Fiscal Years 2022-2026" (ML22067A170)

NUREG-1614, Vol. 1, "Capacity Assessment for Statistics, Research, Evaluation, and Other Analysis, Fiscal Year 2022". (ML22066B054)

Page 20

31310022A0002/31310022F0087 Evidence-Building and Evaluation Policy Statement (86 FR 29683 29683-29685)

Place of Performance Work will be performed from the contractors place of business and will be conducted virtually when interfacing with the NRC.

Page 21

31310022A0002/31310022F0087 D - Contract Documents, Exhibits and Attachments Attachment Title Date Number Attachment No. 1 ML20023B283 Knowledge 1 09/15/2022 Management Strategy 2018-2022 Page 22