Background The National Assessment Governing Board (hereafter referred to as Governing Board) is preparing to develop achievement levels for the 2017 National Assessment of Educational Progress (NAEP) writing assessment at grade 4. Public Law 107-209 mandates that the Governing Board develop appropriate student achievement levels for each grade or age in subjects tested under NAEP. The NAEP assessments cover a range of subject areas and grade levels. The methodology of the assessments is to ensure that assessment items are valid and reliable, developed in consultation with appropriate technical experts in measurement and assessment, content and subject matter, sampling, and other technical experts who engage in large scale assessment programs. NAEP Achievement Levels The Governing Board's legislative mandate is of significant importance, particularly since achievement levels are the primary means for reporting NAEP results. The achievement levels represent an informed judgment of "how good is good enough" and help explain results of the Nation's Report Card beyond simple scores and averages. They also provide the basis for gauging trends in each subject and grade assessed by NAEP. The NAEP achievement levels identify the knowledge and skills that can be measured and verified objectively using widely accepted professional assessment standards. They are also required to be consistent with relevant widely accepted professional assessment standards and be based on the appropriate level of subject matter knowledge for grade levels to be assessed. The Governing Board policy on setting achievement levels for NAEP can be found at: https://www.nagb.org/content/nagb/assets/documents/policies/developing-student-performance.pdf. The three achievement levels established by the Governing Board - Basic, Proficient, and Advanced - are the primary means of communicating about the performance of students on the Nation's Report Cards. Achievement levels have been set for reading, mathematics, writing, science, U.S. history, geography, civics, economics, and technology and engineering literacy. Reports on achievement levels and additional background on the Board's history setting achievement levels can be found on the Governing Board's website at: https://www.nagb.org/publications/achievement.html. While the Board's achievement levels setting methodologies have improved over the years, there is a unique need to develop state-of-the-art methods for this next generation of NAEP assessments. All new NAEP assessments will now be computer-based. Setting achievement levels on these computer-based, innovating NAEP assessments will require the Board to study, develop, and implement innovative standard setting methodologies to address the challenging issues presented by the innovative item types expected on new digital-based assessments. The NAEP Writing Assessment The 2017 NAEP writing assessment is a completely digital-based assessment that will be administered on tablets with detachable keyboards. Student performance will be assessed through exercises that are entirely constructed response and are holistically scored on a multi-point scale. The Governing Board intends to incorporate computerized methods in the achievement levels-setting process. The operational assessment will be administered in 2017, and data are expected to be available for the achievement levels-setting work by September 2017. The 2011 NAEP Writing Framework and Test Specifications adopted by the Governing Board are available on the Governing Board's website at: https://www.nagb.org/publications/frameworks.html#writing. The Writing Framework will be updated for the 2017 administration by the time the RFP is released, with very minor changes from the 2011 version (i.e., new cover, Board membership, and finalized achievement levels descriptions). Preliminary achievement levels descriptions were developed as part of the Writing Framework development project. In 2012, the Governing Board adopted revised achievement levels descriptions for grades 4, 8, and 12 to use in the achievement levels setting process. The contractor will be provided with the finalized achievement levels descriptions for use in the achievement levels setting process for grade 4 conducted under this contract. Achievement Levels Setting for Writing at Grade 4 The Governing Board has a requirement to procure services for the design and implementation of an achievement levels-setting process for the writing computer-based assessment at grade 4. The Board anticipates the need for validation studies as well. This effort will require considerable research and evaluation (including a field trial and pilot study) prior to convening the operational achievement levels-setting panel in early 2018. The computer-based writing assessment is planned to be administered in 2017 to a nationally representative sample of 4th graders. The Governing Board adopted a computer-based Writing Framework in 2007. The NAEP writing computer-based assessment was administered in 2011 at grades 8 and 12, but the first operational administration at grade 4 is planned for 2017. Achievement levels were set for the grade 8 and 12 assessments, but they still need to be set for the grade 4 assessment. Achievement levels have been set for reading, mathematics, writing, science, U.S. history, geography, civics, economics and most recently in technology and engineering literacy. Achievement levels based on the current Writing Framework were set in at grades 8 and 12 with the administration of the 2011 assessments. Plans for setting achievement levels in writing at grade 4 will be based on the administration of the 2017 assessment and will include a field trial, pilot study, and operational achievement levels setting study. In addition, the design procedures will require multiple sources of external validity evidence. The design and methodology for setting achievement levels in writing at grade 4 are not required to be identical but should be informed by the design and methodology used for setting achievement levels in writing at grades 8 and 12. The standard setting software that was used to conduct the achievement levels setting activities in NAEP writing for grades 8 and 12 has been made available to the public pursuant to the government's Open Data Policy and can be downloaded from the Governing Board website at: https://www.nagb.org/publications/achievement/software/bowtie.html. Request for Proposals The Request for Proposals (RFP) will be issued under full and open competition by approximately March 31, 2016. Proposals will be due approximately 60 days after the RFP is released. The award is anticipated to be a hybrid type contract with labor costs awarded on a Firm Fixed Price basis and Other Direct Costs awarded on a cost reimbursement basis. The contract award is projected to be made prior to August 30, 2015. The contract period of performance is anticipated to be 24 months. This is a new procurement (there is no incumbent). All responsible sources are encouraged to respond to this notice, register as interested vendors for this notice, provide capability statements (optional, not to exceed 2 pages), and submit feedback on the proposed solicitation. All timely queries and responses shall be considered by the Governing Board. Information on the Governing Board's work can be found at www. nagb.gov. Questions and comments on this notice are to be submitted only via email to the Contracting Officer
[email protected] . Responses to questions, if received, shall be posted posted no later than February 29, 2016.