Description
NOTICE ID NO. NIST-SSN-26-1002347 Title: Sources Sought Notice for Dataset Generation, Annotation, and Translation for AI systems Testing, Evaluation, Validation, and Verification Services The purpose of this sources sought notice is to conduct market research and identify potential sources of commercial products/services that satisfy the Government’s anticipated needs. BACKGROUND Artificial Intelligence (Al) is one of the defining technologies of our era. Its emergence, together with its multiplying contexts of use and increasing capabilities, presents enormous opportunities as well as significant present and future harms. The U.S. Center for AI Standards and Innovation (CAISI), housed within the National Institute of Standards and Technology, was created to respond to these new national security and public interest priorities. CAISI’s mission is to identify, measure, and mitigate the risks of advanced AI systems while supporting competition, innovation, and AI adoption. As part of this, CAISI is conducting testing, evaluation, validation, and verification (TEVV) on AI systems for their reliability and robustness in common use cases. Task and domain areas include security (e.g., cyber, chemical, biological), criminal misuse of AI systems (e.g., fraud, scams), software engineering, AI research & development, jailbreaks, agentic tasks (such as knowledge work or computer use), AI system propensity, AI-driven productivity, AI-human interaction, simulated AI deployment scenarios (e.g., a simulated scenario in which the AI has the opportunity to deceive the user), hallucinations, manipulation, deception, geopolitical preferences, sycophancy, overreliance, manipulation, privacy, robustness, reliability, and functionality. CAPABILITIES SOUGHT NIST is seeking information from sources that may be capable of providing all of the following services: A. Dataset Generation The Contractor shall design, generate, and curate datasets that can be used to evaluate AI systems. Specifications for the data to be produced (for example, size of the dataset, domain area, tasks to be represented, whether the content to be generated is text, image, audio, video, or a combination, and quality specifications) will be detailed by task order. B. Dataset Annotation The Contractor shall annotate characteristics of the data in a dataset provided by the Government. Characteristics may include the suitability, relevance, or correctness of the data, document summarization, document editing, document synthesis, image or video annotation, analysis of agent transcripts or solutions, chatbot functionalities, information retrieval, and biological or chemical structure, property, activity, or function. Specifications for data modality (i.e., text, imagery, audio, video, or a combination) will be detailed by task order C. Dataset Translation The Contractor shall produce expert translations or localizations of texts. The Contractor must have the capacity and be able to translate and/or localize text between any two languages from the following list: Hindi Indonesian (Bahasa) Farsi Swahili Turkish Urdu Arabic (at minimum Levantine, Maghrebi, and Egyptian dialects) Mandarin Cantonese Korean Russian Japanese Spanish (Mexican, Spanish/Spain) English (at minimum American, British, and Nigerian pidgin) Bengali German French (France, African dialects) Portuguese (at minimum Brazilian and Angolan) Yoruba Any of the above listed languages can be a source language for translation or localization (to be specified in the task order), and any of the above listed languages can be a target language for translation or localization (specified in the task order). For this task area, the Contractor will be required to report audit metrics to the Government, including translator agreement, audit precision, recall, and accuracy. HOW TO RESPOND TO THIS NOTICE In responding to this notice, please DO NOT PROVIDE PROPRIETARY INFORMATION. Submit the response by email to the Primary Point of Contact and, if specified, to the Secondary Point of Contact listed in this notice as soon as possible, and preferably before the closing date and time of this notice. Please note that to be considered for award under any official solicitation, the entity must be registered and “active” in SAM at the time of solicitation response. Please include only the following information, readable in either Microsoft Word 365, Microsoft Excel 365, or .pdf format, in the response: Provide the complete name of your company, address, name of contact for follow-up questions, their email, their phone number and, if your company has an active registration in https://sam.gov, your company’s Unique Entity ID (UEI). Describe services that are available related to the subject task areas. If the recommended services are available for purchase on any existing Federal Supply Schedule contract(s) or other contracts against which NIST may be able to place orders, identify the contract number(s) and other relevant information. Identify any customers in the public or private sectors to which you provided the recommended or similar services. Include customer(s) information: company name, phone number, point of contact, email address. For the NAICS code listed in this notice: Indicate whether your company is (a) a small business or (b) other than small business. See the Table of Small Business Size Standards and the associated .pdf download file for small business size standards and additional information. If you believe the NAICS code listed in this notice is not the best NAICS code for the type of product addressed in this notice, identify an alternative NAICS code that you believe would be more appropriate for the planned procurement. Identify any aspects of the NIST market research notice, including instructions, and draft minimum specifications that you cannot meet and state why. Please offer suggestions for how the market research notice and draft minimum specifications could be made more competitive. Provide any other information that you believe would be valuable for NIST to know as part of its market research for this requirement. State if you require NIST to provide additional information to improve your understanding of the government’s requirement and/or would like to meet with NIST representatives to discuss the requirement and the capabilities of the identified equipment. QUESTIONS REGARDING THIS NOTICE Questions regarding this notice may be submitted via email to the Primary Point of Contact and the Secondary Point of Contact listed in this notice. Questions should be submitted so that they are received by October 3, 2025 . If the Contracting Officer determines that providing a written amendment to this notice to document question(s) received would benefit other potential respondents, the questions would be anonymized, and a written response to such question(s) would be provided via an amendment to this notice. IMPORTANT NOTES This notice is for market research purposes and should not be construed as a commitment by NIST to issue a solicitation or ultimately award a contract. There is no solicitation available at this time. This notice does not obligate the Government to award a contract or otherwise pay for the information provided in response. NIST reserves the right to use information provided by respondents for any purpose deemed necessary and legally appropriate. Any organization responding to this notice should ensure that its response is complete and sufficiently detailed to allow the Government to determine the organization’s capability. Respondents are advised that the Government is under no obligation to acknowledge receipt of the information received or provide feedback to respondents with respect to any information submitted. After a review of the responses received, a synopsis and solicitation may be published on GSA’s eBuy or SAM.gov. However, responses to this notice will not be considered an adequate response to any such solicitation(s). The responses shall not exceed six (6) pages including all attachments, charts, etc. Thank you for taking the time to submit a response to this request!