The US Government is interested in safe uses of large language models (LLMs) for a wide variety of applications including the rapid summarization and contextualization of information relevant to the Intelligence Community. These applications must avoid unwarranted biases and toxic outputs, preserve attribution to original sources, and be free of erroneous outputs. The US Government is also interested in identifying and mitigating hazardous use of LLMs by potential adversaries. The goal of BENGAL is to understand LLM threat modes, quantify them and to find novel methods to address threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more topic domains, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to efficiently probe LLM models to detect, characterize and mitigate biases, threats or vulnerabilities. Topic areas and additional requirements for successful proposals will be introduced at the Proposers’ Day event. The BENGAL Proposer's Day will be held Tuesday, October 24, 2023, from 9:30am to 4:30pm EDT in Washington, D.C. A virtual option will be available for individuals that are unable to attend in person. Participants must register by Wednesday, October 18, 2023, at noon EDT. Same day registrations will not be accepted. Limited spots are available for in-person attendance and early registration is encouraged. Only one in-person attendee per organization or research group will be permitted to attend in person, with the exception of professors at the same university or college at IARPA’s discretion. The link to register is: https://www.client-meeting.net/bengal-proposers-day Questions about this announcement can be emailed to:
[email protected] *UPDATE*: The BENGAL DRAFT BAA has been uploaded for those who desire to bring a copy to the Proposer's Day event. It is not required to bring this DRAFT for attendance, as it will be discussed during the event. The Government will be taking questions during the event, however, we will only be able to answer a limited number of questions at the event, due to the anticipated number of attendees. All questions and answers will be used to complete the Final BENGAL BAA. Questions submitted after the Proposer's Day will also be accepted, however, the Government will not be responding to those groups or individuals directly who submit questions after the Proposer's Day event.