Request for Information to Gain Access to Application Program Interfaces from Commercial Search Engine 1.0 SUBJECT Request for Information (RFI) to provide no-cost access to commercial search engine application program interfaces (APIs). 2.0 SUMMARY The General Services Administration's Office of Citizen Services & Innovative Technologies (OCSIT) invites interested parties to submit a written response to the attached RFI. The RFI's intent is to obtain the rights for USA Search Services and its associated Affiliates websites to access commercial web search engine application program interfaces (API) to help the public find information from federal, state, local, territorial and tribal government web sites. 3.0 BACKGROUND INFORMATION Launched in September 2000, USA.gov (formally FirstGov.gov) allows the public to find official government information through the USA.gov portal, connecting to the National Contact Center where emails or frequently asked questions can satisfy their need, or through using the search functionality provided through Search.USA.gov. The OCSIT, IT Solutions and Services Division brought the USA Search Program in house to better support the general public. The USA Search Program is on the brink of significant expansion by offering its services to its varied constituencies (federal, state, local, tribal, territorial government web sites). This Request for Information seeks access to commercial web APIs, including but not limited to a government-wide web search index. The OCSIT's USA Search Program will use the web index to provide the public access to search millions of government documents using up to 20 million queries per year. The OCSIT also operates an Affiliates Program that allows other government agencies to leverage the USA Search service. Presently, there are approximately 300 affiliate web sites that submit 40 million queries per year. The OCSIT expects to double affiliate traffic to 80 million queries per year by the end of the next fiscal year. The affiliate index will be a subset of the USA Search that currently encompasses approximately 10 million pages. The leading commercial search engine providers are offering more and more APIs, at no cost, to developers who wish to integrate web services within their own applications. In many circumstances, government agencies cannot use free commercial services because of the incompatibility of Terms of Service with government regulations. While the purpose of this RFI is to obtain access to a commercial web search engine index, the OCSIT would like to obtain the rights to access additional web services that the respondents offers for free to web developers. Potential services include, but not limited to, images, news, related searches, spell checking, and video. Additionally, the USA Search seeks the rights to access future APIs that commercial search engines offer for free to other communities. The purpose of this RFI is to ask for expressions of interest from the entire community of interested parties. This RFI seeks information from any provider that wishes to offer a no-cost solution for access to a government-wide web index to be available for use no later than September 15, 2010. Parties are free to include access to other APIs should they deem it appropriate. For purposes of this RFI, GSA seeks a no-cost contract. The no-cost solution must not be tied to required, ancillary fee-based products or services. That is, the OCSIT must be able to use the product at a maximum level of technical functionality without being required to purchase ancillary services and products from the offerer, although the offerer may offer support for configuration, setup and other initial tasks, also at no cost. The no-cost solution must not restrict GSA or other agencies to one provider, and any government agency should be able to discontinue at any time without legal or financial commitments. The OCSIT cannot accept a no-cost solution if the provider's intent is to provide the solution for no-cost now, and then begin charging OCSIT at a later date. 4.0 OBJECTIVES Ideally, OCSIT is looking for no-cost access to APIs solutions that will provide the following capabilities: •· A search service that is capable of crawling and indexing the following document types, at a minimum: HTML, PDF, Microsoft Word, Microsoft Excel, and Microsoft PowerPoint. •· A search service that updates indexes on regular cycles (e.g. daily, weekly, etc., dependent on popularity and frequency of content changes on specific URLs) to ensure the most current document versions are being indexed. The search service must crawl and index commonly visited and other popular sites more frequently than other sites. Additionally, sites on which content changes often must be crawled more frequently than others . •· The API will have the ability to retrieve results in a common format such as, but not limited to JSON, XML, and SOAP. •· The API shall provide access to the following features: •o Automatic spell check suggestions •o Normalization of indexed words so that users are able to enter search words or phrases with or without punctuation, spaces, and/or diacritics and retrieve same items •o Suggested searches based on user's search query •o Ability to view cached versions of indexed pages from the search results pages •o Search term highlighting in result listing summaries •o Advanced search options providing user the ability to narrow searches by language, file format, date, occurrences, domain, and to perform guided Boolean searching. •o UTF-8 index encoding to support all language characters •· The service shall be capable of handling 20 queries per second (qps) from Search.USA.gov and its affiliated domains . •· The commercial web index provider will not have the ability to place any content such as advertising on the search engine results pages (SERPs), but the vendor may place a logo next to the web search results for branding purposes. •· The search service requires 99.5% uptime with responses returned in under 1.5 seconds. •· The search service provider, at their sole discretion, will allow the OCSIT to use other services available through an API with the same uptime requirements as the web index. •· The search service shall allow for the modification of a seedlist (a list of government sites that the OCS wishes to include in the USA Search Service). Presently, there are over 5000 unique URLs that comprise the current seedlist of the USA Search Service, including all .gov, and .mil domains, plus a list of other government sites falling under different top-level domains. •· The search service shall have the ability to recognize Spanish queries and serve Spanish language results and Spanish language spelling corrections. •· The OCSIT would like to have the ability to offer its affiliate customer's access to a Search.USA.gov API that may use content from the offerer's search service. 5.0 SAMPLE RESPONSE OUTLINE Following is a suggested outline and page counts for a response to this RFI. This outline is intended to minimize the respondent's effort and structure responses to facilitate analysis by the government. Nevertheless, respondents are free to develop their responses as they see fit. 1. Company Information and Expertise (1-4 pages) Briefly describe your company, headquarters location, products and services, history, ownership, leadership, financial information, and any other facts and figures you deem relevant. Provide company name and address, and two points of contact, including name, title, phone, fax, and e-mail address. Describe any projects that you've been involved with that are similar in scope and concept to what is described in this RFI, and any relevant lessons learned. 2. The Offering (5-12 pages) The government is seeking responses from qualified providers that address the following. You are invited to answer these bullets. You may answer each question individually or provide a consolidated answer to related questions. Also, you may provide references to any information accessible to the government that address these directed topics (e.g. web pages or additional materials). The government appreciates focused responses for each topic. •· What do you offer at no cost that would meet the government's needs for a government-wide web index to allow the public to search across the millions of publicly available government documents? Please be specific as to what is offered at no cost and clearly differentiate it from any other services that do not meet the criteria laid out in the final paragraph of Section 3.0. •· What other APIs may the OCSIT access on a no-cost basis? •· How does this solution meet the objectives listed in Section 4.0? •· Are you offering a no cost solution that will not require the government to pay for this solution in the future? For example, are you offering a perpetual no-cost license? •· How long has the API existed? •· What services are currently available through your APIs? That is, what functionality is available for Search.USA.gov to present to the public? •· What are the near term improvements, if any, of your search service? •· What administrative capabilities, if any, are inherent in the service? •· What user testing have you done on your product to increase user performance and satisfaction? •· What is the maximum number of queries per second you support? •· Describe your application suite's update or patch release cycle? •· Describe security measures to protect your infrastructure against unauthorized access to or unauthorized alteration, disclosure or destruction of data. For example, is your hosting infrastructure under a SAS 70 examination? •· Describe any service outages in the past year. •· What analytics or reporting, if any, are provided as part of the offering? •· Are regular, planned updates/upgrades to the product part of the offering? Section 3 - Additional Materials •· Provide any other materials, suggestions, and discussion you deem appropriate. No Pricing Information: Since GSA is seeking information for a no-cost solution, no pricing should be provided. You should only describe exactly what it is you are offering the government at no cost. The total written response should not exceed 15 pages. 8.0 CONTACT INFORMATION and INFORMATION EXCHANGE All information received in response to this RFI marked Proprietary will be handled accordingly. Responses to the RFI will not be returned. Responders are solely responsible for all expenses associated with responding to this RFI. Following is the individual point of contact for this RFI. Submit all questions regarding this RFI via e-mail to
[email protected] Herbert Connell Acquisition Analyst, Lead GSA 1800 F St NW, Room 4020 Washington, DC 20405
[email protected] Please submit responses to this RFI via e-mail in Microsoft Office format by 11 :00 AM on 08/12/2010 to the above point of contact. You may also submit supplemental hardcopy materials such as brochures, etc. to the point of contact. If GSA is interested in the solution you offer, we will contact you. You may be requested to provide additional information or details based on your initial submittals. GSA will consider meeting individually about this RFI with respondents based on the information submitted in response to this RFI. If GSA engages in such meetings, it will provide that opportunity equally to all respondents. GSA reserves the right to move forward with one solution, more than one solution, or no solution at all, in response to this RFI.