Using machine learning to improve public reporting on U.S. government contracts

Note: The Innovative Applications in Analytics Award (IAAA) is a prestigious award developed by the Analytics Society of INFORMS to recognize creative and unique application of a combination of analytical techniques in new areas. Presented each year by the Analytics Society along with Kinaxis and Adelphi University, the award attracts submissions from around the world whose work is judged by a panel of experts. Below is the first in a series of brief articles describing the work of 2021 IAAA finalists.

The U.S. government procures more than $500 billion annually in goods and services on public contracts, which it classifies using the hierarchical Product Service Code (PSC) taxonomy. The classification of governmental purchases is important because it improves transparency in the government’s use of taxpayer funds and facilitates effective reporting, tracing and segmentation of government expenditures, leading to better budgeting and management. However, the contract classification process is also tedious, time consuming and error prone. Inadvertently selecting an incorrect PSC may lead to rejections in document workflows or limit discoverability of requests for proposals, resulting in delays and rework.

The PSC taxonomy itself consists of more than 2,000 codes, with classes ranging from unmanned aircraft to laundry services, and selecting the proper code is not always straightforward. During the course of our research, we discovered that procurement officials may spend tens of minutes attempting to identify the correct PSC for any given contract, often resorting to searches through a 300-plus page government manual [1].

We sought to improve this classification process by implementing a machine classifier to assist procurement officials in identifying and selecting PSCs. We trained a multiclass, single-label hierarchical classifier using a character-level convolutional neural network and 3.99 million archival contract records from the Federal Procurement Data System [2]. The classifier is deployed via a web application programming interface (API) and an interactive web front end, the latter enabling procurement officials to use natural language to describe contracts, then receive timely and accurate PSC recommendations. The application has responded to more than 110,000 queries from U.S. Department of Defense users, and many more from industry partners and other agencies across the federal government.

Consider the example of a procurement official who is developing a contract for a new vaccine. If the official did not already possess knowledge of the corresponding PSC, then the official may search through the PSC manual, which contains definitional information on each PSC. Searching using the keyword “vaccine” would be an obvious choice; however, that term does not appear. A more general search for “medical” instead returns 302 matches across 42 different pages; reviewing these matches to identify the most relevant PSC code would be time-consuming. Alternatively, Figures 1a and 1b show our web application interface being used to classify the description “vaccine.” In response to this input, the application uses an artificial neural network to assign probability scores to each possible PSC. Then, all PSCs with model-implied probabilities exceeding 1% – in this case eight classes – are displayed in descending order by match quality. A contracting official can quickly review these results and select the most relevant code for the contract. Estimates derived from current usage patterns demonstrate a significant time savings from the introduction of our analytics application.

Figures 1a and 1b: Terms can be entered in our prediction engine (1a) for product service codes, and classification predictions will be returned in order of predicted match quality, along with relevant information (1b).

A full version of this work will be published in INFORMS Journal on Applied Analytics [3].

References

1. U.S. General Services Administration, 2020, Federal Procurement Data System: Product and Service Codes Manual, March 17, https://www.acquisition.gov/psc-manual.
2. Muir, W.A., Reich, D., 2021, U.S. government contract metadata, fiscal years 2014 to 2020, http://dx.doi.org/10.5281/zenodo.4940111.
3. Muir, W.A., Reich, D., 2021, “Using machine learning to improve public reporting on U.S. government contracts, INFORMS Journal on Applied Analytics, forthcoming.