Infusing Data Science and Analytics into the U.S. Government

The U.S. government established the “data science” series as a job classification in December 2021. Prior to this in 2019, the federal government passed the Evidence Act, which requires federal agencies to develop evidence to support policymaking. Related to this Act, the Federal Chief Data Officers (CDO) Council and Federal CDO roles were created for U.S. government agencies and departments to spearhead data governance and data management as well as encourage the use of data analytics and evidence-based reasoning for informing decision-making.

To further enhance the use of data science/analytics at the federal executive levels as well as help fill the digital talent pipeline in the federal government, a new role was established in September 2022 at Columbia University’s Data Science Institute, through a kind gift, to create an “Executive-in-Residence for Public Service” position to meet the above goals in collaboration with the nonpartisan, not-for-profit Washington, D.C.-based Partnership for Public Service (ourpublicservice.org).

To determine what already exists at Columbia University and the Partnership for Public Service (as well as some other related organizations) in terms of applying data science/analytics in the federal government, about 65 interviews were conducted by the author between September and November 2022 (30 interviews at Columbia University and 35 interviews at the Partnership for Public Service and some related organizations). Figure 1 presents what already exists as well as what will exist in the near term at Columbia University and the Partnership for Public Service (and related organizations) and is categorized in four areas: Faculty/Student Engagement, Talent Pipeline, Institutional Building and Networking.

In addition to these interviews, a “Data Science in Government” survey was administered in fall 2022 to directly reach out to government agencies to understand their needs to better understand and apply data science/analytics to inform their decision-making. We were able to get feedback from 24 government organizations, although the number of respondents was small. However, this feedback allowed us to get a glimpse of some of the challenges and opportunities facing the government in these areas. We also examined myriad recent surveys and publications that deal with this topic. The following is a summary of our survey results.

Data Science in Government Survey Results (24 Government Agencies; Mostly Federal)

(Note: Sample size is 24 respondents and therefore limited; 42% Senior Executive Service (SES); 54% General Schedule (GS) 12-15; 4% others.)

• Midlevel managers need better skills and confidence to use data to inform decisions.
• Not every department has someone with data analytics skills.
• Sharing data across departments is an issue.
• The respondents rely more on their data than their intuition; however, about one-third of the respondents feel that their internal data quality is suspect.
• About 88% are aware that there is a new “data scientist” position in the government.
• Respondents would be interested in enhancing their data science skills by executive short courses/workshops (67%); certificate programs (54%); lecture/webinar series (54%); and “Data Science in the Federal Government Day” (50%).
• Official government statistics (88%) and open data (79%) are the most used types of data in their organizations to inform policymaking.
• Inferential statistical analysis (e.g., linear regression) (75%), data mining (67%), and spatial analysis and geographic information systems (GIS) (67%) are the main data analysis skills used in their organizations. Machine learning (58%) and social/organizational network analysis (29%) were used less frequently.
• Policing and public safety (44%) and transportation (44%) seem to be using data science techniques more than others.
• Microsoft Excel (96%), Tableau/Dashboards (79%), SQL databases (75%), R (71%), Python (67%), machine learning (63%) and SPSS/SAS/Stata/Stat packages (63%) are most used in their organizations. Artificial intelligence (AI) (37%) is one of the least used techniques.
• Training programs for staff (79%), data sharing agreements with other government partners (67%), working with the private sector (50%), and working with the university and think tank sector (50%) are the main practices used to support data science in their organization.
• Other governments (federal, state or local) (79%), professional conferences (75%), private sector (75%) and publications (63%) are the main avenues for new ideas regarding data science for government.
• The main barriers to using data science in the government context are difficulty of hiring and training staff with data science capacity (71%) and difficulty due to data silos (and data sharing) and lack of interoperable databases (71%).
• For business and soft skills, the skills that the respondents feel most comfortable with are as follows: can learn quickly (92%), knowledge transference/presentation/communication skills (92%), can see the big picture (88%), critical thinking (88%), and can develop actionable insight based on data (88%). The respondents generally felt comfortable in all of the business and soft skills.
• For technical skills, the skills that the respondents feel most comfortable with are data reporting (88%), data storytelling/data visualization (88%) and proactive problem solving (75%). The least comfortable skills are software development, database design and management, and technical or computer science skills.
• For analytics skills, the skills that the respondents feel most comfortable with are analytics tools (79%), visualization and dashboard design (63%), and data representation and transformation (50%). The least comfortable skills are deep learning, big data technologies and prescriptive analytics.

Based on the interviews and survey results, as part of the data collection effort, a number of initiatives are being considered for 2023 and beyond to further enhance the data literacy of federal executives and make more students/faculty (at least at Columbia, and perhaps beyond) further aware of data science/analytics opportunities in the U.S. government. These include the following initiatives:

• Federal Executive Boot Camp on “Decision Intelligence.”
• Data Science in U.S. Government Day (will be held virtually on April 27, 2023).
• Information series/clearinghouse on fellowships/internships/full-time employment/sabbatical opportunities in data science in the U.S. government.
• Undergraduate/Master’s in Data Science capstones with federal agencies.
• GoGovernment.org videos on data scientists in government.
• Government “Data Science in Practice” podcasts.
• Federal data science online community of practice/community of interest.
• Publishing books to highlight these topics, such as J. Liebowitz’s edited volume “Pivoting Government Through Digital Transformation” (Taylor & Francis, to be published in fall 2023). In addition, a new book series on “Digital Transformation: Accelerating Organizational Intelligence” (J. Liebowitz, Series Editor, World Scientific Publishing; https://www.worldscientific.com/series/dtaoi) has been established, and one of the volumes will be on digital transformation in the government.

There are many opportunities for students and faculty to be involved in analytics with the federal government, including those previously noted. In addition, enhancing the data analytics acumen of federal executives will better inform their decision making, and we hope others will follow suit.