MainStreet Macro: The private-data revolution

The world is getting faster.

Yes, a normal day is still about 24 hours. But scientists say that Earth has been spinning faster, shaving milliseconds off a typical day. There’s evidence, in fact, that this year the planet is moving faster than it did in 2021 or 2020.

If that’s not enough speed for you, there’s another acceleration under way, one that’s affecting the frequency of data.

The economy has become more complex since the pandemic, and more dynamic, evolving and changing at a faster pace than before. To better understand that rapid change, we need to measure the economy more frequently.

Last week, I joined a forum held by the National Association for Business Economics. The topic of discussion: ADP’s National Employment Report and the data revolution under way in real-time employment analytics and metrics.

Here’s what we came away with.

The data playbook is being rewritten

U.S. statistical infrastructure is one of the best in the world. The government uses data from a survey of households and business to measure the economy across a wide range of activities. Most of what we know about the economy, from inflation to crime rates, is based on those survey responses and techniques employed by government statistical agencies, including the BLS (jobs and inflation), Bureau of Economic Analysis (GDP), and Census Bureau (virtually everything else).

Yet as reliant as we all are on survey data to capture the dynamics of the economy, the government of late has been leaning into private-sector data to supplement its survey results. 

One reason is that the American public is suffering from survey fatigue. Participation in government surveys has fallen enough that it could compromise effective economic measurement. 

The private sector can help fill gaps in official metrics, but they’re no substitute. Most private data sets are too new or too limited in their focus to be nationally representative and reliable.

Pairing private-sector and public-sector data can mitigate the shortcomings of both sources and enhance our understanding of this dynamic economy.

There’s a need for speed

Since at least the 1990s, and maybe even earlier, the Federal Reserve has incorporated high-frequency data from private companies into its forecast.

In 2001, then-Fed Chair Alan Greenspan gave a speech to the National Association for Business Economics on the limits of sophisticated theory and modeling to explain and capture changes in the economy.

He said that while we should endeavor to improve economic models and theories, “I suspect greater payoffs will come from more data than from more technique.”

Toward that end, the Fed, the BLS, and other government agencies leverage private, real-time data to supplement official statistics. They were doing this before the pandemic, but their use of private data has since skyrocketed as the government tries to keep up with the rapid pace of change.

Machines have joined the fight

Part of the reason our economy is moving faster is the rapid adoption of artificial intelligence and machine-learning technology.

Chatbots follow us as we shop for shoes. Artificial brains in our phones direct us to the best tacos or the closest bowling alley. Machines try to predict what movie we want to stream on Friday night.

These digital encounters generate data — a lot of it!

Researchers harness this data and translate it into meaningful metrics based on our online interactions. Developments in data science have strengthened the value of this rich, real-time stream of information, allowing us to see trends that we couldn’t see before.

ADP forges a new path

ADPRI recently revamped its National Employment Report with the goal of capturing rapid, reliable, and  scientific movements in the labor market. We believe five key attributes are necessary for any robust private–sector economic indicator, including ours.

• Independence: Business economists by and large have earned their keep by building models to forecast official data. The NER toolkit aims to shift that focus from forecasting to delivering a new, independent perspective.
• A nationally representative sample: The NER leverages the data of 25 million workers to capture changes in hiring and pay by industry, employer size, and region. This data can be benchmarked with official statistics.
• Transparency: When the ADP Research Institute launched the new National Employment Report in September, we released 12 years of weekly and monthly data, both seasonally and non-seasonally adjusted. It was our open invitation to researchers to come to our sandbox and play with our new data tool. Crowdsourcing adds rigor and strength to our data set. Scholarly interaction in improving private and public sector measures also strengthens the national statistics infrastructure.
• Scientific rigor: ADP partnered with the Stanford Digital Economy Lab to embed into our methodology a way of improving our measurement over time.

• They’re supplements, not substitutions: The NER is not a substitute for government data. Instead, it provides an alternative estimate that sidesteps the response challenges of government surveys. We also provide the data on a weekly frequency, which allows researchers and policymakers to observe labor market dynamics that were previously hidden.

My Take

The National Employment Report for September will be released Wednesday. Our world has gotten faster, and we’re doing our part to make economic data faster, too.