In late 2009 I was catching up at a Starbucks with Bill Buzenberg, the Executive Director at the Center for Public Integrity. I told Bill about some of the interesting research Global Integrity was doing at the municipal, state, regional, and provincial level in a number of countries. But I was frustrated, I said, by the lack of good data here in the United States about how state governments were actually set up when it came to anti-corruption enforcement and deterrence. It was easier for Global Integrity to find teams of qualified researchers in, say, Peru to assess region-level transparency and governance than it was to put together a similar effort in the U.S. states.

I worried that much of the attention in the U.S. at the state level had become too siloed and focused on particular pet rock issue areas: freedom of information, money in politics, open data, and state budgets to name a few. Was anyone looking at the whole picture? Did we have any idea which states were "hardwired to fail" when it came to corruption and governance?

“No,” Bill said. “But let's fix that." And thus the State Integrity Investigation was born.

Today marks an important milestone with respect to the pioneering State Integrity Investigation: we are making pre-publication data -- all 16,500 data points -- available to the public to comment on, raise questions about, and share more than a month before the "official" publication of the project results.

Why did the project partners decide to open ourselves up to this level of scrutiny? It might be helpful to review the history of how this incredible effort came to be.

Global Integrity, one of the three core partners of the State Integrity Investigation, has spent the better part of a decade tracking governance and corruption trends globally, including in North America. We've helped to pioneer some of the more innovative ways of assessing corruption risks and anti-corruption mechanisms at the national and sub-national levels in more than 120 countries by combining journalistic reporting with social science data gathering.

Along the way, we built some technology to help us gather and publish that information more efficiently, which explains how a staff of less than 15 spread across two continents has published more than 10 million words of text and more than 100,000 data points without going absolutely crazy. Those efforts have led to real change in a number of countries by providing both government and non-governmental organizations with solid information that can be used for evidence-based policymaking.

Early on in the project brainstorming, we made several important decisions that are already impacting the public uptake of this project:

• We wanted to invest many months into talking to the country's leading experts and advocates focused on state government before developing our methodology. Seventy-five interviews later we had the project's 330 Integrity Indicators.
• We wanted to work with the best and brightest experts we could find in each state to gather those data, and that meant recruiting leading statehouse reporters in every state.
• We wanted to open the research and reporting process up to the public to ensure our results were as balanced and as accurate as possible. We made early decisions to publish the names of all of the lead reporters in each state, to find engaged and informed citizens to help review the draft data, and to (today) make pre-publication data available for public feedback before the official project launch. Many of those decisions were novel for Global Integrity, and we are anxiously awaiting early feedback to see whether some of this project's experimental techniques can be applied to other efforts.

After all of the hard work by the reporters, project managers, editors, and online community team, a simple reality remains truer than ever: corruption matters in the states because it impacts people's lives. Your state budgets are broken in part because governments have given too many tax breaks to special interests that fund politicians' campaigns. Your roads are crumbling because tenders for infrastructure projects often go to politically connected companies, not those with the most competitive pricing or highest quality. Your elected leaders often operate with impunity because of broken information request systems, gutted state ethics commissions, and patronage controlled civil services.

For all of the attention paid in the past fifteen years to the crisis in governance at the national level in the United States -- from Lincoln Bedroom scandals to Enron to McCain-Feingold, Citizens United, and Super PACs -- we still struggle to understand whether things are better or worse at the state level. That's what we hope this project will answer.

-- Nathaniel Heller is the Executive Director of Global Integrity.