Open Data: Let’s talk about more than just government

Victoria is hosting its open data day and Hackathon Saturday the 23rd (Facebook Link). I plan on being there because I support openness and transparency, I’d like to learn more about available data sets, and hangout with like-minded people. The City of Victoria has taken steps since 2011 through Councillor Marianne Alto‘s initiatives and more to facilitate more open governance. Like any other government entity, there is valid criticism and issues to navigate, but stated goals exist and progress can be tracked and critiqued.

Enough people talk about open government data, and there’s consensus that governments should be more collaborative, open and participatory. But most of us spend more time and money interacting with non-government entities than we do with government entities. Look at your monthly budget. You will spend 30-40 percent on your mortgage or rent, goes to a non-government entity. The next biggest line items, probably groceries, car payments are all to private entities. Should we as consumers not expect the same open data sharing standards from our private entities as we do from government? The book Open Government, released for free by Safari books after Aaron Swartz’s death (does not appear to be free any more) has one chapter by Archon Fung and David Weil titled Open government and Open Society, which outlined my concerns very well:

Enthusiasts of transparency, which most readers of this book are, should be aware of two major pitfalls that may mar this achievement. The first is that government transparency, though driven by progressive impulses, may draw excessive attention to government’s mistakes and so have the consequence of reinforcing a conservative image of government as incompetent and corrupt. The second is that all this energy devoted to making open government comes at the expense of leaving the operations of large private sector organizations—banks, manufacturers, health providers, food producers, drug companies, and the like—opaque and secret. In the major industrialized democracies (but not in many developing countries or in authoritarian regimes), these private sector organizations threaten the health and well-being of citizens at least as much as government.

Open Government – Chapter 8 – Open Government and Open Society – Fung and Weil

I wrote briefly about one aspect of open data in our private interactions, shopping receipts. We spend a lot of time, effort and money shopping, yet we’re very unlikely to leverage the power of data to help us shop better because our individual decisions are captured in paper receipts. But there are many more examples.

1. Mortgages – Do you have to go to every bank/lender’s website to do a comparison? Ratehub is a start, is there an API or download capabilities?
2. Real Estate Data – Realtors control real estate data in Canada, I would call this a major conflict of interest. There are efforts to open this data up a la the US, but slow going. This is the biggest market transaction any of us will undertake in our lives, but information is controlled by the agency that benefits most from our lack of knowledge.
3. Rentals – Craigslist is notorious for hoarding data and going after people who want to present data in more useful formats. Community posted information is created by the community, but captured by private entities due to network effects (everyone’s on craiglist, so I need to be there too, regardless of their data policies).
4. Insurance markets – Government provided insurance information (ICBC – Car, MSP – health) is transparent. Try getting insurance in the open market for condos, homes and more, you’ll find the same pdf/paper quote formats that make it difficult to compare and choose wisely.
5. Corporate governance – There is so much information missing on actual corporate structures, ownership, directorship, brand ownership and lobbying
6. Pollution and resource use. Do we have a good idea what companies pay for water or power? Do we have a way of understanding who pollutes what and where?

My goal on open data is to advocate for openness in all of society, not just in government. Also, just because data is available does not mean it is open. APIs and download capabilities are key.

So, when you think open data, do try and shift your gaze away from government occasionally. Remember that your housing decision is much more critical than the salary information for the assistant city manager, so openness is vital everywhere.

Update: as Kevin pointed out on twitter, the federal tax bill is pretty big. I was talking more in terms of the municipal parts like property taxes. The point nevertheless stands, we pay private entities large sums of money under poor data transparency conditions.