Open Data

Open Data: Let’s talk about more than just government

by  • 

Victoria is hosting its open data day and Hackathon Saturday the 23rd (Facebook Link). I plan on being there because I support openness and transparency, I’d like to learn more about available data sets, and hangout with like-minded people. The City of Victoria has taken steps since 2011 through Councillor Marianne Alto‘s initiatives and more to facilitate more open governance. Like any other government entity, there is valid criticism and issues to navigate, but stated goals exist and progress can be tracked and critiqued.

Enough people talk about open government data, and there’s consensus that governments should be more collaborative, open and participatory. But most of us spend more time and money interacting with non-government entities than we do with government entities. Look at your monthly budget. You will spend 30-40 percent on your mortgage or rent, goes to a non-government entity. The next biggest line items, probably groceries, car payments are all to private entities. Should we as consumers not expect the same open data sharing standards from our private entities as we do from government? The book Open Government, released for free by Safari books after Aaron Swartz’s death (does not appear to be free any more) has one chapter by Archon Fung and David Weil titled Open government and Open Society, which outlined my concerns very well:

Enthusiasts of transparency, which most readers of this book are, should be aware of two major pitfalls that may mar this achievement. The first is that government transparency, though driven by progressive impulses, may draw excessive attention to government’s mistakes and so have the consequence of reinforcing a conservative image of government as incompetent and corrupt. The second is that all this energy devoted to making open government comes at the expense of leaving the operations of large private sector organizations—banks, manufacturers, health providers, food producers, drug companies, and the like—opaque and secret. In the major industrialized democracies (but not in many developing countries or in authoritarian regimes), these private sector organizations threaten the health and well-being of citizens at least as much as government.

Open Government – Chapter 8 – Open Government and Open Society – Fung and Weil

I wrote briefly about one aspect of open data in our private interactions, shopping receipts. We spend a lot of time, effort and money shopping, yet we’re very unlikely to leverage the power of data to help us shop better because our individual decisions are captured in paper receipts. But there are many more examples.

  1. Mortgages – Do you have to go to every bank/lender’s website to do a comparison? Ratehub is a start, is there an API or download capabilities?
  2. Real Estate Data – Realtors control real estate data in Canada, I would call this a major conflict of interest. There are efforts to open this data up a la the US, but slow going. This is the biggest market transaction any of us will undertake in our lives, but information is controlled by the agency that benefits most from our lack of knowledge.
  3. Rentals – Craigslist is notorious for hoarding data and going after people who want to present data in more useful formats. Community posted information is created by the community, but captured by private entities due to network effects (everyone’s on craiglist, so I need to be there too, regardless of their data policies).
  4. Insurance markets – Government provided insurance information (ICBC – Car, MSP – health) is transparent. Try getting insurance in the open market for condos, homes and more, you’ll find the same pdf/paper quote formats that make it difficult to compare and choose wisely.
  5. Corporate governance – There is so much information missing on actual corporate structures, ownership, directorship, brand ownership and lobbying
  6. Pollution and resource use. Do we have a good idea what companies pay for water or power? Do we have a way of understanding who pollutes what and where?

My goal on open data is to advocate for openness in all of society, not just in government. Also, just because data is available does not mean it is open. APIs and download capabilities are key.

So, when you think open data, do try and shift your gaze away from government occasionally. Remember that your housing decision is much more critical than the salary information for the assistant city manager, so openness is vital everywhere.

cart

 

Update: as Kevin pointed out on twitter, the federal tax bill is pretty big. I was talking more in terms of the municipal parts like property taxes. The point nevertheless stands, we pay private entities large sums of money under poor data transparency conditions.

Environment, Immigration

Chinese coal mines in BC: Missing the forest for the trees

by  • 

The story of a Chinese company in BC hiring Chinese workers has received a lot of attention. Much of the attention has focused on the company’s decision to game the temporary worker system in order to avoid hiring “Canadian” workers. Many of the objections are made on nationalistic grounds, “OMG, THEY”RE TAKING CANADIAN JOBS”, which then leads down the path of racist anti-Chinese sentiment. This Tyee article (disclaimer: I am a Tyee monthly funder, but obviously have no editorial input!) summarizes the issues involved very well. Recent changes to Canada’s immigration laws make this kind of hiring logical, because it is now okay to pay temporary workers with little/no bargaining power 15% less than you would pay locals. Of course you have to document that there were no qualified locals, but as this particular incident indicates, there’s little/no actual enforcement unless a fuss is made.

I find temporary worker programs to be problematic because they provide no path to citizenship, no permanence for the people who want it, and cause ugly divisions in the community. If you think there are not enough “workers” in your community, open your borders, let them in and pay well, you’d be surprised.

I wanted to highlight two obvious issues that to my mind are as important:

  1. Carbon Bomb. It’s a coal mine! How many people in BC, which preens gloriously on its carbon tax, are aware that coal is BC’s Number One export? What is the point of having a carbon tax for consumers when producers get to make money off that carbon for free? Whether the coal is burned in BC, or in China, it causes the same damage. Whether the coal is used to generate power, or to make steel, it puts out the same amount of carbon dioxide. Whether the mine uses Chinese workers or locals, it produces the same climate changing emissions. So, why instead of making coal producers pay the real costs of their product, are we enabling them to evade carbon taxes, royalties, and save even more money by reducing wages? Also, coal mining is not employment intensive, as countless other people have pointed out. So it’s not really about the jobs either. Kevin Washbrook of StopCoal made this point as well in the Tyee article I linked to earlier.
  2. Does this mine have right to be there? The West Moberley First Nation, part of a Treaty 8 band is opposed to the project on its land. That should be the end of the story. The state of Canada has responsibilities as a settler entity to obtain free, prior and informed consent on development from the people it colonized. The US is a bit more honest in this regard as it regards the colonization as a thing of the past and gives its indigenous peoples little/no rights. Canada’s different, the indigenous here have specific standing because of Canada’s existing colonial links and Canadian governments and courts routinely confirm this standing. The BC government is currently negotiating treaties with many First Nations communities including the West Moberley First Nation.

We’re trying to set up a climate and environment disturbing, cost and tax evading coal mine on land that belongs to someone else using easily exploited temporary workers we can be racist towards.

coal

Energy, Environment

Methane Leakage, again

by  • 

In a sane world, we would be very concerned about measuring, reporting and closely regulating methane releases during its extraction and processing, especially if we claim that it is clean energy. This is nothing I, or other people haven’t said before, but here’s more research summarized in the very respectable Nature Journal indicating that measurable leak rates of methane can vary widely.

Preliminary results from a field study in the Uinta Basin of Utah suggesting even higher rates of methane leakage — an eye-popping 9% of the total production. That figure is nearly double the cumulative loss rates estimated from industry data — which are already higher in Utah than in Colorado

via Methane leaks erode green credentials of natural gas : Nature News & Comment.

10% is a large number. I’ve posted this picture from Wigley (2011) previously. At any leakage rate other than zero, which no one claims, the benefits of switching from coal to methane are very modest.

It's all about Methane leakage

It’s all about Methane leakage

We absolutely need to measure, control and regulate fugitive methane emissions from every BC site, and need to have solid regulation in place before we keep expanding natural gas infrastructure. We need our political leaders to start talking seriously about capturing methane leaks when they talk about BC’s natural gas “play”.

Politics

It’s not the policy, it’s the racism

by  • 

Articles on people of colour and voting patterns in the recent US election don’t touch on the racist rhetoric that the right has used for years. People of colour are frequent recipients of racist actions against them and the right’s use of racist language is completely internalized into their discourse and worldview. Just look at what Bill O’Reilly said post election:

“Obama wins because it’s not a traditional America anymore. The white establishment is the minority. People want things.”

The republican party thinks hispanics are not part of a traditional America. People of colour tend to notice these things. Obama has deported way more hispanic people than Bush ever did, and has not used his executive discretion to slow down enforcement till the DREAM act deferrals. But the democratic party has not been captured by the ugly racism that pervades  anti-immigrant rhetoric in the US.

So, change positions all you want, and help pass real immigration legislation that helps the millions of Americans living a difficult undocumented life get documented. But, the right needs more than that. It needs to convince its supporters that racism is unacceptable and to punish, not reward people for saying racist things and acting in racist ways.

The attorney general of Utah, Mark Shurtleff, a conservative Republican, said he was part of an “education campaign” to persuade Republican officials that “they need to reject the run-’em-out, deport-’em, enforcement-only approach that people think is the only voice of the Republican Party.”

Republicans Reconsider Positions on Immigration

Update: Mitt Romney’s post election statements where he labels everyone other than White people “special interest groups” are yet more evidence.

Photo courtesy Lorenzolambertino photostream used under a creative commons licence.

Transit, Victoria

Made in China buses: Is the fearmongering necessary?

by  • 

If you live in greater Victoria, you must be aware that BC Transit and CAW Local 333 are negotiating a new contract. Like a number of contracts negotiated in this time of fake austerity,  the negotiation is contentious because there are actual mandates from the government that salaries cannot increase unless “savings” are found elsewhere. I am not privy to how these negotiations are going, so no second guessing here on strategy or tactics. I hope things get settled, because I travel more than a thousand kilometres by bus every month and driving to work is not what I want to do, neither is crossing a picket line.

All that being said,  this new tack is disturbing.

Williams said B.C. Transit “wants the unrestricted right to bring in Chinese-built” community shuttle buses with lower safety standards, which could be piloted by part-time drivers “at a significantly lower wage rate than conventional bus drivers.”“B.C. Transit literally had to go to China and get these buses designed and built there to get around higher safety requirements,” Williams said in the release.

via Victoria bus drivers set for overtime work ban starting Monday, union says

Yes, China bashing is a quick way to gain sympathy. and happening quite a bit this month because the Chinese and Canadian governments are negotiating a secret trade agreement (Leadnow campaign link) that gives corporations of both countries all kinds of rights and privileges that we could only dream of getting for ourselves.

Is there any evidence that Chinese made buses are unsafe, especially when they need to conform to Canadian safety standards? Is there any evidence that these standards are being gamed? These are different questions from “are these buses suitable”? “Are the lifecycle costs for these buses being understated”? I wish CAW Local 333 would take the time to frame this issue more accurately, because this issue is not about China, it is about us.

Lost in all this China bashing and a cynical attempt to appeal to our “other” phobia is the obvious conclusion that it’s not the “made in China” aspect of manufacturing that makes a product less durable or of poorer quality, it is the insistence of markets to lower standards on the products to cut short-term costs or to increase profits. China, like many other countries, probably more than Canada, manufactures large quantities of high quality products routinely. It’s not China’s fault that your crappy London Drugs coffee grinder can’t actually grind coffee and breaks when your cat sneezes near it. It’s the fault of the companies that sell you stuff, and our own inability to balance short-term price vs. long term cost. It is also the oppressiveness of the Chinese government combined with consumers need for cheap, and market profit needs that exacts a high price on the Chinese makers of the high quality IPhone.

So, ask hard questions about the suitability of the buses, and question the market mechanisms that brought us here. Unions are a very necessary buffer against market excess and corporate control. But do we have to use “made in China” as a cudgel again? As my friend says,

Made in China has become a short form for criticisms of the market, which are credible. But the problem is that it slips in the othering too

 

sea turtles

Robot sea turtles for ocean Safety

by  • 

I have only one question: Will these cute robot turtles come up to shore every year to lay eggs that will turn into cute little robot turtle hatchlings?

I have to remind myself sometimes that this blog is named after a sea turtle and that my turtle overlords demand a post or two once in a while that propitiates them.

Robotic sea turtles, on the other hand, can do all sorts of things. They can find out where a pipeline or a ship hull is damaged. Or the extent of an oil spill, or locate bodies in the wake of a disaster.

via Robot sea turtles could help keep the ocean safe and clean | Grist.

twitter

Twitter Weekly Updates for 2012-10-07

by  • 

Powered by Twitter Tools

policy

Facebook and deep customer tracking, I want my data!

by  • 

To nobody’s surprise, Facebook, just like any other entity selling you stuff, or selling you to people who sell you stuff is trying to connect more and more sets of previously unconnected data. This particular case deals with brick and mortar store data that is linked with customers’ email addresses and loyalty cards.

Facebook will be using Datalogix to prepare reports for its advertisers about who, if anyone, bought more of their stuff after they ran ads on the social network. But by matching your Facebook profile with your CVS bill, this means that Facebook has the potential to know some of your most intimate details (my, that’s a lot of bunion cream you’re buying!), and the privacy concerns are enormous. When DoubleClick attempted something similar to this, user-backlash ultimately led them to cancel the project.

Can Facebook Possibly Build a Business Model That Isnt Inherently Creepy? – Derek Thompson – The Atlantic.

Corporations (more than government, open data activists!) have been deep mining our data for years. It is part of creating the information asymmetry that enables profits to be made. You may remember this story about Target (coming to Canada as soon as we can learn to say Tarjay) and how they outed a teen’s pregnancy.

 About a year after Pole created his pregnancy-prediction model, a man walked into a Target outside Minneapolis and demanded to see the manager. He was clutching coupons that had been sent to his daughter, and he was angry, according to an employee who participated in the conversation.

“My daughter got this in the mail!” he said. “She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?”

Companies’ ability to reduce us to a shopping probability statistic is only going to get better as they learn to connect more of our data and computing gets faster. Can regulation keep up? Can customer outrage keep up with companies offering us coupons to keep us temporarily happy as impulse centres in our brain are carefully triggered for profit? Can customer outrage even keep up with the barrage of occurrences?

Receipt 2.0

I don’t think we can keep up. So, I want my data. I want information on what I bought, when I bought it, where I bought it in a standardized open data format. No, not a paper receipt, not a paper receipt scanner, but something that can be beamed to my phone, or emailed to me. I want to know when I buy coffee. Can I correlate my shopping habits with my mood?  Do I buy more random electronics when I need a pick me up? What is the spread in the price I paid for my favourite cereal? Do some stores price it differently on Wednesdays? I want apps that can mine my data and tell me where to buy my cereal, or when not to buy. I want apps that can tap into a product database and give me a carbon footprint, or a fair trade pass/fail, or a local product breakdown.

Also, I do not want to re-enter the same bits of data multiple times and increase error. A payment made to my dentist should be sent directly to my extended health “insurance” provider for a refund. It should also go to my tax receipts virtual pile and await reimbursement. Any tax deductions can easily be tagged and directly entered into my tax preparation software at the end of the year. if I want to expense something for work, I should just be able to tag them and send them along. I can’t imagine how much time and effort this will save in error checking, manual entry, auditing, so much more.

Some of this is possible with systems like Mint, but they operate on a payment level, not on a line item level.

The new mobile payment system Square (not in Canada yet) shows some potential, so does Intuit’s GoPayment, which is available in Canada. But these payment systems emphasize ease of payment on both sides of the transaction, not the ability to mine our own data.

Can this happen via the market with no regulatory push? I don’t see how. Reducing information asymmetry is not in corporations’ interest. So it will have to be regulated. You have our data, just give it to us.

Not to mention, this is the “free market” way to go. Think of all the innovation that can be unleashed on the consumer side. Think of the apps that can provide better financial advice, the apps that can collate data at city/regional level and help consumers make better decisions.

Will companies have to spend money to make this happen? Yes. This will not be challenging for larger companies who already spend millions deep-mining our data. What about small business? This is where small tweaks to new systems like Square or GoPayment can be the game changer. Square already charges less for a swipe than a typical Visa transaction. So, I would see local business as saving money. When I buy local, I usually feel a bit more connection with the product. Imagine seeing my decision reinforced by data from Receipt 2.0. Small local business cannot data mine, but can generate enough goodwill with local consumers to get access to their data. Unlike Target, when my corner soap store does something unethical with my data, they cannot survive the bad publicity.

Are there privacy concerns? Yes. But our privacy is already compromised the minute we pay anything other than cash, or use the internet (Tor users, stop smirking). Good regulation can address most concerns.

My information is being used to make me a consume more, I want it to make me a better “consumer”. Open data efforts have focused intensely on public and government data, while privacy activists have tried unsuccessfully to stop private data gathering. I would like open data advocates to look carefully at liberating corporate mined data as well.

Featured Image courtesy the Culturally Authentic Picture Lexicon used under a creative commons licence.

twitter

Twitter Weekly Updates for 2012-09-30

by  • 

Powered by Twitter Tools

Climate, Politics

Dear Mulcair: Connect your short term oil goals with energy transformation

by  • 

NDP leader Thomas Mulcair has finally listened to the legions of Globe and Mail comment thread participants (and some other people, of course!) who repeatedly urge policy makers and oil companies to build a pipeline West -> East. I believe Bob Rae has talked about this idea approvingly as well. Why? Because Western Canada exports oil at a “discount”, and Eastern Canada pays “full price” from non-Canadian sources.

In a speech to the Canadian Club of Toronto at the Royal York Hotel, the federal NDP leader gave his clearest sign of support yet for the notion of a West to East pipeline that would allow producers to receive higher prices for their crude oil.

The NDP leader’s speech also repeated his concern that western energy developers are not paying the full cost of the environmental consequences of their projects. He said this is leading to an artificially high Canadian dollar, which hurts other sectors of the economy.

Mulcair wants East-West Pipeline

The full text of his comments can be seen at iPolitics and has much more than Globe and Mail Report (it wouldn’t have fanned the flames otherwise).

Mulcair spoke about this pipeline, he also talked a lot about income inequality, robust government, and making polluters pay. He talked about strengthening environmental safeguards, ending fossil fuel subsidies and more.

What he didn’t say: That tackling climate change requires a fundamental transformation of our system.

Sometimes, what is not said is more important than what is said.

If this proposal to use Canadian oil more “judiciously” by building a short-term closed supply chain is just part of a clear plan to go to a renewables and demand-reduction based energy transformation, propose away. We do need to hold both these truths in our heads at once: The tarsands are a big source of short-term revenue feeding our fossil fuel based culture, and unchecked climate change will kill many. It isn’t possible to cut fossil fuel use to zero next year, but it is imperative to cut emissions from fossil fuel use to near zero in the medium-term. Any policy that makes sense within that main objective should be looked at on its merits, but ending fossil fuel emissions soon HAS to be a cornerstone of any progressive energy policy, the crisis demands no less.

So Mr Mulcair, propose oil pipelines if you wish, it may make for good short-term politics (read comments below the article), and who knows, maybe even tolerable policy. But remember to frame it as part of the necessary energy transformation. Politics is messy, and lasting change requires a broad coalition, don’t alienate progressive supporters right away.