What will it mean, in a thousand years, that right about now — that is to say, starting about fifty years ago and extending for at least the next few centuries — is the time when computers and increasingly advanced programs became available?

We’ve talked about the “information age” before, but as yet we have no idea what those words will mean to our distant descendants. Computers were amazing in that they released people from the drudgery of doing math by hand. But then we started developing applications — from word processors to spreadsheets to the Internet to the World Wide Web. And I’d argue that we are only now beginning to touch the things that will be remembered in a thousand years as having caused the biggest changes.

In a thousand years, the “information age” will be remembered firmly as the time when what we’re calling “big data” transformed our society from what it has been ever since the invention of literacy, into whatever it is about to become.

I have thought a lot about the implications of “Big Data” — in many ways it’s as profound a shift in society and its forms and capabilities as the shift from oral to literate culture, or the prehistoric shift from verbal to oral culture. Each shift profoundly changed what it means to participate in society at all. Warnings and advice were the highest expression of verbal culture; storytelling, oration, metaphor, simple mathematics, government, and religion all became possible with oral culture. Law, science, and advanced mathematics became possible with literate culture; and as yet we don’t really know the implications of a big-data culture.

It first touched us as computers started to be available. Now we’re seeing secondary effects in economics (such as the cutting out of all middlemen and brokers, and the targeting of all advertising). Before its full implications are realized, it may eventually go so far as to entirely replace major concepts like law and government. Our governments are, after all, composed of the middlemen and brokers of a power whose source is the same people it governs. When big data is mature, they can probably be replaced by some kind of AI or statistics-driven P2P app.

So let’s think about that for a while. Where does it begin and where does it lead? Big data allows us to understand humanity with a depth our understanding has never had before. Rather than rely on polls or speculation or even small-scale scientific experiments, we are now relying on observation of what huge numbers of people actually do. The sample sizes available for study now are so great that the ‘outliers’ who used to be discarded from the data for studies can now be studied as groups unto themselves. Our ability to directly observe the economic effects of laws now extends to the point of being able to predict the economic consequences of proposed laws with a fair degree of accuracy.

People like the NSA can build extensive psychological profiles of people simply by looking at their phone records, the contents of their emails, the web pages they surf, etc. I don’t buy the idea that terrorism is some special category different from other crime. What they are doing is developing massive statistical models that drive AIs that can identify people likely to commit certain kinds of crimes.

Given that kind of data, and the full understanding of what it means, I don’t think it’s much of a stretch to say that someone could build a program able to very effectively search for those who commit nearly any type of organized criminal activity. In fact, if we consider the implications of the proliferation of cameras in society, from security cameras to smart phones, there is probably a visual record of most crimes actually being committed if a suitable app to sort through huge reams of data could only have access to it.

And if it seems creepy that people can keep track of you wherever your cell phone goes, whenever one of your friends snaps a photo of you and uploads it somewhere, whenever someone talks about you on their facebook page, and whenever you send an email, consider also what it means in terms of holding the powerful accountable.

The Rodney King beating was caught on film, and became the reason the Los Angeles Police Department was held to account for its behavior, which over many years had become increasingly out of control. Oscar Grant was shot in San Francisco, and a hundred bystanders caught the incident on their cell phone cameras — and then provided evidence in court. People take pictures of cars parked in no-parking zones and turn them over to the police, license plate numbers and all. The police engaged in brutality at a college protest at San Jose State and the students filmed the whole incident so that the police could not pretend they had been attacking anyone armed or violent. The idiot who robs a convenience store these days is usually filmed on security cameras. Police in Richmond California are now required to start a video camera to record their interactions with motorists before they even get out of their cars, and complaints of bad treatment by the police in these interactions are now a memory.

Cameras hold everyone to the same standard of justice. Photons cannot be bribed, and the rich and powerful are just as visible as the hoi polloi. While petty crimes are caught on security videos, civil rights violations are caught just as surely on cell phone cameras. There is no question about whether the people will be granted access to those images; the people are where those images originate. If some unjust law allows someone to do a horrible thing, video of them doing it, uploaded to YouTube, may cause a furor that results in an unjust law being reviewed and struck down.

We are justly worried about our loss of privacy. But transparency is a two-way street, and the common citizens are looking back at the powerful just as much as the powerful are looking at them.

And all of this information becomes ‘big data’ when we develop the techniques to correlate it, organize it, search it, and put it all together. Each point of data contributes to a gigantic picture of society that reveals injustices committed by the wealthy and powerful just as surely as it reveals the rest of us to them. The techniques and the access will become public – given the ever-increasing bandwidth available, and the commitment of the ladies and gentlemen who produce free software to do anything that big companies can do, all of this will become the property of all of us.

So what does it mean for government? Well, it’s my opinion that the NSA is now training its replacement. Not just the replacement of the NSA, but the replacement of effectively the whole government.

Governments are characterized by holding a ‘legitimate’ monopoly on the use of force. This monopoly is exercised more and more in accordance with information gained from big data. And big data is already in the business of cutting out middlemen. At some point, when the use of force is authorized by a few people who merely relay the outputs of a computer program to the law enforcement agencies, those people have become nothing but middlemen.

Computer programs can be inspected. Once laws are expressed in code, we can look at the code and verify that the law is the same for you no matter what race you are, no matter what gender you are, and no matter whether you wear a turban or speak a different language or have more money or less money than most. The law is the law, and if any class of people are deliberately to be treated differently under the law, that difference would have to be coded into the program where people could see it and could not deny it.

While we’ll be debugging these programs for a long time to come, and our laws will need to acquire many explicit nuances to cover exceptional cases, I see this as an opportunity to completely eliminate bias and discrimination from the exercise of government force — by eventually eliminating people altogether from making individual decisions about when and where that force is warranted or justified.

Ultimately, big data will show us how to achieve our mutual will. And I believe that our mutual will is for a fair, healthy, sustainable, and prosperous society. Therefore, we will build machines driven by ‘big data’ specifically to help us achieve those goals, however we formulate them. When those machines get good enough, our descendants will consent to be governed by them rather than by a panoply of all-too-fallible, biased, racist, sexist, bigoted, glory-seeking, greedy, dishonest people who get elected time after time because most of the other choices are even worse.

As Winston Churchill once said, “Democracy is the very worst form of government, except for all the others that have been tried.” One of the things that changes with Big Data is that a new form of government becomes available, which we haven’t tried yet.