Human imperfection - Saving us from the machines
Artificial intelligence is big business, it has helped in many fields and is being applied to everything from winning board games to keeping cars in their lanes on the motorway. It has also been a driving force bringing us towards the technological singularity - the point at which technological growth exceeds human control and leads to a superintelligence being created that far outweighs human intelligence.
At such a point this superintelligence would have the ability to independently recreate itself and iterate through billions of versions getting better and better until reaching something close to perfection.
Now in reality this superintelligence might simply be a software agent who's job it is to solve the "travelling salesman problem" - finding the shortest path to a given set of destinations. In this instance having a self improving agent would be beneficial, as calculating distances between cities to get the optimum route - especially if additional cities are added / removed - isn't something that could be calculated quickly or accurately by a human.
Doesn't sound much like the introduction to The Terminator, The Matrix or any other sci-fi dystopia, but listen to Elon Musk or the late Stephen Hawking discuss "the singularity" and it seems like these low-level activities would be a precursor to more dangerous and broader leaps that will eventually lead to technological servitude.
What I believe is that what is missed from this point of view are those elements of life that AI has never, and will never be able to master. There are a lot of clever systems out there striving for perfection but therein lies the problem; as humans we don't actually want perfection.
Looking from a purely procedural and logical standpoint, for a society to run "perfectly" you would see something akin to George Orwell's 1984; citizens living in identical homes, taking up as little space as possible to ensure for the maximum number of lives to fill the space available. There would only be employment for those who were physically and/or mentally able to complete the few specific tasks required to generate the resources for the society [food, energy sources, etc].
There would be no need for anything - or anyone - else. Arts would be completely redundant and anyone showing individuality would be cast out of the society [disposed of] as they would not be performing perfectly with the rest of the citizens and would be leaching off "the system".
The reason this scenario seems unimaginable is the same reason that the worry over AI creating this future is not plausible. It's the human element. As people we curate and celebrate imperfections, anomalies and the randomness of life. Where a machine would see a part not working in a uniform manner as faulty and to be removed, we often see this as an opportunity to overcome adversity.
There are countless stories of individuals who have come from the humblest of beginnings, who have been held back by physical or environment impediments to excel against all odds to thrive in a particular field. Often found in sports, the underdog story is a favourite both in reality and in fiction and underlines the need for imperfection in society.
So with the superintelligence striving for uniformity and utilitarianism, where anything that does not fit the system is considered wrong and to be removed there will be push back and outcry from society that will result in either abandoning the system or forcing it to adapt to a more diverse position. In either case the post-apocalyptic future is averted, with the superintelligence working alongside humans in order to solve problems in evermore efficient ways.
There is another possibility though, and one that is mentioned in the movie The Matrix; that we are already enslaved to the machines, being harvested for our biological content and that we are simply living in a simulated world already. Agent Smith goes on to describe how the first version of the simulated world was a paradise, where there was no disease, war or hardship - but that humans rejected this "perfect" world as we seem to be programmed to need imperfection in order to function..
But, but, but! That was from a film, dreamt up by and created by humans. A creation, with just enough credible elements to make you consider it could happen, or happened. Surely. "They" wouldn't allow for us to dream up blueprints of the machine we are existing in would they?
I choose to believe we are masters of our own destiny and that life's rich pageant is not something that could be brought down by a system working against the parts we hold most dear. Ignorance it may be, but perhaps it's better than the alternative.