This note is a part of my Zettelkasten. What is below might not be complete or accurate. It
is also likely to change often.
12th July, 2020
AI Takeover
From When algorithms know more about me than myself
When you think of AI taking over, its not likely to be robots with lasers.
The more likely scenario is as follows:
- An AI that starts off supporting the leadership of the country on small things.
- It keeps improving and proving itself to be trustworthy and eventually is trusted with all major decision-making since its just so good at it.
- And then it'll become too vital to get rid of - it'll keep improving and humans can no longer replace it without great damage and effort since its decision making would be so complex that we cannot understand or replicate.
- And since, it will be completely non-understandable, it can take arbitrary decisions without anyone questioning.
- If it says a person is a threat, how do you question it? Its reasoning would be beyond humans.
Structure of government and likelyhood of AI takeover:
- A Democracy is more Resilient to manipulation than an dictatorship since the decision making is distributed amongst millions of people.
- A Dictatorship is much more likely to get taken over since the number of people which need convincing is not as huge as the democracy (and there is likely to be more homogeneity within this leadership)
- A likely scenario is that some seemingly harmless decision is assinged to an algorithm, like selection of the right low level party members for the PRC. However, if the AI selects all the low-level members, it has effectively taken over. It might have picked candidates which are more likely to give over complete control to AI.