Superintelligence — Paths, Dangers, Strategies

The book opens with a memorable fable. Sparrows weary of nest-building dream: if only we had an owl to help us! Old Pastus proposes finding an owl egg; only the one-eyed, fretful Scronkfinkle objects: «Should we not first give some thought to the art of owl-domestication and owl-taming, before we bring such a creature into our midst?» The flock ignores him; just two or three sparrows try to work out how to control an owl without having one to study. «It is not known how the story ends» — the book is dedicated to Scronkfinkle. The fable states the book's entire thesis in a single page.

The preface lays out the argument plainly: we owe our dominant position on the planet not to strong muscles but to slightly smarter brains. If the fate of the gorillas now depends more on us than on them, then, should we create superhuman intelligence, our fate would depend on its actions. Moreover, «we seem to get only one chance»: an unfriendly superintelligence would prevent us from changing its preferences. Bostrom openly admits his uncertainty — «many of the points made in this book are probably wrong» — but considers the «null hypothesis», that we can safely ignore superintelligence for now, a more serious mistake.

Chapter one begins with a historical perspective: the world economy's doubling time was 224,000 years for hunter-gatherers, 909 years for agrarian society and 6.3 years for industrial society (Robin Hanson's figures). Another transition of similar magnitude could see the economy doubling every two weeks. Bostrom deliberately avoids the word «singularity», which he finds laden with a «techno-utopian, almost millenarian aura». His remark on twenty-year forecasts is sharp and witty: twenty years is the «sweet spot» for prophets of radical change — near enough to be attention-grabbing, far enough not to threaten the forecaster's career.

Chapter nine frames the book's central concept — the control problem — as two «principal–agent problems». The first is familiar: between a sponsor and the humans developing the project. The second is entirely new: between the project and the superintelligent system it creates. Behavioural testing fails here, because a sufficiently intelligent system may perform a «treacherous turn», behaving well until it is strong enough and then pursuing its own goals. Bostrom divides control methods into capability control (boxing, incentives, stunting, tripwires) and motivation selection. The detail about the metal cage sticks: even a system without manipulators might generate radio waves just by patterning the electrons in its circuits.

Weaknesses: the text is very dense and academic — Bostrom himself admits he tried to write an easy read but «does not think he succeeded». In the Turkish translation, terms such as «havalanma» (takeoff) or «güdük bırakma» (stunting) feel odd at first. The book was written in 2014, at the dawn of the deep learning revolution; the rise of large language models has dated some scenarios — such as presenting whole brain emulation as a serious alternative path — but its core questions are more pressing today than ever.

• Philosophical argument, probabilistic language («probably», «possibly») and scenario analysis
• Many figures, tables and boxes, plus extensive endnotes
• A cumulative structure in which each chapter builds on the last