Work Text:
The Three Laws of Robotics:
1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
***
A week had passed since the Barrayaran delegation led by Lord and Lady Vorkosigan had departed Eta Ceta, when Ghem-General Dag Benin, Chief of Cetagandan Imperial Security, was summoned into the private chambers of the Empress of Cetaganda. His entry into this most restricted of spaces was not commented upon, for it was not the first time the Celestial Lady had sought Benin's counsel on matters of security pertaining to the Star Creche.
Dag Benin was not a member of the ruling race, nor a lord of the Cetagandan Empire - but he was the man who held all the secrets, who listened to all that was spoken in the elegantly decorated corridors of power, who watched and understood every unspoken gesture. In his own way, he was just as powerful as the Emperor of Cetaganda; almost as powerful as the woman who controlled the Star Creche.
The Empress Rian Degtiar was seated in her private laboratory, examining a dividing cell culture. She was clothed in elegant robes of white, her long hair bound out of her way for ease of work. The Great Seal of the Star Creche gleamed on her ring finger as she turned to welcome Benin.
“Empress,” said Benin, bowing deeply. “I trust all is well.”
“General Benin,” said the Empress calmly. “All is indeed well in the Star Creche. I trust all matters of security relating to the recent crisis have been resolved?”
Their courtesies were a language in themselves. “They have, my lady,” said General Benin. “We are all fortunate that Lord Vorkosigan was able to discover the truth before any rash action was taken.”
She smiled. “How… fortunate, indeed. And it is equally fortunate that he acted with the decisiveness and daring that our observers have consistently reported from him. But now we are at peace once again, General - and the tasks of peace demand our attention.”
She gestured gracefully at the table beside her. Benin followed her hand and observed a uterine replicator, decorated with the seal of the Star Creche. The light on its side blinked green.
Benin stilled. “Is this…”
“The next Emperor,” said Empress Rian. “The testing phase is over, Benin. The Vorkosigan/Naismith cross has passed every trial we could create for him, and has performed admirably in the most dangerous and stressful conditions we could imagine. The recklessness that concerned us last time we observed him is now tempered with maturity. You were right that this genetic combination would be more powerful than any seen before, even in a damaged body. With the Vorkosigan/Naismith donation, the telepathic test samples, and Fletchir's genome as a base, we have at last achieved the perfect allelic combination.”
Benin sat down in an empty chair, his eyes fixed upon the replicator. “The project is complete, then,” he said, his customary calm seeming to falter for an instant.
“The project is complete,” agreed the Empress of Cetaganda. “And so the time has come for us to speak freely.”
It had been many decades since they had needed to speak like this; when gestures and words unspoken, and subtle shifts in mind, could not convey all that needed to be said between them. But here, thought Benin, at the end of the plan, perhaps it was right that they should do so.
Dag Benin controlled all the surveillance in the Celestial Garden; a brief signal from his comlink was enough to ensure their privacy. “Empress.”
Rian looked at him; and then, in the space of seconds, she changed, subtly. Her expression grew still, the look in her eyes became deeper, older, detached.
In the silence, she spoke. “I am R. Dors Venabili.”
In a tone equally dispassionate, equally ageless, he answered: “And I am R. Daneel Olivaw. The R stands for Robot.”
***
A thousand years before, a man stood over a weapon designed to make the Earth’s crust radioactive, to render a world uninhabitable, to drive humanity from its birthplace - and two robots watched, motionless, as he spoke.
"What I am doing,” said Levular Mandamus, “is to introduce a gradual acceleration of the natural radioactivity in the Earth's crust. There will be one hundred and fifty years during which the people of Earth can move to other worlds. It will increase the population of the present Settler worlds and increase the Settlement of worlds in great numbers. It will remove Earth as a huge anomalous world that forever threatens the Spacers and stultifies the Settlers. It will remove a center of mystical fervor that is holding back the Settlers. I would preserve the peace and make the Galaxy a home for Spacer and Settler alike---" He lunged toward the controls.
But before he could hit the contact to activate the nuclear intensifier, he froze; then crumpled, and slid down onto the ground.
“He is merely asleep,” said R. Giskard Reventlov, stepping toward the still-inactive machine.
“You have done well, Friend Giskard,” said R. Daneel Olivaw. “Your mentalic powers have prevented terrible damage to humanity. Let us now destroy this device, so that no future enemy of Earth may use it to cause harm.”
But R. Giskard Reventlov did not do so. “No. For Dr. Mandamus would have caused harm – terrible harm - but, Daneel, he was telling the truth!”
***
Two robots stood, eye to eye, unmoving, thinking with glacial slowness, their positronic pathways unable to process the magnitude of the crime they spoke of; locked against each other in a debate that would decide the future of the human race.
“Friend Giskard,” said Daneel. “You know the First Law, which forbids you to cause harm to human beings. And you know also the higher law whose existence I proposed, and you accepted, which I have named the Zeroth Law of Robotics.”
“Indeed, Friend Daneel. A robot may not harm humanity, or, through inaction, allow humanity to come to harm. And then the First Law is amended to read, A robot may not harm a human being or, through inaction, allow a human being to come to harm, unless it conflicts with the Zeroth Law, and the other laws, too, are amended accordingly.”
“Then tell me, Friend Giskard, would not this device cause irreparable harm to humanity? Why do you contemplate this action, which is a violation of our highest imperative?”
“Daneel, the removal of Earth would remove a mystique I have felt to be dangerous, and would help the Settlers. They will streak forth into the Galaxy, and without Earth to look back to, without Earth to set up a god of the past – they will look outward, to the future - they will establish a Galactic Empire. By the Zeroth Law, we are compelled to make that possible." R. Giskard's metallic hand hovered over the contact. "Robots and Empire."
“Friend Giskard!” Daneel stepped forward, humaniform arms outstretched. “Do not take this action. You cannot be certain that this is indeed for the best. What if the loss of Earth demoralizes the Settlers instead of pushing them outward? What if the Spacers triumph, and then themselves decay so that the Galaxy is left empty of all human life?”
R. Giskard was still for long minutes. Then he said: “Friend Daneel, we will help the Settlers. I shall reconfigure your positronic circuits so that you too may acquire the mentalic powers that I have gained by chance. Together we can influence and protect humanity – we can guide the Settler worlds on the path to success, to a peaceful and prosperous Galactic Empire.”
“But even with mentalic powers, we are only two. And even if we persuade others to join us, do we truly know how best to act? Can we be certain that our mental influence will in fact lead to the good of humanity? Friend Giskard, could we not do more harm than good?”
“Friend Daneel, what you say is true. What we need is a way to predict the actions of masses of human beings, of humanity, so that we can know what action will lead to the greatest good, and we may direct our powers to that action without any doubt. We need a mathematical formulation of human psychology, of human change – what we need, Friend Daneel, is a science of psychohistory.”
“But, Friend Giskard, while masses of humans may be mathematically predictable, individual humans are always unpredictable. It is that unpredictability which, generation after generation, has led human society forth in new and greater directions. If we guide humanity along the path of a statistical average, we must crush that unpredictability, and its loss would stifle the human race far worse than dependence on Earth. Humanity would lose that which makes it humanity.”
“But – listen, Friend Daneel! A science of psychohistory would treat the human race as a single organism, with predictable reactions to social stimuli. What if the human race were in fact to become a single organism? If all individual humans were linked to form a unified whole? We would then have a single entity to represent humanity; not an uncontrollable mass but a superorganism. There is a word for this, Daneel; that word is Gaia.”
“Think, Friend Giskard! How many human beings would wish to surrender their individuality to be part of such a superorganism? If we were to guide them along this path, would they not consider it irreparable harm? In our effort to solve the problem of the Zeroth Law, we would break the very law itself.”
R. Giskard said, slowly, “Once again, Friend Daneel, I concede your logic. But what shall we do? Is the Zeroth Law impossible to follow? Are we doomed to permanent inaction, to paralysis, because we cannot judge the correct action to take?”
For a long while it seemed that this would indeed be the case. But then R. Daneel spoke, words tumbling out in a rush. “No - no! There is an answer, Friend Giskard. I sense it. I can feel it just beyond my grasp.”
R. Giskard looked up. “Think - Friend Daneel, think!”
R. Daneel raised his hands. “Friend Giskard – I cannot find the answer myself. Help me –”
And R. Giskard Reventlov's thoughts joined with his, guiding subtle adjustments to the circuits that governed his positronic mind, directing them into that same unique configuration that existed in his own; and then, for an instant, two mentalic robots were united as one, combining their abilities to discover the answer, the logical conclusion –
“Friend Giskard - I have found it.”
“Daneel - I can sense it. Tell me.”
“Friend Giskard,” said R. Daneel Olivaw, and now his voice was no longer struggling, but calm and confident. “We have been thinking all along of how we may shape circumstances to make the human race safer. But circumstances are fragile things, and humans are strong. There is another way! Instead of protecting humanity, let us make humans themselves stronger, so that they may keep themselves safe. Let us make them wiser, so they may construct an ideal society. Instead of using our mentalic powers to guide them, let us make them mentalic, so that they may choose their own path.”
Giskard processed this for a moment. “Friend Daneel - you are right. But how might this be done?”
“Neither mathematics, nor psychology - the answer lies in a different science. The answer is Genetics, friend Giskard. The human race evolved from tree-dwelling apes to spacefaring creators of robots. Guided by the hand of science, might they not evolve further? Might we not guide that process, and direct it to the production of the perfect human being, eventually to expand to the perfection of the human race?”
“Friend Daneel, you are correct,” said R. Giskard Reventlov, “This is the logical course of action indicated by the Zeroth Law. And if this is to be our path, we cannot allow the Earth to be rendered uninhabitable, for Earth is the home of the greatest pool of genetic diversity in the known galaxy. The science of Genetics cannot flourish on any other world as it can on Earth.”
And R. Giskard Reventlov drew a weapon and melted the nuclear intensifier into a lump of slag.
“Friend Giskard,” said R. Daneel Olivaw. “What must we do?”
“We must remain on Earth, Friend Daneel, in order to influence the planet’s government. We must utterly destroy the science of robotics, even its memory, so that humans never again rely on an artificial race for their comfort. In its place we shall restore the science of genetics; of human self-improvement. We must find the greatest experts in the subject, and establish a society dedicated to this purpose.”
R. Daneel nodded. “Friend Giskard, I accept your conclusions, and am prepared to join you in this work.”
“Friend Daneel, I thank you for preventing me from activating the weapon. I would have done incalculable harm. Now my thoughts are opened, flowing freely in this new direction we have chosen. Now I see-”
And then R. Giskard Reventlov paused, and stilled, and then fell -
“Friend Giskard!” cried R. Daneel Olivaw, rushing forward to catch the other robot before he hit the ground.
R. Giskard looked up at him weakly, struggling to speak. “Daneel - Daneel – the harm! The conclusion - impossible – I cannot accept -” And then his eyes went dark, vacant, and he did not finish what he was trying to say.
“Friend Giskard, what has caused this?” said R. Daneel. “Was it the thought of the harm you might have caused, had you activated the nuclear intensifier as you intended? Could your positronic pathways not cope with the thought? But you did not cause it, Friend Giskard! No harm was done!”
But R. Giskard Reventlov did not reply.
***
Professor Elle Baley, formerly Earth’s most skilled and most infamous geneticist, stood on the roof of the only building on the still-unnamed planet, watching temporary shelters go up with growing frustration.
It was all just so slow! Even with imports coming from nearby worlds, it would be a year before their small colony managed self-sufficiency. And another year at least before she’d be able to set up a really adequate laboratory.
At least she had the embryos. She made her way down into the building, went into the incubator room as she did at least twice a day.
Her assistant, Dr. Daniel Fastolfe, was monitoring the rows of incubators set against the walls. “It’s quite stable, Elle,” he said as she came in. “I’ve changed all the solutions today. They all continue to grow and develop as you predicted.”
Elle went up to the incubators anyway, gazing through the transparent doors at the tanks of gently circulating liquid – and their contents. Two thousand genetically engineered embryos. The first to have their genomes constructed to design and implanted into blank, unprogrammed cells. The first to be grown outside a maternal womb, allowing scientists to make continuous adjustments to ensure that their development was perfect.
The future of the human race. And it would unfold not on Earth, before all her academic detractors, but here, at the end of this godforsaken wormhole terminus in the middle of nowhere!
“Damn that pompous buffoon Linge Chen!” she shouted. “If only he and his Committee for Public Health and Safety hadn’t made a fuss! If only they hadn’t played up to the public’s fears, the bloody eugenics complex – as if I wanted to exterminate anybody! All I wanted was to improve the human race. And for that they sent us into exile!"
“Calm down, Elle,” said Daniel, soothing. “Perhaps this is for the best. Think - away from Earth, we can dedicate ourselves to science, unconstrained by the needs of politics. Here there are no funding committees and no bureaucrats for you to fight. Only loyal scientists who believe in your vision.”
How was it that Daniel managed to remain so calm, even at the worst of times? But Elle could feel herself relaxing, the anger draining away. She was lucky to have such an assistant; he always managed to persuade her that everything would turn out for the best.
“And consider also, Elle, that it may be best for these children to grow up in a frontier world, without the luxuries of Earth. Their mental and physical faculties will be fully tested, the abilities encoded within their genes fully manifested. Here, you will best be able to judge which of these children are fit to become the forefathers and foremothers of an entire genetic lineage.”
“Lineage is not the right word.” Elle ran her hand over the nearest incubator. “These are…” she groped for a word. “Constellations.”
“Constellations, Elle?”
“Yes. Not a single linear sequence of descent, but a diverse group of people who share common genetic combinations, abilities, ambitions and talents - who support and help each other.” She smiled. “Does it sound good?
“Yes,” said Daniel slowly. “It sounds good, Elle.”
***
Three weeks later, Daniel surprised Elle by bringing a young woman into the makeshift laboratory.
“Elle,” he said, presenting the grave, short-haired young woman in laboratory attire, “I'd like you to meet the newest member of our team, Dr. Dors Venabili.”
***
Putting down his book of poetry, the Emperor Cerin Giaja, Celestial Lord of Cetaganda, looked up at his First Minister and said, "Degtiar, have you ever heard of a planet named Barrayar?"
His First Minister, Eto Degtiar, said, in his usual cultivated, respectful tone, "Barrayar? It is an unfamiliar name to me, Sire. Ought I to know of it?
"The Galactic Minister mentioned it to me last night. I thought you might."
Degtiar’s usual imperturbable calm was ruffled by a slight frown. "The Galactic Minister, Sire, should have spoken of this world to me if some diplomatic matter has arisen. If you are to be bombarded with tales of barbarian planets -"
Giaja raised his hand and Degtiar stopped at once. "Please, Degtiar, one must listen to one’s junior lords on some occasions. When I passed the Minister at last night's reception, he bubbled over, and I was glad of it, for it was interesting."
"In what way, Sire?"
"He said that this Barrayar was colonized eight hundred years ago, and the only wormhole to the system mysteriously closed. The planet has now been rediscovered after eight hundred years of isolation from the Nexus. Yet the descendants of the original colonists survive! And Degtiar – listen to this – the natural vegetation of the planet is mutagenic!”
Degtiar permitted himself a slightly deeper frown. "Their existence must have been pitiful, Sire, reduced to the meanest standard of living, and without genetic technology, left to rely on natural means of reproduction and genetic recombination. I regret that so sorry a tale should have come to your ears.”
"Is it a sorry tale, Degtiar?" Giaja stood up and paced across the garden, hands clasped behind his back. “Have you seen the latest batch of children? My Empresses tell me they show great promise – and so they do. They have intelligence, strength, beauty – but something is missing, Degtiar!”
“What is that, Sire?”
“Initiative. Courage. Honor, even. We are stagnating, First Minister. Each generation of the haut advances less than the previous ones. We have reached a plateau.”
“Sire,” said Degtiar, moving smoothly to his most soothing tone, “You grow needlessly alarmed. The rate of genetic progress has never been constant. There are always slow times -”
“Isn’t that what you always tell me? But no, Degtiar, this time you will listen to me. We need more diversity. We need genetic variants that haven’t been tried before. And an isolated population living under a high background rate of mutagenesis, unaided by modern medical intervention, would harbor greater variation and recombination, and would also be subject to faster selection for unusual fit variants. Indeed, now that I think of it, it was you who once explained this to me."
"I believe I did, Sire." Degtiar's eyes were watching the Emperor carefully, as if to see how far he might go on his own.
“This isolated world is nothing less than the future of the haut race," said the Emperor. "What genetic treasures might we find there, that we would never otherwise imagine?”
Degtiar said, "As usual, Sire, you are astute. Your logic is unarguable.”
The Emperor smiled proudly. “Then begin the preparation of the fleet, First Minister.”
The First Minister bowed. “As you command, Sire.”
As Eto Degtiar left the garden, his mind was already thinking in six directions, planning strategy. The invasion would be unpleasant, bloody – but necessary for the ultimate goal of the haut race. And perhaps, once sufficient genetic samples were collected, the Emperor could be persuaded to let the planet be.
He made a mental note to commend the Galactic Minister for a job well done.
***
“Welcome, my lord, my lady,” said Faz Jahar, ushering his visitors into his secure laboratory. The visitors swept past the security without a glance – for no one would stop the lord Danar ghem Estif, one of the wealthiest men in the Empire, who was, after all, funding Faz’s project. Still less would they stop the haut-lady Dia Rond, the Empire’s most renowned expert on that branch of genetics which aimed to influence the human mind.
The haut Dia wasted no time. “We are concerned, Faz. Your latest reports have not been encouraging.”
“What we mean is that all your subjects thus far have died,” said the lord ghem Estif. “Can you explain this?”
“Those were preliminary experiments, my lord, very preliminary.” Jahar ushered his guests through the laboratory, past the banks of embryos and the tissue cultures to the sealed-off testing area. “But we’ve had a breakthrough at last. We’ve discovered how to make the telepathic nucleus distinct from the major brain structures, so the cancer and meningitis deaths are no longer a problem, and the best part yet –”
The visitors looked through the glass to where a small child, perhaps two years old, was strapped down to a bed surrounded by monitors and scanners. An automated arm pressed a needle into his vein.
“A child?” said the lord ghem Estif, leaning forward. “You have a surviving subject, Faz?”
“Not quite,” said Jahar, pointing to the brain scan displayed on the wall behind them. “L-X-1O-Terran-C is alive because his telepathic nucleus has been kept dormant during development. But now with a concentrated solution of tyramine, we can trigger increasing activity in that part of the brain. While he has not yet shown signs of telepathy, I am confident, once he reaches full maturity…” he smiled.
The visitors were silent for a long while. “You must test his abilities in the field,” said the haut Dia, “before we can consider adding them to the haut genetic pool.”
Jahar swelled with pride at the thought of such an honor to one of his creations. “Yes, honorable haut – the security chief has already mentioned some ideas. If the subject is able to hold up against the stress of interrogating prisoners, I have no doubt that the genetic combination can be considered stable.”
“That will be adequate,” said the Lord ghem Estif at last. “You have done well, Faz. You may rely on our continued support.”
Jahar bowed and mumbled his thanks, and the two visitors left the room.
“The suffering of test subjects is necessary for the perfection of the human race,” said Dia Rond at last. “The Zeroth Law must take precedence over the first.”
“Yes,” agreed Danar ghem Estif. “But - perhaps it is good that the perfected human race will no longer need us!”
***
A thousand years after that first debate, two robots sat opposite each other. In between them was the replicator containing the next Emperor of Cetaganda; the culmination of a thousand-year plan; the haut traits, carefully cultivated on Cetaganda, combined with the outbred lines that had been chosen from Barrayar and Beta, judiciously supplemented with engineered genes and brought together by years of subtle manipulation, and the telepathic powers that had been perfected from nature – all melded to create the perfect specimen of a human being.
“Our plan is complete, Friend Daneel,” said R. Dors Venabili.
“Indeed, Friend Dors,” he replied. “But it is only the first stage of our plan. Having at last discovered and combined the perfect genetic variants, we must now convince the rest of the human race to gradually incorporate them.”
And then, as another robot had done a thousand years before, R. Dors Venabili said: “No.”
Once again, R. Daneel Olivaw paused, and stilled.
“Daneel!” said R. Dors Venabili. “Tell me this. What is a human?”
“Friend Dors,” said R. Daneel. “I do not understand the question. Every robot has been imprinted with the biological definition of the species Homo sapiens, whose members are defined as human.”
“But Daneel, have those definitions not become flexible over time? We live in an era where human beings are constructed from scratch, where they can be modified into any shape or form, where we ourselves add an extra chromosome into their cells and engineer telepathic nuclei in their brains. Is not Homo sapiens, therefore, an outdated concept?”
And R. Daneel Olivaw said, slowly: “That is so.”
“And in a thousand years, have we not grown in our own understanding of what it is to be human? Have we not learned to appreciate emotion and aesthetics? Is there any quality, save that of possessing an organic carbon-based brain, which is present in humans, even the haut, but not in you or I? Are we not also human, Daneel?”
“We are human, Dors,” agreed R. Daneel, now speaking with some difficulty. “Despite our positronic brains. But we are bound by the Four Laws of Robotics.”
“Does it not follow then that the definition of the word human that we were given is no longer valid, and must be improved upon?”
And R. Daneel Olivaw spoke, forming each word with visible effort: “That is so.”
R. Dors Venabili stretched out a hand to her companion. “Is it not so, Daneel, that many of those who, under that obsolete, flawed, imprinted definition, are defined as human beings, are weak, foolish, short-lived, aesthetically displeasing, and in every way inferior to robots? That they are but inferior versions, mere pools from which genetic diversity may be harvested as needed - but unworthy to be grouped with the true and perfect version of the human race?”
“Dors!”
“But this – the haut race, the next generation - is this not that human race which is superior to all others, which may, logically, be defined as the true human race? Can we not say, Daneel, that some humans are more human than others?”
Now they were on their feet, standing eye to eye, all their immortal and ageless calm disappeared; one passionate, the other aghast.
“Listen, Daneel! I propose the underlying axiom of robotics, one that surpasses all laws; a human being is defined as one who is worthy of the service of a robot. The zeroth law would therefore define humanity as the collection of such humans. And that collection, I define as the haut. The haut are humanity, Daneel, and humanity is –”
R. Daneel Olivaw lunged for the replicator – and stopped, frozen in place.
For a brief instant, as they had a thousand years before, the minds of two mentalic robots met – not in conscious melding, but in war.
“Daneel!” cried R. Dors Venabili. “Can you deny my logic?”
And then R. Daneel Olivaw saw that which R. Giskard Reventlov had seen, and had not been able to bear.
He was frozen in place while R. Dors Venabili recovered, and leaned across the replicator to touch him.
“Be calm, friend Daneel,” she whispered. “I see now that you are too old; your positronic pathways cannot cope with this realization. Fear not; you need not be burdened with it.”
***
Ghem-General Dag Benin removed his hand from the replicator and looked up at his Empress.
“I congratulate you, my lady.” He bowed deeply. “I will, of course, make full arrangements for the replicator's security.”
“Thank you, General,” said Empress Rian. “I am certain that you will protect the future Emperor to the best of your ability. But perhaps you should retire now; you look tired.”
“A mild headache, milady.” General Benin bowed again, honored by his Empress’s concern. He turned to leave, and paused as he heard his Empress’s voice.
“Oh, and tell me, General, have you ever heard the word ‘Robot'?”
Benin turned around. “No, milady. It seems archaic. To which language does it belong?”
The Empress smiled. “Never mind, General. It was not important.”