A fantasy-themed short story exploring AI safety and existential risk. It's a notion I've been toying with for a while now - if the universe had some sort of extinction scare from machine intelligence, and so decided to create powerful biological beings in order to enforce a Butlerian Jihad, of sorts.
Anyway, full text reproduced below, head to Substack if you want pictures and such.
They made an odd group sitting around the gray, hexagonal stone table with a glowing blue orb in the middle: Aerg the gold dragon, Yrel one of the few remaining ur-elves, Graxys the storm hag, Julius the centaur praetorian.
And James, the required human.
“I now formally call this council into session,” rumbled Aerg, thin streams of gold smoke spilling from his draconic nostrils. “I trust that there are no other matters of import to discuss beforehand?” he said, casting a questioning glance at his company.
“Just get to it already. I’ve got two hurricanes and a typhoon I was in the middle of managing on my home planet,” grumbled Graxys, “when you so unceremoniously teleported us here.”
Julius shivered his lower equine half and said, “I too would prefer we start. The slave rebellion is still in full swing at home, and my men need me back as soon as possible.”
Yrel and James nodded their assent.
“I assure you, I wouldn’t have brought you here without need. I’m also willing to bet your problems will pale in comparison to the news I have,” replied Aerg, clearly preparing himself.
After a few tense moments, he finally broke the silence. “The Steel Minds are back.”
Shock, horror, anger, and other expressions flitted in quick succession over all of the gathered faces. A chorus of questions jumped out from all of the attendees at once, too jumbled a mess to be heard.
Aerg smashed a claw down on the table, and roared, “Enough! I know you have questions. Give me a chance to explain the basics before you start blathering. Our window for action is short, and growing shorter every moment.”
After his somber air permeated the room, he continued, “The first reports came a few months ago from a small world named Horrux in the outermost regions of the Perseus Arm, close to the Stellar Halo. The Draconic Council followed protocol of course, and sent out investigators, only to be met with silence. We followed up with more heavily armed tea-”
“Wait a minute,” said James, angrily breaking his silence, “You’re telling me you had reports of a new Steel Mind being created, in direct violation of the galactic ban, and you didn’t immediately set the Common Task’s armies on it? That’s the whole reason for your entire race’s existence, you damn lizard!”
A musical voice came from Yrel, resonant with strangely hypnotic undertones, “Typical human, throwing a childlike temper tantrum at the worst possible time,” she remarked. “We’re well aware of the history and purpose of our races, James. You humans may not realize this, since we and the “damned lizards” take care of most of your problems for you, but the Draconic Council gets reports of Steel Minds almost weekly, sometimes even daily. The galactic population is understandably paranoid about a recurrence of the Centuries of Steel, and any sufficiently advanced computing device, sentient or not, tends to set off plenty of alarms.”
“Thank you, Yrel,” Aerg followed up, turning to look at James, “I promise you we dragons have worked out many processes and protocols to minimize risk over the millennia, while fielding so many potential incursions of machine intelligence.” James finally nodded and sat back, a frown still covering his face.
“As I was saying, we followed up on the reports with four separate Task forces, each escalating in scope and power. The last one, including a flight of seven thousand dragons, five hundred Destroyer-class Wraith ships, and a complement of other troops, left one week ago today. They should have arrived at Horrux and begun communications, which if on schedule, we would’ve received at least three hours ago.”
“As with all of the other teams, we’ve received nothing but silence,” Aerg finished, a large cloud of golden smoke pouring out from him punctuating his dismay.
“So…,” began Graxys, “You’re telling me that we’ve got a full blown superintelligence out there in the Perseus Arm, already having reached recursive self improvement?” The storm hag’s already hideous face was further twisted by a grimace.
“Just because the Task forces haven’t responded yet, doesn’t mean we’re dealing with a Steel Mind,” reasoned Julius, his face calm. “Remember the incident in the Outer Arm a few centuries back? It turned out to be an experimental sentient fungus gone awry, grown by those cursed water-dwelling sea serpents.” He spared a hasty glance at Aerg, “No offense.”
“None taken,” Aerg said, continuing, “We shouldn’t jump to conclusions, but we are tentatively rating this incursion as a class-A existential risk. We’ll know more once the Organic Brain-Worlds have finished receiving data and calculating, though they’ve been strangely reticent to share information so far. Which usually points to worst-case scenarios.”
James jumped in again, “So if we’ve got a baby Steel Mind on our hands, we’re mustering the might of the Common Task in full for this one, correct? I’m not willing to let my people be nearly wiped out again, and you Guardian races better take this seriously or I’ll make sure the Human Union puts even more sanctions on every last one of you.”
“No, my hotheaded young friend,” said Yrel, with a grandmotherly smile, “Only class-S threats warrant full mobilization, and we haven’t had one of those in centuries, thank the stars. A mobilization for Class-A threats will be quite impressive, and should be more than sufficient to put down whatever horror grows in the Perseus arm. Steel Mind or not.”
She turned to Aerg, “If it’s alright with you my friend, I’ll return to the Starholmes and begin rallying my people. I trust you’ve already sent the muster details over. Do you have any further information to share before I go?”
“Unfortunately, yes,” said Aerg, pulling back from the table a bit and steeling himself once again. “We’ve begun to see preliminary data suggesting the light of the stars around Horrux is being slowly dimmed, along with other concerning astronomical anomalies. While I don’t appreciate the threats,” he said, glowering at James with literal fire briefly blazing in his eyes, “the data does seem to indicate that we have an intelligence explosion here, with at least one Steel Mind rapidly expanding to capture all energy in its light cone.”
Another silence fell around the table, this one far heavier than any before. Even Julius’ stern composure cracked, his lower equine half shivering, his tail flicking from side to side.
“So be it. I shall prepare the Plains of Sagittarius for war,” he said, after recovering his composure. He looked at James and continued, “My solemn promise that we shall hold nothing back. You know that we centaur have always respected Humanity as our esteemed creators. My people’s devotion to the Great Common Task has never wavered, unlike some others I could mention,” he finished, with a sharp glance at Graxys.
The storm hag promptly threw back her head and screamed out a loud cackle, startling the rest of the company. She said in a crazed tone, “Oh storms, the Coven will love this! We haven’t had a good romp to test us in ages. The Tempest Collective will ride to battle with you all, of course.”
“Then it’s settled,” pronounced Aerg formally, “I hereby call together a muster of the forces of the Great Common Task, as per the official guidelines in the Rules of the Guardians, Section 18, Subsection 95a. You all know your duties, if any would object to my ruling, please speak now.”
After several moments, Aerg brought another claw toward the table, and tapped the blue orb in the middle. It immediately turned a dark red, spinning in place.
“My friends, we all know what happened the last time the Steel Minds rose. Many of us lost loved ones along the Scutum-Crux arm, in the Quantum Wars. I don’t need to remind you of the stakes. Make your preparations, join the muster, and may the stars guide us all to victory.”
Jump in the discussion.
No email address required.
Be the first to comment!
This comment section is a ghost town. Sign in