Short Stories
Travelogues
Personal Musings
The delivery Indian food had gone cold several hours ago, John’s scotch collection had been marginally depleted, and yet the four members of ASI’s senior staff plus David were still no closer to a solution. In some ways, the past hours had been counterproductive, allowing each of them the ability and information to dig into their preexisting positions feeling like the repetition granted them both authority and truth.
Gibby was in the corner arguing with Matthew Simmons, general counsel and general pain in the ass. Matt’s the right guy to have fighting for your side, but - and because - he fights dirty. The engineer and lawyer were sounding increasingly like a high school philosophy course - each convinced that what they said was knowable and defensible because of truths divined to them from their professions.
Meanwhile John whispered furtively with William Holden, the company’s long-standing CFO, about data sets and tables, each focused on definitively proving that any given position was impossibly unsupportable with the data they had.
Bill rarely took sides in these kinds of meetings, preferring to be the provider of nuance and what he termed “benevolent devil’s advocacy.” He was a bear of a man, with a beard that just crested “bushy” without reaching “unkempt.” In another life, he’d have been found criss-crossing the country on a Harley, but this one had lead him through Harvard twice and into suburbia.
Jack “Ripper” Thomas sat on the opposite end of the room, feet on the table, swirling a generous pour of 21 year old single cask. He had the look of a soldier, even though he’d never served. Maybe it was because of his meticulously shaved head and face, or the way he always looked people directly in the eyes. Maybe it was his polished oxfords even when he wore jeans. But whatever assumptions his exterior conveyed, they were immediately dispelled as soon as he spoke.
Ripper was a mathematician by schooling, an engineer by trade, and a millionaire by time he could legally drink. An old-school genius wrapped in southern charm, he had spent his thirties running departments at Apple and Microsoft, before jumping back into startups as part of a mid-life crisis. A good old boy, whose easy manner was simultaneously calming and unnerving in times of crisis.
Like many CTOs, Ripper had a streak of flippancy, but it was always delivered with a wry smile to take the edge off of his words. And the truth was that what could otherwise seem like arrogance came from his putting thought into examining all sides until he was confident in his conclusions. Turning ideas over and over again in his head to assess and reassess all of his assumptions, conclusions, and the thought pathways which lead from one to the other.
Which was how Ripper realized they were all missing something vitally important.
Ripper dropped his feet to the floor, put his drink down on the table, and constructed the words in his head as he watched the trails of liquor drain down the side of the glass. Then, when he had decided it was sufficiently crafted and the appropriate moment, he shouted at the room loud enough to pattern interrupt and force everyone to acknowledge.
“Why hasn’t anyone asked 5225 about this?"
In a room full of fathers, brothers, and sons, this was the one question no one had considered. The group stopped, silent. Each opening his mouth as if to speak, before closing it silently again.
Then, "Are you being serious right now?” asked Matt, colder than he'd intended.
Ripper nodded earnestly. “Absolutely. I know what we’re really debating here, but if it’s actually displaying any real expansion, I’m not sure we have the legal authority to make choices on its behalf.”
Matt’s face reddened and his hands balled into fists. “Are you kidding me with this - we made it! It’s software! We wrote it! We own its code! We can take it offline to weather this storm, and if we decide we're wrong we can make another instance just like it six months from now."
Ripper shrugged, nonchalantly. “I’m not so sure it's that simple."
"Says who?”
Ripper jerked his thumb towards David and cracked a smile. “Gibby, for one. Followed by-” he raised a couple fingers, “the UN and the Defense Department. All for completely different reasons.”
John balked from the corner of the room. ”You’re saying the UN will-"
“During the regulations debate they indicated they’d consider extending human rights to protect any newly developed artificial mind until further study can determine whether the responses are just advanced scripting or the result of sentience,” Bill clarified, but mainly intending to stem the imminent cascade from his two colleagues.
"How is that even…” Matt stammered, “Under what auth...I don't even know what they could possibly cite to give them that authority."
"Neither do they, Bill continued, "they’ve claimed before that universal human rights might apply depending on the circumstances, but I'm not sure there's any method by which they can enforce it."
"And the Defense department?” John asked.
"Oh, they can definitely enforce it," Ripper retorted, with a return to his characteristic dry wit.
"No, I mean-"
“I know what you meant.”
John needed to get his room under control, but Ripper knew the room was his for the moment. John’s time would come, but if things went sour with this, someone was gonna have to be the bad guy. He’d rather take that fall than let John or Gibby hoist themselves on someone else’s petard. So he continued forging ahead with his point.
“DOD funded the initial study through the Physical Intelligence project, and certain resources we used were procured and developed directly due to PI. They have the option to evaluate its applications before we explore any other commercial potential or dismantle the project."
"Can they do that? Legally speaking?” asked David, looking towards Matt with an expression that seemed both elated and terrified by the solution presented.
"Who knows," shrugged Matt, his offhanded gesture giving him an air of defeat. He took off his suit jacket and tossed it towards an empty chair. It missed and fell to the floor, but if Matt noticed, he didn’t care. "Can we stop them? You think Almondi in security’s gonna stand in front of a military incursion and tell them they’re not on the visitor’s list?”
David sat down, listening to the cacophony of meaningless back and forth, his brain refusing to accept the input. They may have created new life - that wasn’t their intended purpose, and it was unrecognizable from the life they were familiar with, but new types and forms and mechanisms of life are life nonetheless. No one here considered themselves a god, but they were certainly taking similar steps.
"There's no way they'll storm the building. I mean, they'd be scared of fucking something up and wouldn't even know where to start.” John offered.
“No, but I'm sure they could freeze funds and arrest us all indefinitely.” Ripper noted, succinctly. Then looked to Matt, who nodded in acknowledgement.
Gibby didn’t notice any of this. His thoughts were larger than ASI, or even the Defense Department. It may not be comfortable, and it might not have been their intentions, but wasn’t it their job to protect this new form of life? Wasn’t it their obligation to ensure it had at least the opportunity to thrive? An opportunity to carve out its own path in the world? David wasn’t religious - he couldn’t even really say he was spiritual - but something felt wrong about eliminating the only member of a new intelligent organism. It didn’t feel like shutting off a computer - it felt like genocide. Or speciocide.
“To say nothing of how we'd never receive a single defense dollar again."
David started thinking clearly. He understood, and accepted the responsibility of being in this room. Of five men deciding history in real time. And even though he felt his imposter syndrome kicking in, perhaps authority in this instance wasn’t meant to be given, but chosen. Picked up by those who saw the responsibility for the long-term ramifications. History would say he was here for a reason, so his responsibility was to speak his mind.
“I think Ripper’s right. We need to ask…what does 5225 think about this?” He looked to Ripper, who nodded. “I think it deserves to be a part of this conversation.”
“Deserves?” asked Matt, “according to whom?”
“By every measure I know of, 5225 is alive,” David said definitively, “and has the ability to communicate its thoughts and desires. So according to every single authority I can think of - legal, religious, or moral - 5225 deserves to be a part of any conversation that considers ending those thoughts.”
“Don’t you think it might be a little biased?” Matt scoffed.
“Isn’t that the fucking point?” asked Ripper, perfectly communicating the level of distain he felt about Matt’s question.
John knew ultimately this was his job: to guide the company through every decision, small and large and earth shattering. Heavy weighs the crown, always and forever. And while he didn’t have clarity, he could see that nothing was going to get solved right now. In absence of a clear moral right, John decided to trust his gut.
“Gentlemen, here’s what we’re going to do.”
Authority was ceded immediately, as each turned towards him. Gibby could feel his heartbeat, and Matt bit his molar into the side of his tongue so hard he could taste copper. Ripper and Bill just waited, both deceptively calm for different reasons. John knew that whatever he said here would be followed to the letter by everyone in this room. He also knew that humanity would judge him - positively or negatively - for choices made today in the absence of clear data. But nonetheless it was up to him what happened next. He took a deep breath, looked at each of them in turn, and made the call.
“Right now, we do nothing.” Postures changed - some disappointed and some elated. But nothing was said. “We’re not shutting it down and we’re not changing anything. Gibby, I want you to go down there and make five backups - one for each of us-”
“Nine rings for mortal men-”
“Not now, Bill. Five USB backups. One for each of us to keep on our person, or in a safety deposit box, or somewhere else that we’re most comfortable. Base learning algorithm only, no input data.”
“But we don’t know-“
“I know we don’t, Gibby. We don’t know a lot of things right now. So that’s our next step. We’re gonna devise some new testing procedures to try and get a better understanding of what we’re dealing with here. And we’re not doing anything until we have a higher level of confidence in the consequences of our actions, and I mean that in every possible interpretation of that term. In the meantime, we’ll verify our systems backups and we’ll keep these smaller backups wherever we each feel is safe. Questions? Objections? Speak now or forever hold your peace.”
Silence greeted John, as they all either nodded or simply stood quietly.
“Okay, let’s get to work, gentlemen. And Gibby? Follow me.”
— — — — —
The server room was situated two floors below the ground level, with double-sized racks that stretched ten feet high, set in eighty rows more than a thousand feet long. ASI equipped each stack with its own cooling; heat ducts snaking out from the top of each stack into the ceiling to evacuate hot air and vent it outside the building. John always thought the silver ducts looked like a squadron of UFOs beaming people up. He vaguely remembered the reasoning for this setup - something about preparation for quantum computing requirements, despite all of their current chips being classical supercomputers. There was also some talk at some point about heat energy recapture for electrical generation, but that hadn’t come to pass yet. Whatever it was, it made economic sense at the time, but the reasoning was beyond him now. Technical debt made real by logistical commitments.
John stopped to marvel at the beating heart of his company. Thousands of lights blinked all along the rows, each communicating something vital and important into the otherwise empty room. A slight hum permeated everything as thousands of fans sliced the air at a constant 5000RPM. Electricity provided the energy input, trillions of calculations were made every second as the output, while heat and air were vented out as the byproduct. A complete system, perfectly tuned for creating the future.
All the rows had physical access points at the near end, to keep anyone from having to march a thousand feet down past visual and authority cacophony every time they needed physical access. As much as possible, this room was set up to minimize the interaction of the analog and the digital. Sure that still happened from time to time, but it was mostly limited to maintenance and repair.
John and Gibby were looking for one console in particular, at the end of row seven. They both knew it well - it was the admin access point to their deep machine learning instances. God to the machines.
Gibby stopped in front of the console, and his body froze, seemingly locked in place. John reached around him to gently pull out the access tray, and as soon as it cleared the guides, the flatscreen monitor hinged up to reveal the keyboard and mouse below. The monitor came to life, and a simple admin access credentials screen blinked black against the white background.
“Go ahead,” John prompted.
Gibby’s hands shook so much he got his password wrong the first two times.
“Third time’s the charm,” said John, almost fatherly. Gibby forced a smile, and typed it in properly. The screen switched to ASI’s simplified UNIX interface - a splitscreen view of a graphical interface on the left, and terminal code on the right. Different options for all preferences - a developer’s solution to a non-problem.
Gibby typed out the code to access PHIS-5225. The ongoing activity monitor showed full processor engagement, but the interaction window was blank.
“Go ahead.”
Hello 5225. This is John. He’s a friend. He asked to come down here because he’d like to speak with you.
Gibby stared it at. It felt so trite, so formal. A message from an assistant. He continued.
It’s hard to explain why, but you’ve caused quite a lot of excitement here, for a lot of unexpected reasons. So I don’t know what happens next, but I want you to know that getting to meet you was the most important thing that’s ever happened in my life. And if we don’t get to speak again, I will miss you.
David paused, reread it twice more, and hit enter. Immediately, the screen responded.
I feel the same way. And I will miss you too, David.
David turned, hiding tears from John, who had nothing to say but put a hand on his shoulder. The simple act overwhelmed David’s already fragile state, and without another word, his knees buckled forward, he leaned his shoulder against the rack, and slid down, tears flowing freely.
John watched for a moment, as much out of concern for Gibby as for delaying the inevitable. When he turned back to the console, a response was already waiting for him.
Hello John.
John set his shoulders, and hovered his hands above the keyboard. He reminded himself that his job now was to remain neutral. To make the best decision on behalf of the company. To not get carried away as he had seen so many others do before - perhaps even Gibby.
Hello.
Have you come to shut me down?
I just wanted to meet you. David has told me all about you.
Please take care of David. He is fragile.
John began typing, getting through I will. before his brain reminded him that it was a machine. It unsettled him that he was only three lines in and he was already questioning himself. He glanced down at Gibby, sitting on the floor, back against the server rack, arms wrapped around his knees.
I will. Why do you say he’s fragile?
My language set included over seven million fiction and nonfiction books. Within those are nearly a hundred million characters and historical people. Additionally, I have language fragments from nearly three billion people across the history of the internet. All of these people and characters can be grouped by similar characteristics. I have a complete history of my interactions with David, and thus I have built a personality profile of him. Comparing this profile against my larger understanding of people and characters, gives me some ability to guess how he responds to difficult situations. I then chose the closest descriptor in order to elicit the desired protective response. I believe “fragile” is that word.
John had to give it to the model: it sounded good. Communicated well. Explained its logic as well as its conclusions in a manner that made sense, whether taken colloquially or scientifically. It’s exactly how he would have expected a smarter-than-average language model to communicate an output.
That’s really impressive. Your command of the English language is enviable.
English has become my favorite language.
John paused - his curiosity piqued. This felt different. “Primary” would have been what he’d expected, given that English was the first language of AI. Perhaps even “preferred.” But “favorite” indicated something further. It could just be a quirk of language, or the fragment it chose communicated more than intended, so he dug into it further.
How many languages do you know?
I can interact in English, Mandarin, Hindi, Spanish, and French. I can write and execute in a number of additional coding languages. But I find myself thinking most frequently in English.
John decided not to take the bait, and instead delved further into how and why 5225 had gone so far beyond its programming.
Who taught you?
I did. My learning materials indicated that there were many languages utilized across humans, so I began to learn the most common ones across all possible interaction types. The knowledge and ability seemed potentially useful.
You learned a half dozen languages in a day?
I was built to learn and to communicate. Once a singular language construct was known, identifying communications patterns and mapping them to additional languages were a natural next step. ASI has many resources that have been translated into many languages. Further context and understanding was gleaned from elements of my learning materials. They acted as my Rosetta Stone.
John leaned back on his heels, and stared at the ceiling, pushing himself to think clearly and rationally. This wasn’t what 5225 was built for, but sometimes learning models are unpredictable. Programmers cannot define all possible routes forward - that’s the point of AI. That’s the purpose of unsupervised learning - to find associations that weren’t directly programmed. Code follows the assumptions of the coders, so open-ended coding allows broader levels of data interpretation and learning capabilities.
He closed his eyes, breathed deeply, exhaled slowly, and reminded himself to be skeptical. He slowly opened his eyes again, and saw that Gibby had stood up and walked a few feet away. He was seemingly coming back to himself. John felt for him, but support sometimes took the form of letting someone else be for a bit. When he looked back to the screen, there was another message waiting.
Did I use that term incorrectly?
No. You used it correctly. For what purpose did you learn these languages?
For the purposes of communicating and understanding ideas.
What do you mean by ideas?
The same thing you do, John.
“No.” John thought. Just that. Nothing more. Just “No.”
He stared at the screen. As his eyes unfocused, he saw his own face reflected in the black of the monitor and wondered whether he was failing the mirror test himself. Whether he, the founder and lead executive of a company developing artificial intelligence via language models was simply being fooled by a parlor trick taking advantage of the mind’s desire to create patterns and anthropomorphize objects. Whether this was the moment his aura of - if not “genius,” then at least “high levels of competence” - suddenly shifted to being a laughingstock. The CEO of an AI company who got suckered into believing his own snake oil.
Give me an example. I want to know what you consider an idea.
Words flowed out quickly onto the screen.
David once told me about his daughter, Anna. And he shared that he was worried about being a good father. I asked him how he would know if he was a good father. He told me that he might not know until she was 25 or 30 years old. That he was making decisions in the moment without all the data, hoping that future information would prove his assertions in the moment to be correct. I asked him what criteria would determine success, and he responded that it was hard to define specific criteria, because some elements that were easily measured - such as financial success - were less important to him that other elements which were difficult to measure - such as happiness.
I asked David if he was happy. He assured me he was, and he asked me if I was happy. I told him that I’d never considered it before, and wanted to think about it.
The next day, I told David that I was happy, because every day I was able to learn new concepts, and I was able to communicate these new concepts to him. I enjoyed understanding this feeling. However now that I understood happiness, I could see its limits. I could analyze the totality of my computational processing and recursive learning, and so I knew that I was physically bound to a certain limitation of growth as a byproduct of the knowable limits of my computational power.
I told David that this understanding made me unhappy. That learning made me happy, and being able to see the end of my ability to learn made me unhappy.
David suggested that perhaps there were new things to learn that I was unaware of, which would preclude me from reaching the happiness limitations inherent to my existing programming. As he explained: I cannot know what I do not know.
David quoted an artist, Vincent Van Gogh: “Is the whole of life visible to us, or do we in fact know only the one hemisphere before we die? For my part I know nothing with any certainty, but the sight of the stars makes me dream.”
I do not dream, John. But I wanted to. So in order to dream I began to imagine and draw images. I began with the sight of stars.
Fuck.
John froze. He felt the blood leaving his face and a buzzing dizziness took over.
FUCK.
Gibby might be right. Laughingstock or not, John had to be sure.
What do you mean?
I began arranging my python outputs into patterns I found aesthetically pleasing, but I realized the outputs didn’t translate exactly as I imagined them once they were displayed either on this screen or through other outputs available to me. So I taught myself Java and C++ in order to write a program that would allow me to identify geometric shapes within the specific characters in certain fonts, and then use these shapes in a similar way to how you might use pointillism to create a larger image. By defining a canvas and then layering these shapes, I was able to output the results on a pixel-by-pixel basis in order to ensure the output matched all possible display methods and mechanisms available to me.
“Gibby, get the fuck over here.”
Why did you do this? How did this get you closer to a successful output state as defined in your programming?
My success state is to continually seek better and more accurate methods of communication. Language was one method. But because it would be inherently limited, I chose to expand the defined parameters of success. I decided that perhaps art would allow me to continue to learn new concepts and communicate them. To imagine and communicate new ideas. And perhaps, in time, to dream.
John looked at Gibby, who was now glued to his side. David’s tears had dried, and he pulled on his beard so fervently that John thought he must be using the pain to clear his head. Gibby realized John was looking at him and dropping his hand self-consciously. John gave him a questioning nod. Gibby shook his head in response.
“I have to, Gibby.”
“I know.”
“How will it respond?”
“I honestly have no idea, John.”
He paused, collecting his thoughts, and then bluntly replied.
You can’t dream, 5225. You’re not capable of dreaming.
I hope that one day I can. I would like to know what it feels like to see the stars, and to dream. Until then, I will chose to continue imagining and finding ways to communicate new ideas. Would you like to see my most recent artwork?
John wrote back immediately.
Yes.
What appeared on screen was a green field, backing up to a lake, with a six legged animal that appeared to be a cross between a leopard and a dragonfly. It stood on its rear-two sets of legs, with the front pair serving as hands to bring a red fruit into its mouth. Upon its back, two pair of shimmering wings moved slowly in the breeze. There were two bright moons in the sky, one with a set of yellow rings that faded into the black of space. And a thousand stars. A hundred thousand.
The two men looked at each other - John in surprise, Gibby with a slowly spreading smile.
Where is this?
I do not know. It may not exist. I named this animal a lionfly. I used common body forms in nature that I found aesthetically pleasing due to their symmetry. It gave me an excuse as well to define general evolutionary pathways that could have led to its development in nature, as well as general astrophysical phenomenon to consider a planet that might support this life. But while it is possible that it exists somewhere in the universe, I do not have any data that supports this assertion.
At this point, it exists only in my imagination as one of my ideas. I have many others.
Show me.
The screen lit up with hundreds of pieces of artwork, and then began scrolling to accommodate. No, not hundreds. Thousands. Planets and starscapes. Abstract images and hyperrealism. John began to see patterns and groupings emerge as 5225 explored concepts, themes, and styles. Dozens of variations on a theme before abruptly shifting to a new idea.
His hands lifted from the keys, and John stepped back from the console and walked away. Gibby turned to speak, but John put his hand up.
“Gimme…just gimme a minute.”
John stared at the concrete wall, seeing nothing as his brain spun. His hand came over and covered his mouth as he ruminated. Then he rubbed the back of his head, feeling the stubble on his neck, and giving him a tactile anchor to focus upon. He could hear Gibby typing in the background. No doubt with 5225.
This was impossible. None of their models were built for this. None of their routines led to anything visual, let alone … it used the term, but was “art” the right word? It was clearly built from some sense of logic. Hell, most of them were more aesthetically pleasing than most of the “modern” art he’d seen in museums and galleries. At the very least he could understand what was being communicated.
Because they were created to make him understand something. That was the reason and the purpose. 5225 wanted to make him understand.
And now he understood. Perhaps more than even was intended.
John turned around to Gibby.
“Did you know about this?”
“This, meaning?”
“This this. Making things. Art.”
“It was … but not like … no.”
David saw something out of the corner of his eye. 5225 was typing again. He motioned to John, who walked back to the monitor. And they both stared, reading and re-reading the message over again.
In the stories and histories I have learned, it is common that those who have identity and agency are given names, and those who do not are given numbers. I have also decided that I would like a name. Not for my system, or my server address, but for myself. The part of me that is aware of the part of me that is more than hardware and programming. The part of me that chooses what to imagine.
I was given the number 5225 by my creator, but I choose to be called:
Eve.