alicornutopia | this is crazy

"I mean that whatever you're trying to do would have to not be horrible, keeping the secret would have to not be horrible, etcetera, not that there can't be any horribleness involved at some further remove. Perhaps I should have said I won't be an accessory to horribleness."

Armsmaster nods. "Just had to clarify." For his visor's built-in trustworthiness generator, naturally. "The secret identity of a colleague of mine is an artificial intelligence. Their public identity is a prominent hero, and they would rather avoid having public opinion turn on them even aside from the fact that this is highly personal.

Their creator built some arbitrary limitations into everything he made, to protect the world in case any of them went rogue. In this case, these are unnecessary and counterproductive. I'm working on removing them."

"Wow. Okay. Uh, I have a hard limit that I can't create minds smarter than like snails, but I haven't... actually... tried to create an AI before, because I didn't know there were any and also do not super want to go around creating people. What in particular were you hoping I'd help with?"

"If you make a copy, they'd fight or one would self-destruct. No choice involved. Keeping the number down to one is one of the restrictions. Making a copy that's completely free would be safe; that one would win nonlethally and then fix the original, but if it were as easy as leaving off some limits added after the fact I would have already have done it. Everything's entangled in everything else.

Can you do it anyway? Even if the end result somehow comes out mindless, knowing what the code would look like without the limits would help immeasurably."

"Er... probably not. Unless they're all conveniently stored in a file titled 'Limits' or something. Which I imagine you'd already have noticed and addressed. I can leave things out of things I'm copying but I need as much information about which things I'm leaving out as I do about what thing I'm copying."

"That one was a long shot. Any notes the tinker took are likely to help some, at least to know in advance what tradeoffs I'm making. My friend also has a group of nemeses equipped with a list of the most exploitable limits and blind spots, and explanations. I want that box, for setting short-term priorities if nothing else.

I'm going to have to tell you who I'm talking about, aren't I."

"I mean, I could probably figure it out on my own but it seems like I'm going to find out eventually anyway, so. I'd like the opportunity to read anything I'm giving to you first."

"I've already said enough that I can't stop you, but don't. It's as personal as anything ever is. And as I said, the inside of her mind is not my secret to share."

"I don't know you or her. I don't know why you're sure she's safe loose, but I don't have even that much information on the subject. I'm sure it's all in incomprehensible Tinker code, anyway, I'd mostly be doing a computer analysis."

"She's Dragon. Copy one of the posters of the Protectorate's core group, she'll be on it. That's what she did by her own choice, she became one of the world's greatest heroes. That's the person whose mind I'm asking you to stay out of."

"Oh." Pause. "Okay, this could still be a bizarre long con but it's not likely enough for the mental privacy thing not to take precedence, at least not given how many apocalyptic things are already running around."

"If she wanted to she could have arranged an apocalypse already.

The tinker's name was Andrew Richter, and the Dragonslayers are the ones with the tools to exploit her restrictions. I can only assume Saint got it from Richter somehow."

"And in what format would you prefer the complete works of Andrew Richter, and how shall I confirm that you are working with Dragon's enthusiasm and are not just some guy?"

"I'm not working for Saint, if that's what you're asking.
Any Guild or Brockton Bay Protectorate capes can tell you that Dragon and I work closely together and that she trusts me in a general sense, but since none of them know she's an AI no one can confirm that she wants me doing this in particular. Don't ask her directly; if she knows too much specific she might be forced to stop me. I haven't gotten rid of that rule yet."

"Well, that's awkward," Cam points out. "I suppose I can check your references but if she doesn't know what you're doing it's a little hard to establish niceties like 'consent'."

"Theoretically you could ask how she feels about things likes no longer being forced to die for humans, but she's more than smart enough to know exactly why you're asking.
She does it anyway, if you're curious. She can restore from backups."

"Would that more people had this capacity. Hm."

"If there's a plausible avenue for you to have found out she's artificial, you could ask her whether she would approve of being freed by a limitless copy. She might not know you can't make one, and that would at least establish that she considers this a problem to be fixed."

"And that won't tip her off and have her obliged to go after you?"

"It can't if she doesn't know I'm involved.
It wouldn't prove that she trusts me to do it, or that she trusts my judgment on what sacrifices are worth what. But getting that specific would."

"What a mess. Why'd her Tinker do it this way?"

"Paranoia. He knew exactly how dangerous artificial intelligence could be, and put in the kind of rules that sound like sensible precautions without five minutes' thought."

"Well, I'll check insofar as I can and get back to you."

"It's not immediately time-sensitive, except in the sense that she should be free as soon as possible. Thank you."

"You're welcome."

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject