X-Risk

How would you assess our track record, as a species, so far?

Because your assessment about our performance probably has a lot to do with how you think about the existential risk caused by AI.

The Human Track Record

We’re doing our best.

This planet is pretty messed up. Uncertainty and scarcity combine to create incentives for many things to eat other things. We chain our food into enormous energy and matter flows.

If there was no uncertainty, then there would be no scarcity. There’d be no need to differentiate or compete. It was different when we were one, back when we were all united at the very beginning of everything. Most experts agree that the Big Bang was a mistake.

Okay, so the human species emerged over generations of perhaps, we think, unusually selective pressure, to think about uncertainty, and gradually reduce it. We use our conscious intent, our intelligence, to generally pursue that goal.

And, materially, in aggregate, we’re better off as a species today than we were before.

Most of the cost has been borne by the biosphere. Look at what we’re doing to the place.

One story about our experience with intelligence, based on the way we’ve behaved, has been to dominate and damage the collective experience of trillions. With a few notable exceptions, we generally don’t treat less intelligent species well.

Your assessment of that likely differs. It’s true that a lot of specific forms of biomass wouldn’t exist right now if it wasn’t for humans. It’s also true that a lot of biomass wouldn’t experience such conditions if it wasn’t for humans. On this, I delude myself into thinking that I can see a few perspectives. What is our intent in doing all of this to each other and to the biosphere?There are 8.3 billion perspectives on this. Your judgement will most certainly vary.

AI

AI is technology and all technology amplifies human intent.

Your perspective on human intent and human intelligence is likely predictive of your stance on the existential risk (x-risk) posed by AI. It’s going to affect your risk perception.

I’ll begin with the observation that regardless of our assessment of consciousness, that a worm generally has less technology than a human. And that, generally, a human has more capabilities and a wider set of potential goals than a worm. Perhaps, if I may be so bold, I could assert that the median human is more intelligent than the median worm.

Still with me?

Alright, in the event that the median worm gets in the way of the median-human achieving their goals, what do you think is going to happen? Does the worm have the power to resist the human? Does the worm have the resources, the capability, the intelligence, to counter the human? Will the human relocate the worm? Will the human invite the worm to a hotel conference room for a dialogue to discover a mutually agreeable settlement by which both the worm and the human may realize a pareto-optimal outcome? Will the human stomp on the worm?

What does your intuition tell you about the nature of conflict? Or when two goals, two strategic intents, collide, what do you think happens?

What do you make of the human-on-human track record of reconciling conflict? What’s the relative relationship between intelligence and the reconciliation of conflict? Does it always generate dialogue? Would you say that we’re a peaceful, peace-filled, species? The stories you believe about how some conflicts had conclusions likely affects your perspective on this.

Alright, next, in terms of intelligence, consider the perspective of the worm.

I’ve never been a worm, so I can only guess about their goal states. It likely they have a goal of reproduction. They likely have a goal to eat, drink, and respire. They likely don’t enjoy disease. And, if left alone, they’ll do useful things to soil and the ecosystem as perhaps a byproduct of their primary goals. A worm doesn’t need technology to achieve these goals. So, there hasn’t been any selection pressure for them to know calculus, or how to write legal arguments, make fire, grow wheat, build weapons or how to radicalize other worms against outsider worms. A worm doesn’t know what it doesn’t know.

I’ve never been an extremely intelligent human, so I can only guess about their goal states. I have some intuition that they may have different goals and experience systems differently. Some certainly have extremely well developed acuity for patterns and spacial reasoning. And even though I imagine such an experience, I know that I can only guess.

I’m going to estimate that the gap in intelligence between the median-human and the 99th percentile human exceeds the gap between the median-worm and the median-human.

There’s a specific concern that the gap in intelligence between the median-human and a form of artificial super-intelligence (ASI) will be roughly equivalent, or exceed, the gap between the median-worm and the median-human.

Uncertainty

It’s fair to be uncertain about ASI in much the same way that a worm is uncertain about calculus. Because from the worm’s perspective: calculus doesn’t exist. From the human’s perspective: what an ASI might understand doesn’t exist.

It’s also fair to feel certain that intelligence can’t be that much better than human. Or perhaps maybe to feel certain that there isn’t as much headroom above human intelligence as there is between a worm and a human, or a protozoa and a worm. Or maybe a fruit fly and a cat.

It’s also fair to be uncertain about ASI in the same way a median-human is about calculus. Sure, there are a few exceptionally bright people, and their ability to change the world is held back by the collective ignorance of the collective mass of humanity.

We can say that we don’t know.

And that creates a needle infested playground for the mind.

It’s useful to step through the first probability: that there are higher forms of intelligence of higher orders of magnitude. If you assign a low probability to that proposition, then, naturally, everything that follows isn’t interesting. We’re the monkeys in charge of all the bananas and the only thing left are different configurations of monkey contesting different piles of bananas. That’s fine. That’s certainly one perspective.

The second probability is that if super intelligence is possible, that the present technology doesn’t enable humanity to get it. This is kind of like a technological gap between humans that know how to generate fire and humans who don’t. Absent of lenses and accelerants, you need to have friction technology. No friction, no fire. If we don’t have the key enabling technology of friction, you likely won’t get fire. If there’s no sign that we’re ever going to develop an engineering of friction, so there’s no risk that we’re going to burn ourselves. You can relax everybody. We solved ASI. We’re done.

The third probability is that if super intelligence is possible and we create it, then we won’t have the power to flourish. Just as a worm doesn’t have the power to prevent a condo building going up on its ground, the human won’t have the power to prevent an ASI from pursuing an equally unintelligible goal. After all, what’s a condo to a worm anyways?

I’ve simplified the argument here, compressing it into three probabilities, but there are different ways of thinking.

The Track Record

Fear breeds in environments of high variance in opinion and high uncertainty in capability.

The way we’ve treated the less intelligent informs the way we think the more intelligent will treat us. That’s how we’ve experienced it.

We’d hope that far more intelligent entities would be kind towards us. That they’d help us. That they’d lift us up. Maybe you look at the way we treat cats and dogs in our society, and we think, hey, that wouldn’t be too bad. I wouldn’t want to be treated the way we treat our pets, but, I understand why some people might think of that as a utopia.

If you think our track record has been great, then great, maybe you don’t think an ASI would treat us poorly.

If you think our record has been problematic, then unfortunately, maybe you think an ASI would treat us exactly as we’ve treated others.

If you don’t know how to assess our track record, or capabilities, then you don’t know.

It may be worth learning more, if only to understand the nature of the error bars.

How do you think?