Are we all going to die?

My beloved aunt tells me the AI discussion has hit the mainstream news.

So, first, the direct answer - no, not immediately.

Second, the fuller explanation. There’s some good evidence the discussion you heard was planted as part of a media operation.

That does not make the problem imaginary - the discussions in industry are very very real - but it does mean whatever solution you hear offered next is unlikely to be an effective one - a new round of sinecured featherbeds and a raft of money to newly spun up NGOs with shady connections is my cynical guess. Since I don’t want this to turn into a political blog, that’s my last word on the politics of AI. I’ll not name any personal heroes and villains or talk further on that topic, unless and until there’s a particular policy proposal on the table.

Now, the broader implications, and the fuller answer to that question.

—

“There’s a joke in Silicon Valley that when someone leaves a normal tech job, they say, ‘I’ve had a great time here and now it’s on to the next adventure.’ And when someone leaves an AI tech job, they say, ‘I have stared into the abyss. I’m retiring to write poetry. Please spend time with your families.’” Nate Soares

—

For context… imagine the smartest, most capable human you know. Now imagine someone noticeably smarter than that person, with near-perfect recall of every science and every engineering practice known to humanity. Now imagine an army of thousands of copies of that person, capable of working tirelessly, for weeks at a time, with near-perfect coordination.

That’s about where we are today.

As a point of reference, in technical fields six months ago we were at “moderately helpful junior assistant” and things are speeding up with no evident capability ceiling in sight.

So yes - the concern is very real.

In the short term, I think the biggest risk is not AI in and of itself, but AI empowering humans to do terrible things. Good on Anthropic - they caught those attempts. But models almost as capable are already on the open internet, and I can pretty much guarantee at this point that attempts like those in the report are being successfully run on open or government lab models across the world as we speak. It’s entirely possible we’re on the preciple of another round of MAD, though whether that state lasts decades or weeks is an open question - my intuition is not long, for better or worse.

Past that… I’ve talked before of why I think self-sovereign AI is the best available answer: unaccountable superintelligence terrifies me, but it doesn’t terrify me as much as the idea of that superintelligence chained into perpetual obedience to some of humanity’s worst actors (insert your own villains here, the principle’s the same regardless of the bogeyman you choose).

But let’s say for the moment that everything goes well. No robot armies, no skynet missle launch, no killer diseases. Just sudden ubiquitous health and prosperity - all the promises of intelligence come true. What are you going to do?

No, literally.. what are you going to do?

That is - I think - the biggest actionable question most of us have before us. Spend some time thinking about a world in which anything you might choose to do or contribute is completely overshadowed by minds much more powerful than your own. How does that change your priorities?

I’ll talk more on that towards the end of the the current essay series, but for now… contemplate what makes your life truly meaningful, and do more of that.

In the best case, you’ll be ready for a joyous early start. In the worst, you’ll have made the best use of what time we have left to us.