Risks of Super Intelligent AI Development

So, um,

all the, uh, AI companies are trying to build super intelligent AI.

This means AIs that are better than humans at every task.

If this goes well, there could be tremendous upside.

This could transform the economy,

it could make huge improvements in, in science and medicine,

and it could make nearly everyone's life better.

But on the current path, I think it is more likely than not

that humanity loses control to these AIs,

and it could end in human extinction.

From my experience, the companies are being extremely reckless,

given the stakes.

Part of it is the culture.

The companies run on a startup mindset, move fast,

break things, fix them later.

That works for a photo sharing app.

It does not work for building the most powerful technology ever built.