I believed in the Star Trek future as a kid. Right now, it seems like we are heading somewhere between Wall-E future (best case) and Idiocracy future (arguably worst case).
Sobering piece. So can we work through this scenario and come out alive and thriving? All of our problems are man made. And, the root of those problems is egocentrism, our fatal flaw. You are correct: psychology has been no help and I would argue has really only made the problem worse. If you look at why and how modern science enabled so many advancements it was because it was self critical. It challenged what we thought we knew and forced knowing to rigorous tests. Today, however, we have “my lived experience”, and “my truth” that trump any attempts of humble learning. Can we turn this around? There are some signs. We are , I hope, becoming more broadly aware of the destruction from Progressivism, although this egocentrism has been with humans for centuries and has raptured the younger generation. To move forward and survive, we need to address and confront this fundamental flaw.
I have a little more faith in our potential, as somebody who's done a few startups. A startup done right is a group of talented and high performing people all working together towards a goal that can positively impact millions to billions of people.
I think the key to my optimism is the fact that progress on pretty much any front is driven by outliers. A startup is a team of outliers working together and creating something new, condensing fact and tangibility from the vapor of nuance and possibility. A lot of science, back when science was actually progressing, was similarly driven by outliers. Your Einstein's and Feynmans and Von Neumanns and Shannons, and further back, your Newtons and Leibniz's.
As long as the rest of us don't mess up so bad we totally prevent outliers from doing anything, we should still have progress. And sure, some places like China and the EU are making a good go at making that impossible, but the world's a big place. Even if the US screws up, there's your Prosperas and smaller units of (theoretical) organization like clades or phyles that still have some hope if enough outliers migrate there to coordinate their work.
On the "fostering outliers" side, we have some pretty exciting advances in embryo selection, and hopefully soon gengineering, that can foster the creation of more of them. Mass deployment of tutoring-capable and cofounder-advice-capable AI's like G4o and soon G5 should also allow us to access more outliers in the developing world, which is where the bulk of humanity and human population growth are right now and for the next 50-100 years at least.
And of course, there's the possibility of getting a Von-Neumann-or-better level AI at some point in the pretty near future. In which case, you can arbitrarily spin up clusters of 10-10k Von Neumann's to all coordinate on solving your problem and driving progress, and as long as that doesn't destroy everyone, I feel like the capability for progress there is pretty high too.
Mediocrity is DEFINITELY our thing in the aggregate - probably 80% of us are pointless and not really contributing to the future human light cone in any meaningful way. But progress isn't driven by average people, it's driven by outliers. And there's enough trends in that space that there's realistic hope that we'll keep having enough outliers to keep progress engines going.
So I would suggest that if this framework is useful, to advocate for and adopt "outlier promoting" technologies and social practices wherever you can. Be pro gengineering, be anti regulation, be pro weird-experimental-political-and-organization structures, be pro-elite, be pro anything that can help create more and better outliers and allow them to band together to create things together.
Thanks for this. It’s a good case for the optimistic side. I’m generally on board with “the pro engineering anti regulation pro weird experimental political and organization structures” thing. I hope more people get on board with it, I hope those outliers keep doing their thing, and I hope that’s enough for us to keep making progress and avoid backsliding.
I think the problem here is that logic is bullshit, it has it's uses but every great leap forward threw logic out of the window from a great height. In retrospect, of course, we told ourselves it was logical becuase it soothes our brain to think so. Logic is mostly our friend but can be the worst enemy we have. To use logic, the best of humanity always wins out and always will, any setback to this rule is temporary. The world is beautiful so make the most of another probably bullshit expression, have faith in the future. PS. Love this blog.
Fascinating! I totally buy your mediocre hypothesis, and I agree it seems like the most plausible outcome. I have hope for some other directions we could go, but those would be best served in a post of their own.
To be fair, stagnation scenarios are considered by longtermists. Mediocrity (in your sense) would prevent us from achieving humanity’s potential and that’s something both Ord and MacAskill discuss in their respective books. They just don’t count that as an x-risk because, by definition, it doesn’t threaten the very existence of civilization. But they would largely agree with you.
On a more pedantic note, it’s virtually impossible to preserve civilization without mediocrity, if we assume that the distribution of talents, skills, wellbeing, etc., is likely to be a normal distribution across most possible scenarios. Most people are, by definition, mediocre. That is literally what mediocrity means.
Thanks, good points. Didn’t mean to imply that Ord and MacAskil would necessarily disagree (maybe I framed it too oppositionally), but I do think there is insufficient work in EA fleshing out this idea of mediocrity, bringing all the different pieces together, thinking of it as a potential solution to the Fermi paradox, and comparing it to other x-risks in terms of importance. Hopefully this mediocre piece will lead the way. :)
Just when we thought we've plumbed the depths, we go deeper. Another fearless post - awesome stuff. David, one suggestion. In your statement "We’re tribal, factional, nepotistic, moralistic, short-sighted, close-minded, superstitious, self-deluded, and surreptitiously selfish," you left out "at the population level, dumber than a bag of hammers." The average IQ is, of course, 100, and that's far too low to appreciate or avert our impending Malthusian collapse. All the scientists know we are absolutely fucked inside 50 years. A much smaller, far more hardscrabble population will likely survive, and hopefully they will be permanently wiser for it.
This prognostication for humanity sounds right. Our DNA was constructed to win battles and to compete for food and mates. Generation after generation, 10000 times over, only those babies endowed with lucky DNA that coded for competitive victory could survive long enough to mate and pass their genes to the next generation, making each more competitive than the last. During 20,000 years, we Homo sapiens unwittingly synthesized a lethal genome. Our lab was Earth; our chief scientist Darwin.
Have you read David Graeber? I also liked Rutger Bregman's 'Humankind: A hopeful History' I find that both of these scholars help to overturn this common perception that humans are basically selfish and that the worst of humanity will inevitably conquer all. Some of the new breed of economists like Kate Raworth and Mariana Mazzucato are also providing some useful frameworks for how to re-shape our societies to the benefit of all and the planet. We should never have allowed our societies to become so vulnerable to the whims of some truly monstrous individuals, but we still outnumber them. It won't be easy but we must keep fighting for better. Nihilism only helps them win.
You clearly haven't read the Selfish Gene (as much about altruism as selfishness) and need to read Enlightenment Now by Pinker. There are some negative trends and reversals of late is some areas but the fact that negative clickbait is poisoning our perspectives, does not mean the reality we live in isn't getting better. In almost all areas from economics to violence, to health, macro trends are overwhelmingly positive.
I’ve read both. I’ve read Dawkins multiple times. I agree many things have gotten better over time. But even Pinker doesn’t say with confidence that these trends will continue. Was it a lucky run? Are we running out of luck? I think there’s a case to be made that we are (there’s certainly plenty of suggestive evidence if you click on the links), and that was the point of this post. Not saying we’re definitely doomed; just saying it’s a risk, at the very least on par with nukes, AI, and bioweapons (I personally think it’s a much larger risk than those).
Very much agree that the trends are not inevitable and more reversals are possible. Politically and culturally we seem to be going backward and this is very disturbing. And given rising threats from AI, nukes and other civilization destabilizing tech we are in a more fragile place than ever. Nonetheless we need to maintain hope and perspective and not slip into nihilism (not saying you are) in order to fight the good fight.
Hey link your local indie bookstore to order those [searches memory] author -deprecated 30 y.o. books? And your take from both of those is that $8 Starbucks (to pick a mean work) are cannily worthwhile?
Enlightenment Now was published in 2018 and his follow up book Rationality in 2021 demonstrated that the trends were continuing. The Selfish Gene was published in 1976 but its relevance is only greater today. Dawkins does not argue that evolution converges on selfishness or some negative quality but rather marvels at how altruism emerges in the individual from otherwise selfish gene strategies.
Oh and coffee is cheaper than ever. Try Tim Hortons or McDonalds or god forbid brew at home for a pennies a cup.
Life is just getting better and better but doomers are gunna doom and haters are gunna hate. It's the way it works.
More like the orbital link railway reliably discovering component risks that land in the high damage / large zonality quadrant. And discovering them kind of late! Or Zeno developing their paradoxes under constant wrestling club challenge with no timeout for being at the bone-setter's.
I believed in the Star Trek future as a kid. Right now, it seems like we are heading somewhere between Wall-E future (best case) and Idiocracy future (arguably worst case).
Sobering piece. So can we work through this scenario and come out alive and thriving? All of our problems are man made. And, the root of those problems is egocentrism, our fatal flaw. You are correct: psychology has been no help and I would argue has really only made the problem worse. If you look at why and how modern science enabled so many advancements it was because it was self critical. It challenged what we thought we knew and forced knowing to rigorous tests. Today, however, we have “my lived experience”, and “my truth” that trump any attempts of humble learning. Can we turn this around? There are some signs. We are , I hope, becoming more broadly aware of the destruction from Progressivism, although this egocentrism has been with humans for centuries and has raptured the younger generation. To move forward and survive, we need to address and confront this fundamental flaw.
“Everybody who is incapable of learning has taken to teaching.” -Oscar Wilde
I have a little more faith in our potential, as somebody who's done a few startups. A startup done right is a group of talented and high performing people all working together towards a goal that can positively impact millions to billions of people.
I think the key to my optimism is the fact that progress on pretty much any front is driven by outliers. A startup is a team of outliers working together and creating something new, condensing fact and tangibility from the vapor of nuance and possibility. A lot of science, back when science was actually progressing, was similarly driven by outliers. Your Einstein's and Feynmans and Von Neumanns and Shannons, and further back, your Newtons and Leibniz's.
As long as the rest of us don't mess up so bad we totally prevent outliers from doing anything, we should still have progress. And sure, some places like China and the EU are making a good go at making that impossible, but the world's a big place. Even if the US screws up, there's your Prosperas and smaller units of (theoretical) organization like clades or phyles that still have some hope if enough outliers migrate there to coordinate their work.
On the "fostering outliers" side, we have some pretty exciting advances in embryo selection, and hopefully soon gengineering, that can foster the creation of more of them. Mass deployment of tutoring-capable and cofounder-advice-capable AI's like G4o and soon G5 should also allow us to access more outliers in the developing world, which is where the bulk of humanity and human population growth are right now and for the next 50-100 years at least.
And of course, there's the possibility of getting a Von-Neumann-or-better level AI at some point in the pretty near future. In which case, you can arbitrarily spin up clusters of 10-10k Von Neumann's to all coordinate on solving your problem and driving progress, and as long as that doesn't destroy everyone, I feel like the capability for progress there is pretty high too.
Mediocrity is DEFINITELY our thing in the aggregate - probably 80% of us are pointless and not really contributing to the future human light cone in any meaningful way. But progress isn't driven by average people, it's driven by outliers. And there's enough trends in that space that there's realistic hope that we'll keep having enough outliers to keep progress engines going.
So I would suggest that if this framework is useful, to advocate for and adopt "outlier promoting" technologies and social practices wherever you can. Be pro gengineering, be anti regulation, be pro weird-experimental-political-and-organization structures, be pro-elite, be pro anything that can help create more and better outliers and allow them to band together to create things together.
Thanks for this. It’s a good case for the optimistic side. I’m generally on board with “the pro engineering anti regulation pro weird experimental political and organization structures” thing. I hope more people get on board with it, I hope those outliers keep doing their thing, and I hope that’s enough for us to keep making progress and avoid backsliding.
I think the problem here is that logic is bullshit, it has it's uses but every great leap forward threw logic out of the window from a great height. In retrospect, of course, we told ourselves it was logical becuase it soothes our brain to think so. Logic is mostly our friend but can be the worst enemy we have. To use logic, the best of humanity always wins out and always will, any setback to this rule is temporary. The world is beautiful so make the most of another probably bullshit expression, have faith in the future. PS. Love this blog.
Thank you for writing -- fascinating. Some months ago I wrote something very similar but framing with "complacency" instead of "mediocrity": https://inexactscience.substack.com/p/what-if-the-great-filter-is-complacency
Fascinating! I totally buy your mediocre hypothesis, and I agree it seems like the most plausible outcome. I have hope for some other directions we could go, but those would be best served in a post of their own.
To be fair, stagnation scenarios are considered by longtermists. Mediocrity (in your sense) would prevent us from achieving humanity’s potential and that’s something both Ord and MacAskill discuss in their respective books. They just don’t count that as an x-risk because, by definition, it doesn’t threaten the very existence of civilization. But they would largely agree with you.
On a more pedantic note, it’s virtually impossible to preserve civilization without mediocrity, if we assume that the distribution of talents, skills, wellbeing, etc., is likely to be a normal distribution across most possible scenarios. Most people are, by definition, mediocre. That is literally what mediocrity means.
Thanks, good points. Didn’t mean to imply that Ord and MacAskil would necessarily disagree (maybe I framed it too oppositionally), but I do think there is insufficient work in EA fleshing out this idea of mediocrity, bringing all the different pieces together, thinking of it as a potential solution to the Fermi paradox, and comparing it to other x-risks in terms of importance. Hopefully this mediocre piece will lead the way. :)
Not a mediocre piece at all!
On fire, as ususal. Well done, David!
Just when we thought we've plumbed the depths, we go deeper. Another fearless post - awesome stuff. David, one suggestion. In your statement "We’re tribal, factional, nepotistic, moralistic, short-sighted, close-minded, superstitious, self-deluded, and surreptitiously selfish," you left out "at the population level, dumber than a bag of hammers." The average IQ is, of course, 100, and that's far too low to appreciate or avert our impending Malthusian collapse. All the scientists know we are absolutely fucked inside 50 years. A much smaller, far more hardscrabble population will likely survive, and hopefully they will be permanently wiser for it.
This prognostication for humanity sounds right. Our DNA was constructed to win battles and to compete for food and mates. Generation after generation, 10000 times over, only those babies endowed with lucky DNA that coded for competitive victory could survive long enough to mate and pass their genes to the next generation, making each more competitive than the last. During 20,000 years, we Homo sapiens unwittingly synthesized a lethal genome. Our lab was Earth; our chief scientist Darwin.
Thank you for this piece David. I think you will find this one well worth your time: https://www.elidourado.com/p/collapse. Eli has the same concern.
Thanks, Joel. This looks like a very interesting read.
Have you read David Graeber? I also liked Rutger Bregman's 'Humankind: A hopeful History' I find that both of these scholars help to overturn this common perception that humans are basically selfish and that the worst of humanity will inevitably conquer all. Some of the new breed of economists like Kate Raworth and Mariana Mazzucato are also providing some useful frameworks for how to re-shape our societies to the benefit of all and the planet. We should never have allowed our societies to become so vulnerable to the whims of some truly monstrous individuals, but we still outnumber them. It won't be easy but we must keep fighting for better. Nihilism only helps them win.
I think you're giving outlier as-builts and other labor short shrift.
You clearly haven't read the Selfish Gene (as much about altruism as selfishness) and need to read Enlightenment Now by Pinker. There are some negative trends and reversals of late is some areas but the fact that negative clickbait is poisoning our perspectives, does not mean the reality we live in isn't getting better. In almost all areas from economics to violence, to health, macro trends are overwhelmingly positive.
I’ve read both. I’ve read Dawkins multiple times. I agree many things have gotten better over time. But even Pinker doesn’t say with confidence that these trends will continue. Was it a lucky run? Are we running out of luck? I think there’s a case to be made that we are (there’s certainly plenty of suggestive evidence if you click on the links), and that was the point of this post. Not saying we’re definitely doomed; just saying it’s a risk, at the very least on par with nukes, AI, and bioweapons (I personally think it’s a much larger risk than those).
Very much agree that the trends are not inevitable and more reversals are possible. Politically and culturally we seem to be going backward and this is very disturbing. And given rising threats from AI, nukes and other civilization destabilizing tech we are in a more fragile place than ever. Nonetheless we need to maintain hope and perspective and not slip into nihilism (not saying you are) in order to fight the good fight.
Hey link your local indie bookstore to order those [searches memory] author -deprecated 30 y.o. books? And your take from both of those is that $8 Starbucks (to pick a mean work) are cannily worthwhile?
Enlightenment Now was published in 2018 and his follow up book Rationality in 2021 demonstrated that the trends were continuing. The Selfish Gene was published in 1976 but its relevance is only greater today. Dawkins does not argue that evolution converges on selfishness or some negative quality but rather marvels at how altruism emerges in the individual from otherwise selfish gene strategies.
Oh and coffee is cheaper than ever. Try Tim Hortons or McDonalds or god forbid brew at home for a pennies a cup.
Life is just getting better and better but doomers are gunna doom and haters are gunna hate. It's the way it works.
Wouldn't mediocrity merely imply that it would take a LONG time?
More like the orbital link railway reliably discovering component risks that land in the high damage / large zonality quadrant. And discovering them kind of late! Or Zeno developing their paradoxes under constant wrestling club challenge with no timeout for being at the bone-setter's.
Sounds like they still got there, haha
Nice essay....but what you're really saying is that you are a pessimist. So am I.