“Godfather of AI” Geoff Hinton, in recent public talks, explains that one of the greatest risks is not that chatbots will become super-intelligent, but that they will generate text that is super-persuasive without being intelligent, in the manner of Donald Trump or Boris Johnson. In a world where evidence and logic are not respected in public debate, Hinton imagines that systems operating without evidence or logic could become our overlords by becoming superhumanly persuasive, imitating and supplanting the worst kinds of political leader.
Why is “superhumanly persuasive” always being done for stupid stuff and not, I don’t know, getting people to drive fuel efficient cars instead of giant pickups and suvs?
Because superhuman persuasion generally only works on things people don’t have to change to accept.
Racist dogma is so persuasive to some because it lets them recontextualize their hardships as inflicted on them by {insert race here} without materially changing their life in any way and despite it being horseshit.
Ultimately it’s not superhuman persuasion: it’s extremely human. It only works when it’s playing to your existing biases, connects to the lazy, gut-based feeling method of navigating life rather than logic or reason.
But if popular enough it can be extremely dangerous: because the reason that persuasion machine is wrong will not be readily apparent until after the mistakes have been made.
Great write up. I would also include the establishment of an ‘enemy’ to blame shortcomings or adversity on. I’m not going to be able to explain as well as you do but your post had be thinking of how young, depressed, disadvantaged, etc wind up indoctrinated into racism, incels, and other such hate groups.
Because there’s no money in getting people to use less. Persuasion in capitalism is about marketing. It’s about convincing people to buy things they don’t need.
Wanna be the bigwig on your block? Have I got a product for YOU! Solar Panels! Make your house shine with newfangled tech that’ll be the envy of all your neighbors! Go solar, baby! Stick it to the electric company and make THEM pay for a change. Solar! You’ll be beaming.
Why is “superhumanly persuasive” always being done for stupid stuff and not, I don’t know, getting people to drive fuel efficient cars instead of giant pickups and suvs?
Because superhuman persuasion generally only works on things people don’t have to change to accept.
Racist dogma is so persuasive to some because it lets them recontextualize their hardships as inflicted on them by {insert race here} without materially changing their life in any way and despite it being horseshit.
Ultimately it’s not superhuman persuasion: it’s extremely human. It only works when it’s playing to your existing biases, connects to the lazy, gut-based feeling method of navigating life rather than logic or reason.
But if popular enough it can be extremely dangerous: because the reason that persuasion machine is wrong will not be readily apparent until after the mistakes have been made.
Great write up. I would also include the establishment of an ‘enemy’ to blame shortcomings or adversity on. I’m not going to be able to explain as well as you do but your post had be thinking of how young, depressed, disadvantaged, etc wind up indoctrinated into racism, incels, and other such hate groups.
Because there’s no money in getting people to use less. Persuasion in capitalism is about marketing. It’s about convincing people to buy things they don’t need.
Wanna be the bigwig on your block? Have I got a product for YOU! Solar Panels! Make your house shine with newfangled tech that’ll be the envy of all your neighbors! Go solar, baby! Stick it to the electric company and make THEM pay for a change. Solar! You’ll be beaming.
ok, I suck at faking ai chat