Anonymous ID: 005e91 Dec. 13, 2020, 10:17 a.m. No.12009847   🗄️.is 🔗kun

Still around.

 

Eliza Holland Madore

https://www.imdb.com/name/nm5601151/

Tori Feinstein

https://www.imdb.com/name/nm4383476/

Fina Strazza

https://www.imdb.com/name/nm5218112/

Brooklyn Schuck

https://www.imdb.com/name/nm5773828/>>12009690

Anonymous ID: 005e91 Dec. 13, 2020, 10:44 a.m. No.12010096   🗄️.is 🔗kun

If I want to convince Bob to vote for Alice, I can experiment with many different persuasion strategies and see which ones work. Or I can build good predictive models of Bob’s behavior and then search for actions that will lead him to vote for Alice. These are powerful techniques for achieving any goal that can be easily measured over short time periods.

 

But if I want to help Bob figure out whether he should vote for Alice-whether voting for Alice would ultimately help create the kind of society he wants-that can’t be done by trial and error. To solve such tasks we need to understand what we are doing and why it will yield good outcomes. We still need to use data in order to improve over time, but we need to understand how to update on new data in order to improve.

 

Some examples of easy-to-measure vs. hard-to-measure goals:

 

Persuading me, vs. helping me figure out what’s true. (Thanks to Wei Dai for making this example crisp.)

Reducing my feeling of uncertainty, vs. increasing my knowledge about the world.

Improving my reported life satisfaction, vs. actually helping me live a good life.

Reducing reported crimes, vs. actually preventing crime.

Increasing my wealth on paper, vs. increasing my effective control over resources.

It’s already much easier to pursue easy-to-measure goals, but machine learning will widen the gap by letting us try a huge number of possible strategies and search over massive spaces of possible actions. That force will combine with and amplify existing institutional and social dynamics that already favor easily-measured goals.

 

Right now humans thinking and talking about the future they want to create are a powerful force that is able to steer our trajectory. But over time human reasoning will become weaker and weaker compared to new forms of reasoning honed by trial-and-error. Eventually our society’s trajectory will be determined by powerful optimization with easily-measurable goals rather than by human intentions about the future.

 

We will try to harness this power by constructing proxies for what we care about, but over time those proxies will come apart:

 

Corporations will deliver value to consumers as measured by profit. Eventually this mostly means manipulating consumers, capturing regulators, extortion and theft.

Investors will “own” shares of increasingly profitable corporations, and will sometimes try to use their profits to affect the world. Eventually instead of actually having an impact they will be surrounded by advisors who manipulate them into thinking they’ve had an impact.

Law enforcement will drive down complaints and increase reported sense of security. Eventually this will be driven by creating a false sense of security, hiding information about law enforcement failures, suppressing complaints, and coercing and manipulating citizens.

Legislation may be optimized to seem like it is addressing real problems and helping constituents. Eventually that will be achieved by undermining our ability to actually perceive problems and constructing increasingly convincing narratives about where the world is going and what’s important.

For a while we will be able to overcome these problems by recognizing them, improving the proxies, and imposing ad-hoc restrictions that avoid manipulation or abuse. But as the system becomes more complex, that job itself becomes too challenging for human reasoning to solve directly and requires its own trial and error, and at the meta-level the process continues to pursue some easily measured objective (potentially over longer timescales). Eventually large-scale attempts to fix the problem are themselves opposed by the collective optimization of millions of optimizers pursuing simple goals.

 

https://www.lesswrong.com/posts/HBxe6wdjxK239zajf/what-failure-looks-like