AI cannot predict the future. But companies keep trying (and failing).

While the world gushes over large language models and text-to-image tools, a far more consequential kind of AI tool has been proliferating: AI for making decisions about individuals based on predictions about their future behavior. We call this predictive optimization.Governments, banks, and employers use predictive optimization to make life-changing decisions such as whom to investigate for child maltreatment, whom to approve for a loan, and whom to hire. Companies sell predictive optimization with the promise of more accurate, fair, and efficient decisions. They claim it does away with human discretion entirely. And since predictive optimization relies entirely on existing data, it is cheap: no additional data is needed.Velogica, Upstart, and HireVue are three of the many companies selling predictive optimization tools.But do these claims hold up? Our hunch was: no. But hunches are merely the beginning of a research project. Over the last year, together with Angelina Wang and Solon Barocas, we investigated predictive optimization in-depth: We read 387 news articles, Kaggle competition descriptions, and reports to find 47 real-world applications of predictive optimization. From these 47, we chose the eight most consequential applications. We then read over 100 papers on the shortcomings of AI in making decisions about people and selected seven critiques that challenged developers’ claims of accuracy, fairness, and efficiency. Finally, we checked if these seven critiques apply to our chosen applications by reviewing past literature and giving our own arguments where necessary.The table below presents our main results. Each row in the table is one of the eight applications…AI cannot predict the future. But companies keep trying (and failing).

Leave a Reply

Your email address will not be published. Required fields are marked *