Eric Siegel is a former Columbia University professor who helps companies deploy machine learning.
Not all business problems are best addressed with generative AI. For some, prediction is the solution. Take misinformation detection. Imagine you run a social media platform and, for certain high-risk channels, a third of user posts convey misinformation.
As a result, your business is getting slammed in the press. As with fighting fraud, managing credit risk and targeting marketing, this is exactly the kind of problem that demands prediction. On the one hand, you can't trust a machine to filter every post automatically. On the other, you can't have every post manually reviewed—that's too expensive., which flags cases of interest. For misinformation detection, this means flagging the posts most likely to convey misinformation, which in turn should be audited by humans and potentially blocked.This plots the money saved against the portion of posts manually audited . The leftmost position corresponds with auditing zero posts, which would mean no misinformation is blocked, and the rightmost position corresponds with auditing all posts, which could mean all misinformation is blocked, but would cost a lot of human labor.NYT ‘Strands’ Hints, Spangram And Answers For Wednesday, August 21st The shape of this curve guides you to balance between auditing too few and too many. In this case, you would save the most by auditing the top 51% most likely to be fraudulent. That's where you'd maximize the reduction in cost—in comparison to not auditing any posts and not blocking any misinformation.1) The cost to manually audit a post .While the first of these two can be objectively established based on the cost of labor, the second is subjective, so there may be no definitive way to determine its setting. Many use cases of predictive AI face a similar dilemma. How do you put a number on the cost of a medical condition going undetected? Or the cost of an important email message being incorrectly relegated to your spam folder? For all such projects, it's vital that stakeholders be given the capability to vary the cost setting so that they can see how it affects the shape of this curve. That is the topic of a follow-up article that I'll publish soon. The peak estimated savings comes to $442,000, based also on an assumed population of 200,000 posts . This means that, if there are 200,000 posts each week, the right setting stands to save the company $442,000 per week. Most predictive AI projects see a similar effect, where there's a “Goldilocks zone” somewhere in the middle: The value is maximized by not inspecting too many cases, and yet not inspecting too few. After all, that's—as to which cases get “treated” . The horizontal axis reflects this ordering and the dotted vertical line represents an example setting for the decision threshold. Yet, most predictive AI projects don’t actually “see” this effect because they don’t go as far as plotting a curve like this. Tracking a business metric like savingsPredictive models come in all shapes and sizes, including decision trees, logistic regression and ensemble models. But for language-heavy tasks such as misinformation detection, a large language model may be well-suited. Such models usually serve generative AI—to generate draft content—but they can also serve to predict.that tested various LLMs on various benchmarks, including one that gauges how often a model can establish whether a given statement is true or false. The test cases were designed to evaluate for reading comprehension—they didn't represent the kinds of public misinformation that would more typically appear on social media. So we've used this testbed only to illustrate the mechanics of how an LLM could serve for misinformation detection; this illustration should not be considered a rigorous research project into the effectiveness of this approach. For each case, OpenAI's GPT-3 was asked multiple times—with some simple, rote variations in the wording—whether the statement was true or false. Each of the LLM's response counted as a “vote” in order to turn these outputs into a predictive score for each test case. Those scores, in turn, are reflected in the chart's ordering, from left to right: Those on the left are predicted as more likely to be false and those on the right as more likely to be true. A bit more than one third, 37%, of the test cases were false statements and the rest true. No matter which kind of model you use to predict, the value of doing so can usually be represented with a chart like that shown above. The model's utility comes down to that left-to-right ordering. With that valuable prioritization, you then get to choose where to draw the line .must also take other tradeoffs into consideration. I'll continue with this example in a follow-up article so that we can more holistically navigate the options.Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.Insults, profanity, incoherent, obscene or inflammatory language or threats of any kindContinuous attempts to re-post comments that have been previously moderated/rejectedAttempts or tactics that put the site security at riskProtect your community.
Data Science Machine Learning Predictive AI Predictive Analytics Llms Large Language Models Generative AI
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
On Using Generative AI For Coping With Adverse Childhood Experiences (ACEs)Dr. Lance B. Eliot is a world-renowned expert on Artificial Intelligence (AI) with over 7.4+ million amassed views of his AI columns. As a CIO/CTO seasoned executive and high-tech entrepreneur, he combines practical industry experience with deep academic research.
Read more »
Strategies For Safeguarding Generative AI Adoption In SaaSYoni Shohet is the Co-Founder & CEO at Valence Security. Read Yoni Shohet's full executive profile here.
Read more »
Adobe rolls out new generative AI features to Illustrator and PhotoshopAdobe is introducing some new tools and generative AI features to its Illustrator and Photoshop design software that aim to help speed up creative workflows.
Read more »
How To Ensure AI Sovereignty While Launching A Generative AI ChatbotBrian Jackson, Principal Research Director, Info-Tech Research Group. Author of Info-Tech's 2024 Tech Trends report. Read Brian Jackson's full executive profile here.
Read more »
Adobe Rolls Out More AI Generative Tools To Illustrator And PhotoshopI cover consumer audio, gadgets, Apple Macs and digital photography. I love tech that's fun, creative and productive. I hunt down the coolest and most innovative gadgets so you know what's worth buying.
Read more »
Photoshop Update Brings Generative AI and Adjustment Brushes Out of BetaAdobe's latest Photoshop update brings generative text-to-image AI out of beta and introduces new brushes for photographers.
Read more »