Elon Musk and former OpenAI chief scientist Ilya Sutskever claim that AI companies have run out of real-world data to train generative models. Both suggest that the internet's data pool has been exhausted, leading to challenges for new AI models like Orion and Gemini. Musk proposes synthetic data, generated by AI itself, as a solution. However, experts warn that relying solely on synthetic data could limit AI's functionality due to inherent biases in the training material.
Elon Musk and former OpenAI chief scientist Ilya Sutskever say that AI companies have run out of real-world data to train generative models on. “We’ve now exhausted basically the cumulative sum of human knowledge … in AI training,” Musk tells Stagwell chairman Mark Penn in an X livestream yesterday, Musk’s comments came just a few days after Sutskever, who helped build ChatGPT, told the annual Neurips event that “we have achieved peak data and there’ll be no more.
” If true, it means that all of the available data on the internet has already been used up to train AI models.back in November when it came to light that OpenAI was struggling with its new model, Orion, which is allegedly not hitting internal expectations. Similarly, Google’s newest iteration of Gemini is not much better than the previous one. While Anthropic has also delayed the release of its Claude model. One of the reasons cited is that “it’s become increasingly difficult to find new, untapped sources of high-quality, human-made training data that can be used to build more advanced AI systems.”Musk suggested that the way for AI companies to plug this gap is synthetic data, i.e. the content that generative AI models themselves produce. “The only way to supplement is with synthetic data, where the AI creates ,” Musk says. “With synthetic data … will sort of grade itself and go through this process of self-learning.”“If a species inbreeds with their own offspring and doesn’t diversify their gene pool, it can lead to a collapse of the species,” says Hany Farid, a computer scientist at the University of California, Berkeley.that Microsoft, Meta, OpenAI, and Anthropic are all using synthetic data to train AI models with. While this method has obvious benefits such as cost-cutting, the model’s functionality could be compromised because of inherent limitations within the training data.OpenAI Again Refuses to Say if It Used Your Content to Train Sora
Artificial Intelligence AI Training Data Limitations Synthetic Data Openai Chatgpt
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Companies That Don’t Adapt To Climate Change Could See Earnings HitIn this week's Current Climate newsletter, companies that fail to adapt to climate change could lose 7% of annual earnings by 2035; factory-built apartment complexes that may have climate advantages; and potential clean energy policy changes under...
Read more »
From Nike to Intel, CEO departures at U.S. companies hit a record this yearCEO exits at U.S. public companies were a record in 2024 as they faced competitive and strategic challenges.
Read more »
From Nike to Intel, CEO departures at U.S. companies hit a record this yearRetired, ousted or poached, CEOs headed for the exits this year.
Read more »
Elon Musk Defends H-1B Visas, Citing Importance for Tech CompaniesTesla and SpaceX CEO Elon Musk defended the H-1B visa program, arguing that it is essential for tech companies to attract highly skilled workers. Musk's statement comes amid ongoing debate about the program, with critics alleging that it exploits foreign workers and displaces Americans. Supporters, including Musk, maintain that H-1B visas are crucial for innovation and competitiveness in the global tech market.
Read more »
Inflation and Changing Trends Hit Companies HardThis year was particularly challenging for many well-known companies, with soaring inflation leading consumers to cut back on discretionary spending. Numerous businesses have faced financial difficulties, with some filing for bankruptcy and others shedding jobs. The retail sector has been particularly hard hit, as the post-pandemic surge in spending has waned.
Read more »
Manhattan Congestion Charge to Hit Drivers, Delivery Companies, and TaxisNew York City's new congestion pricing plan will impose significant fees on drivers entering Manhattan below 60th Street, starting January 5th. Car drivers will face charges ranging from $9 to $27, while trucks and buses will incur even higher tolls. Delivery companies and taxis are expected to be among the hardest hit, with delivery costs and taxi fares set to increase. The plan aims to reduce traffic congestion but has been criticized for placing a further burden on New Yorkers already struggling with high living costs.
Read more »