While Intel suggests that application developers will soon infuse all software with AI, and PCs must be ready for them, the workload that currently matters most is inferencing to power large language models. Anyone can run a 'good enough' chatbot on a PC – provided the PC has enough RAM and a mid-range GPU.
While Intel suggests that application developers will soon infuse all software with AI, and PCs must be ready for them, the workload that currently matters most is inferencing to power large language models . Inferencing is the process that transforms a submitted prompt into its response. Doing so needs a machine that can grind through multiple gigabytes of data – and even more matrix multiplications.
While such a compute-intensive task sounds like it could well stretch the capacities of a desktop PC (ChatGPT and its ilk run in colossal datacenters), I've discovered that a lot of the hardware we already have can do an adequate job of inferencing. Anyone can run a 'good enough' chatbot on a PC – provided the PC has enough RAM and a mid-range GPU. That means a GPU with at least 8GB of VRAM (and not one from Intel – at least at the moment, for lack of driver support). That's not too pricey – which is good, because it's table stakes. For RAM, you'd be happier with 32GB than 16GB, while 8GB simply won't cut it. And that's about it. That's all you're going to need
AI Chatbots Pcs Inferencing Language Models RAM GPU
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Study Evaluates Human-Like Behaviors of AI ChatbotsResearchers assess the human-like behaviors and personality traits of AI chatbots in a recent study published in PNAS. The study aims to determine if AI chatbots can mimic human behaviors and pass the Turing test.
Read more »
Long-Lost Microsoft OS/2 2 Discovered and Run on Modern PCsTwo experts have managed to run the long-lost Microsoft OS/2 2 on modern PCs, giving a glimpse of how different the computer world could have been if it had been released in 1990.
Read more »
Desktop GPU shipments jumped by a third – no thanks to AI PCsPrices stabilized, so buyers opened their wallets
Read more »
Xbox Series X joins PS5 in running Pentiment at 120fpsObsidian Entertainment has rolled out a 120fps Pentiment update on Xbox Series X, closing the performance rift caused by the game's release on PS5.
Read more »
Mum went from running a care home to living in one within yearsHannah Gardner was in her early 20s when she started to notice changes in her mum Susan's behaviour
Read more »
BBC bosses respond to claims long-running drama is set to be axedThe BBC has spoken.
Read more »