Future of AI entails 1-bit large language models (LLMs) since generative AI runs faster, slimmer, and can work on smartphones and edge devices. Here's the inside scoop.
In today’s column, I explore the exciting and rapidly advancing realm of faster and slimmer generative AI that is being devised via the latest advances in so-called 1-bit large language models . No worries if you don’t know what those are. I’ll be walking you step by step through what these emerging 1-bit LLMs are all about.
The loftiest desire would be to get things down to 1-bit. You can’t get much better than that . Getting 1-bit is the dream goal.Assume that you are using a generative AI app and opt to enter a prompt that says just one word, let’s go with the word “Dream.” You have already told the AI that it should respond with just one word in return, preferably a word that would normally follow the word that you’ve entered.
That’s an easy calculation, but it does require that we make use of something akin to a floating-point representation. Rather than using the number 0.8592, suppose we decided to round the number to either 0 or 1. If the number is closest to 0, we will round down to zero. If the number is closer to 1, we will round up to the number 1. It is apparent that the 0.8592 would be rounded up to the value of 1.You can directly see that the multiplication by 1 is going to be a lot less time-consuming.
Yes, you are losing information by lumping things into one of two buckets. No doubt about that. The question is whether this makes a material difference or not. It all depends. Other factors come into play. Sometimes you can do this without much material loss, while sometimes it is so bad that you have to regrettably abandon the 1-bit approach.
But you can potentially arrange things so that you sometimes use 1 bit and sometimes use 2 bits, which might average over thousands or millions of values to end up using approximately 1.5 bits. You can’t really have half a bit per se, and this is just saying that since you have a mixture of 1-bits and 2-bits, the average of how many bits are used altogether comes out in a fractional calculated way.
“In this work, we introduce a significant 1-bit LLM variant called BitNet b1.58, where every parameter is ternary, taking on values of {-1, 0, 1}. We have added an additional value of 0 to the original 1-bit BitNet, resulting in 1.58 bits in the binary system.”
Generative AI Large Language Models Llms 1-Bit Low-Bit Artificial Neural Networks ANN Openai Chatgpt GPT-4O O1 Anthropic Claude Google Gemini Meta Llama Compression Compaction Weights Activations Matrix Advances Future
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Look Up: The Scariest Part When We See the Big, Big, Big PictureBill McKibben is the Schumann Distinguished Scholar at Middlebury College and co-founder of 350.org and ThirdAct.org. His most recent book is 'Falter: Has the Human Game Begun to Play Itself Out?.
Read more »
ChatGPT Can Tell You What Scientists Are Doing With LLMsConfused about LLM architectures? Ask a model. They’ll tell you.
Read more »
When training medical LLMs, specialization may not always be better.General AI models rival specialized ones in most medical tasks, proving that optimized prompts and strategic use may outpace costly domain-specific training.
Read more »
The illusion of thought in machines that compute, not think.As LLMs emulate thought, true cognition is still uniquely human.
Read more »
Small businesses can capitalize on Small Business Saturday with some planningSmall Business Saturday — the Saturday after Thanksgiving — is coming up. But it doesn’t always translate to big sales for small businesses. There are some things small business owners should keep in mind when marketing themselves for the big holiday shopping weekend.
Read more »
The Best Small Loveseats, According To Small-Space DwellersWe've rounded up the top-reviewed loveseats out there, according to the small-space and budget-savvy dwellers who bought them.
Read more »