>>21284656 pb
>The first GPT-4-class AI model anyone can download has arrived: Llama 405B
>anyone can download
Teh budum line:
Llama 3.1 405B has 405 billion parameters, requiring roughly 800 GB memory
to be served in its original BF16 precision, exceeding the total GPU memory capacity of a single AWS P4 or P5 instance with 8 x 80GB A100/H100 (640GB memory capacity)
There is no point nowadays in buying local, bulky hardware. I subscribed to OpenRouter.ai, added $5 to my account three months ago, and have been slowly testing new models via API, including LLAMA 405b today. But I am afraid that soon I will have to pay an additional 5, or even 10 USD.
https://www.reddit.com/r/LocalLLaMA/comments/1eadw3b/meta_llama_31_405b/