SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 3 months agoWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?message-squaremessage-square18linkfedilinkarrow-up133arrow-down111file-text
arrow-up122arrow-down1message-squareWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 3 months agomessage-square18linkfedilinkfile-text
minus-squareplutopos@lemmy.ziplinkfedilinkarrow-up1·2 months agoThere are no lower or upper bounds for local LLMs. Just get the best GPU/RAM combo you can with your budget. Some laptops also have NPUs that are dedicated specifically to AI iirc
minus-squareSocialistVibes01@lemmy.mlOPlinkfedilinkarrow-up1·1 month agoThanks. What about NVidia vs AMD? Should I go with AMD if cheaper?
minus-squareplutopos@lemmy.ziplinkfedilinkarrow-up1·1 month agoAMD has better Linux support if you plan on using it
There are no lower or upper bounds for local LLMs. Just get the best GPU/RAM combo you can with your budget. Some laptops also have NPUs that are dedicated specifically to AI iirc
Thanks. What about NVidia vs AMD? Should I go with AMD if cheaper?
AMD has better Linux support if you plan on using it