pull down to refresh

768GB of cheap Intel Optane DIMM memory sticks used to run 1T-parameter LLMwww.tomshardware.com/tech-industry/artificial-intelligence/enthusiast-runs-1-trillion-parameter-llm-from-768gb-of-intel-optane-dimm-memory-sticks-local-kimi-k2-5-install-achieved-roughly-4-tokens-per-second
381 sats \ 4 comments \ @zuspotirko 25 May AI tech
write
compose
related posts
90 sats \ 0 replies \ @sime 25 May

I love that people are pushing local LLMs.

Hosted LLMs are really hurting my open source heart.

And my internal fear that hosted LLMs will become mega expensive in the future.

reply
16 sats \ 0 replies \ @ealvar39 25 May

super cool. i love running local llms it feels like I'm actually talking to my computer

reply
16 sats \ 1 reply \ @BITC0IN 25 May

man, maybe i could do a full RAM full node with this setup

reply
17 sats \ 0 replies \ @sime 25 May

now we are talking!

reply