jeffw@lemmy.world to Technology@lemmy.worldEnglish · 1 day agoOpenAI hits back at DeepSeek with o3-mini reasoning modelarstechnica.comexternal-linkmessage-square16fedilinkarrow-up164arrow-down117
arrow-up147arrow-down1external-linkOpenAI hits back at DeepSeek with o3-mini reasoning modelarstechnica.comjeffw@lemmy.world to Technology@lemmy.worldEnglish · 1 day agomessage-square16fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up1·18 hours agoDense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn’t a SOTA MoE for that size yet. So, unless you need tons of batching/parallel requests, its… kinda neither here nor there? As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).
Dense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn’t a SOTA MoE for that size yet.
So, unless you need tons of batching/parallel requests, its… kinda neither here nor there?
As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).