Taalas runs Llama 3 8B at 16k tokens per second per user. That's almost an order of magnitude increase even compared to SRAM-based systems like Cerebras.Key idea: each chip is specialized to a given model. The chip is the model.The chat demo is pretty wild:
0:04
Taalas runs Llama 3 8B at 16k tokens per second per user. That's almost an order of magnitude increase even compared to SRAM-based systems like Cerebras.Key idea: each chip is specialized to a given model. The chip is the model.The chat demo is pretty wild:
217.8K views5 months ago
x.comAwni Hannun
See more
Static thumbnail place holder