You can currently play around with the 7B[0] and 14B[1] parameters models hosted in Hugging Face Spaces.
AGI fears Fabrice Bellard
This seems to be made for mass parallelism.
Question: Would this scale to 100 million of collaborating 4G/5G smartphones? We have operational federated learning code and decentralised learning code (trivial stochastic gradient descent). Application: simple learn-to-Rank based on ClickLogs like its 2005 [1]. Then you have a true decentralised Google.
Would love to link RWKV to other pure decentralised tech. In the past we have build the first self-compiling Android app and first Android-to-Android P2P overlay network. Without any helper peers for carrier-grade NAT puncturing. My university systems lab lacks the size to keep up with the recent pace of innovation.
He also wrote an engine: "LibNC: C Library for Tensor Manipulation"
> [ ...]/LLMs
That could be a wish come true.
Edit: in fact, already one of his marks can be seen: «Larger models work optimally on lower cost GPUs (e.g. RTX 3090, RTX A6000) thanks to efficient quantization»
Prompt:
A website can be built in 10 simple steps
Output:
1. Research
2. Research
3. Research
4. Research
5. Research
6. Research
7. Design
8. Design
9. Design
10. Design