First image ('Describe', photo of my desk)
- 15.6 seconds to first token
- 2.6 tokens/second
- Total 180 seconds
Second image ('What can you see?', photo of a bowl of pasta)
- 10.3 seconds to first token
- 3.1 tokens/second
- Total 26 seconds
The Edge Gallery app defaults to CPU as the accelerator. Switched to GPU.
Pasta / what can you see:
- It actually takes a full 1-2 minutes to start printing tokens. But the stats say 4.2 seconds to first token...
- 5.8 tokens/second
- 12 seconds total
Desk / describe:
- The output is: while True: print("[toxicity=0]")
- Bugged? I stopped it after 80 seconds of output. 1st token after 4.1 seconds, then 5.7 tokens/second.