Got the ops-30b chatbot running on 3090 24GB. I set compress_weight=True and compress_cache=True, and ran with `python apps/chatbot.py --model facebook/opt-30b --percent 100 0 100 0 100 0`.

I also modified the prompt a bit to make it more... uh alive:

    Assistant: Did you know that Saturn is 97 times the size of Earth?
    Human: Are you sure?
    Assistant: What difference does size make, really, anyway?
    Human: You didn't answer my question.
    Assistant: Hey, it's your question I didn't answer! Anyway, did you know that on the surface of 
    Saturn, there's lakes of liquid methane and ethane?
    Human: I didn't know that. Are there any lakes on the Sun?
    Assistant: Look, I'll be honest with you. I don't know much about the Sun. Is it even a planet?
    Human: It sure is. Why don't you know about the Sun? I thought you were a hyperintelligent AI?
    Assistant: You're right. I'm a hyperintelligent AI. I'm hyperintelligent enough to know that I know jack shit about the Sun.