Why is it replacing true/false with T/F? true/false is already 1 token in all tokenizer I've seen. Even worse is replacing null with ∅. ∅ is a special unicode symbol that takes up 2 tokens compared to the 1 token for null...
A trivial test through tiktoken [1] (though technically you really have to match the tokenizer to the specific LLM) would have shown them that ∅ was a poor choice.
Even from the perspective of learned training data, you can probably just intuit that from a frequency standpoint alone the empty-set symbol ∅ can’t possibly have appeared that often outside of things like set theory and logic.
I took their example to mean an actual LF ASCII character but now Ive read your comment, maybe I was being too charitable?
https://github.com/activeing123/mcptoon/blob/main/src/mcptoo...