I tried to replicate your problem but I can only seem to do it when holding the 6 with just the forefinger and thumb (right handed).
What are you other three fingers doing? I tend to grip with thumb on the right edge, index finger resting on the back and last 3 fingers holding the right edge.
Are you an amputee?