I replaced the silicone tips with memory foam, which fixed this problem for me.