I'm also not a native English speaker, and one area where I know there's a huge difference between my Swedish vocabulary and my English is when it comes to names of plants and birds and spices and animals and trees and fish and rocks and flowers and vegetables and fruits. I know maybe thousands of names of such things in Swedish, but in English I know much fewer names. That's a few thousand words I lack and probably will never learn because it's so specialized.
(Btw, bananas are berries, but cows are not deers. :) )