Everyone always (perhaps South Asians more commonly) asks where I'm from. I say I'm British. They ask "no, really". I say Trinidadian and Guyanese heritage. They say "you look like you are from India, your family must be from India". It does get a bit tiresome.
The only time it actually _really_ annoys me is when I have to fill out forms with ethnicity. There is never Asian-Caribbean, but always Black-Caribbean, so I always feel that I am never being represented.