Being able to mimic a human's behavior is neither a necessary nor a sufficient condition for having subjective experience.
> Being an algorithm doesn't feel like you are an algorithm, from the inside.
How do you know that? How do you know that an algorithm "feels" anything at all, the way humans do?
I think that's not the point he tried to make. It's just that it happens that we're human and thus it's easier for us to relate to human-like experiences/feelings. An algorithm is not limited to that.
If we accept that our "feeling like something" is just an emergent phenomenon of our complex model of reality I see no logical reason why we shouldn't ascribe any sufficiently complex model of reality an inherent feeling. Due to our limited physical configuration we will never have access to all kinds of feelings. Much like we will never know how it feels like to be a bat.
I am arguing that it is. Once you move past the "giant lookup table" idea to actual algorithms that explain human behavior. In fact once you find the smallest such algorithm, it must necessarily be an exact simulation of a human brain, and all of our internal experience.
>How do you know that an algorithm "feels" anything at all, the way humans do?
It must feel exactly the same way humans do if it produces exactly the same output.