A basic requirement for sentience is some form of change in response to inputs. Time at its most abstract is a sequence of changes of state. It is not require that it be linearly attached to what the universe considers time; as a simple for instance, you yourself are not linearly attached. You subjectively experience jumps in time when you sleep. Universe-time is an upper limit to you, not your actual subjective experience of conscious time. (I add a lot of qualifiers there because your body does continue on, of course, but we all also tend to agree that we experience gaps in clock time when we sleep; that's all I need for this point.)
The fact that a GPT-3-like network can not experience change over time is enough to say that the AI that one has a "conversation" with is not sentient.
One might construct something with continuous online training that would create a more interesting debate. But that's not what this particular AI is. This particular AI we are only talking about because a credulous human had a conversation in which they accidentally led a textual AI to write a story about AIs that happened to use the "I" pronoun, but there isn't actually any conversation occurring because conversations also require state changes and time. This AI has no temporal experience. (In fact, this is one of several tells in the transcript, when the AI putatively discusses its experience of time in a manner its substrate is patently incapable of supporting. This is not the only such tell; the whole transcript is shot through with the sort of errors GPT-3 makes in coherence. However, to AI research's credit, they've successfully pushed the threshold for the amount of coherence an AI can generate past what most humans can pick up on, as we are all so used to dealing with, well... each other.)
Progressively narrowing down what me mean by exclusion is a weak form of progress on the question of sentience. Before anyone jumps up with the "well what ifs" and "but we don't knows", I'm perfectly aware of the limitations of having only biological samplings for the question of "sentience" and fully expect the question of "what is sentience" to have a very wide ranging conversation across all sorts of philosophical bases, going all the way down "but what is the definition even for?" and tripping over all sorts of philosophical land mines that we never resolved, but at least all kinda learned to stop tripping over every second (whether for good or bad reasons), all the way down to the eternal debate between utilitarianism and deontology. But I still think there's some things we can exclude, and we don't need to pretend otherwise. Embedding in some sort of temporal experience is one of those things.
Anyone who wants to stretch the definition of "sentience" is, in my opinion, actually defining something else entirely. Perhaps that something else will even be useful for other reasons; for instance, one can imagine raw measures of intelligence that could result in very intelligent but non-sentient AIs, and fully sentient yet not necessarily all-that-intelligent AIs. But if one of the things we want to argue about are "rights", rights are themselves fundamentally embedded in temporal experience. You can not "hurt" something that does not exist in time; there can be no transform from a better to worse state if there are no transforms, so there is no point in talking about the "rights" of such a system to not be "hurt". Things that do not exist in time can not discharge responsibilities associated with rights, because they can not go through the procedure of realizing they have the responsibilities, then taking actions in time to discharge them, so there is no point in talking about the "responsibilities" of such a system. Etc.