That is not enough to solve the general case due to locking considerations.

If we have one transaction running delete and then insert concurrently with one transaction doing a normal update then the update will find zero rows to update if it is ran after the delete+insert. The inserted row will be invisible to the update while the old row will be locked. This means the update will wait until the other transaction is committed and then find neither the old nor the new row.

A correct REPLACE implementation would make sure the UPDATE would UPDATE the replaced row.

The above is assuming READ COMMITTED isolation level.