You're right about hospital care, but if you need insulin you'd get it from a GP, not a hospital.
Just about anyone can register for primary care. There isn't an "ordinarily resident" test for primary care. P142 of this: https://www.england.nhs.uk/publication/primary-medical-care-...
Ordinarily resident applies to secondary care, but not including detention in a mental health hospital under the mental health act and not including A&E treatment in the A&E unit.
---begin quote---
8.4 Who can register for free primary care services?
8.4.1 A patient does not need to be “ordinarily resident” in the country to be eligible for NHS primary medical care – this only applies to secondary (hospital) care. In effect, therefore, anybody in England may register and consult with a GP without charge.
8.4.7 It is important to note that there is no set length of time that a patient must reside in the country in order to become eligible to receive NHS primary medical care services.
8.4.8 Therefore, all asylum seekers and refugees, students, people on work visas and those who are homeless, overseas visitors, whether lawfully in the UK or not, are eligible to register with a GP practice even if those visitors are not eligible for secondary care (hospital care) services.