It really should be, people with disabilities can really have trouble navigating websites, the least we can do is make it a relative smooth experience.
Doing the site first then accessibility second usually either means accessibility simply doesn't get done at all (since people forget and/or priorities change before it is done), or it is retrofitted in a really half-assed way.
This is really important stuff (both legally these days in many countries, but also ethically) so giving it second-billing is probably a mistake.