As someone who does teach tutorials as a side gig, I would argue that implementing matrix operations in a tutorial on neural networks is overkill. No matter what the level of the tutorial you always need to draw a line and assume a certain amount of background knowledge and knowing how to use standard tools isn't too much to ask. (yes, I know numpy isn't part of python's standard library, but it comes with pretty much any Python distribution as many other libraries depend on it.)

If we're talking about a longer format, such as a book, then we might consider digging deeper and implementing as much as possible using the barest of Python requirements. Indeed, Joel Grus does implement everything from scratch in his great (although a bit dated) book https://www.amazon.com/Data-Science-Scratch-Principles-Pytho....

EDIT: This is still a work in progress (and relies on numpy and matplotlib), but here is my version: https://github.com/DataForScience/DeepLearning These notebooks are meant as support for a webinar so they might not be the clearest as standalone, but you also have the slides there.