eduardoleao052

Train and deploy Neural Nets and Transformers in the browser with one simple tag!

Three months ago I made a post here with an early version of this project, and received some great feedback! One of the most asked features was GPU support, so I added it alongside many other improvements. Hope you like it!

Thanks for the feedback, glad you liked it! It does, and they work similarly to their PyTorch counterparts. The explanation is on the “Running it Yourself” part of the README.

Thanks for the tip, will look into it! Yes, I think it would always be better to leave a "vanilla" option available.

That's an interesting idea as well, could definitely see that working in some use cases.

I’m planning on creating a small article explaining the syntax and the functionalities of the Deep Learning library. I think that could be a useful learning tool, do you think it would help?

I plan on adding Typescript support soon! Thanks for the feedback

I think that’s it. I’ll probably add that soon

It would be with some simple tweaks. For instance, the current block does not support Cross-Attention, just Self-Attention.

Thats true, it’s a limitation of working between these languages. I tried to mitigate it by using clear JSDoc, so that each variable pops up alongside an explanation when calling a function.

Thank you so much for the feedback! I’m open to pull requests, absolutely. And about the article, that’s a great tip, I’ll definitely do that as well.

Fair enough, I’m currently studying what would be the best option, but I think you are right!

Really cool implementation. Thanks for the feedback as well!

Cool! Will take a look

Thank you! That’s going to help out a lot!

Thank you for the feedback! If you have any questions or suggestions looking it over tomorrow, I’d love to hear them!

I had read about this as a possibility, but will definitely look more into it now, thank you for the tip!

Thanks for the tip! I’ll definitely take a look. Adding GPU support is my next step!

Adding GPU support soon is absolutely my goal in the future! I think a PyTorch-based JavaScript library could be useful, as PyTorch has been way more dominant than TensorFlow recently.

I centralized the entire backpropagation around the Operation objects. They store the data about the forward prop in the cache, and serve as the connections in the graphs between tensors. Each tensor has a “parent”, “child” and “operation”. These store who generated the tensor, what tensors it generated, and how it was generated (what operation). I could store the backward function inside of each tensor instead of an Operation object, but I chose the slightly more verbose option because I think it is a little more interpretable and simpler to add new operations.

I can try implementing it in the future lol It would surely be a quality of life improvement. But with the current tools, I tried my best to make the syntax as similar as possible to PyTorch’s!

Could not agree more hahaha! I tried to work around it building methods like “torch.add(a, b)” for operator overloading and “torch.at(index)” for slicing. But it’s definitely not as seamless as these features you proposed.

Initially I'll stick to Javascript (plus some libraries to improve performance), but I'll likely look into that in the future.

Now that I've tried Javascript, I could branch out to other languages in the future. I chose Javascript initially for its popularity and easy use on web browsers.

That’s true. I myself learned ML in python first, and I’m just now trying to implement some stuff in JavaScript. I guess it could be useful to make web integration with neural networks easier in this case, or at least as an educational package!

I am currently studying Typescript and other JavaScript libraries to improve the library’s performance. So adding GPU support is in sight in the future.

You are absolutely right! Will solve that as soon as possible. Thanks for the feedback!

Thank you very much for the feedback! If you have any question about it, let me know.

Thanks for the tip! I will look into it. Speed in the tensor calculations is definitely one of the largest challenges in this project.

Thanks for the tip! I will look into your project, for sure! I am still a beginner on Typescript, but when I get the hang of it, I think I might refactor my package to Typescript. Again, thanks for the tip!