How do people cope with this? Do you supplement it with other tools? I spend a lot of my time in an IDE and then just paste some of the code in to cells. That seems easier.
Work (and often debug) in jupyter -> open the notebook from pycharm when it's got some completed thoughts and write into a python module + test module, tidying up and adding type annotations.
Sometimes doing that multiple times so that the notebook is importing from modules which were originally pulled out of the notebook.
It sucks having to use two tools but I don't think there's any one tool that can do both as well as pycharm/jupyter, short of me getting a lot better at emacs or writing a lot of custom Atom extensions (I think).
(Relevant issue: https://github.com/jupyterlab/jupyterlab/issues/2163)
Folks that try to do all programming in notebooks typically drown in complexity and suffer.
This is a trade-off between how much code you're writing and how much data you're processing. If you're writing maybe 20 lines of code but you have enough input that it takes several minutes to run, the notebook becomes a clear win for your development process.
P.S. Disclaimer: I lead this project at JetBrains, Inc.
Is it possible to use it as what seems like a drop-in replacement for jupyter notebooks?
We have more data then I think would make sense to transfer out of our clusters/datacenter and privacy issues would probably be raised but I would love to use something like this.
We are seriously considering on premises version.
>Is it possible to use it as what seems like a drop-in replacement for jupyter notebooks?
Jupyter import/export will be released soon.
I also notice that developing in this way encourages me to create smaller, more testable functions that i can easily work with inside a single notebook cell.
If you're writing a lot of code in them, it's probably better to put that code into libraries that get imported and reused.
And I do agree that default code environment is unbearable. Particularly the auto insertion of completing quotation marks, which has me continually fighting with the editor to get correct code into a tiny web text box.
What I'm specifically talking about is even that kinda hacky experiment code you end up writing. I don't try to implement whole projects in there, but even just "train this model" type code ends up being a hassle because of how bad the editors are.
My above comment was more referencing wishing I could spend more time writing experiment code in jupyter without copying and pasting all the time.
Maybe due to often importing and naming (something you don't do in Java.)
E.g
Import matplotlib as plot
putting models (in a sense of more complicated models, not just a SVM), data-pipelines, shared visualization-code in a src folders and experimenting in the notebook divides stuff that's interactive by nature from "real" coding. I don't context-switch that much to be honest.
I don't really copy code into cells, because I only experiment there.
Also, what happens if you need to share code between notebooks?
I think notebooks should be simple and explain the experiments and the reasoning behind them to your coworkers. Otherise it's hard to coordinate and learn from each others insights into the data.