Jupyter notebooks in the IDE: VS Code vs. PyCharm
towardsdatascience.com
towardsdatascience.com
The other issue is the language server seems to get into a funk regularly - the completion stops working as does the display of docstrings so I regularly use the "Developer: reload window" feature to reset everything.
Also the plots view is poor - it's basically just an SVG you can magnify - there's no proper zoom or rotation for 3d plots.
I was also irritated by the "smart" environment discovery stuff - just let me specify which python install or vent I want to use. Don't make be google to see what names/locations I have to use in order for an environment to magically appear in the list.
Also the editor extension (I use vi mode) is ignored. And even saving the notebook doesn't work the way every other editor pane works in vscode.
Hmm... writing this list makes me wonder why I haven't looked at alternatives.
(Disclaimer: I work on VS Code, but am not working on notebooks)
I LOVE this feedback! Keep it coming (though entering issues https://github.com/Microsoft/vscode-python would be ideal)
Yes, as conner4312 VS Code is investigating how to better incorporate a juypter notebook experience within it, natively. As has been pointed out here, there are a number of limitations that the python extension notebook support has run into. We have been working closely with the VS Code team to find solutions to these limitations. The custom editor work (https://github.com/microsoft/vscode/issues/77131) is one such mechanism. We plan on rolling out our use of that work soon after it is officially released (sometime in February). However, this will not address all of the problems mentioned in this thread. Those issues will be addressed as native jupyter support arrives in VS Code.
Here's some additional info: - Find/replace. (Personally, this drove me NUTS! Just ask my team! :) ) That being said a simple find (ctrl-f) is avaiable in the current official release. Unfortunately, it does not include replace. While we recognize that this is a severe limitation for some, implementing a full find and replace solution would end up requiring the extension to duplicate most of the functionality provided by VS Code itself. Therefore, we currently planning to piggy back off of that work when it becomes available. If that winds up taking an inordinate amount of time, we will consider further stop gap work.
- The markdown loss issue has been addressed and will be in the February release.
- For the language server problem discussed above, we need more info. I would encourage you to enter a new issue at https://github.com/Microsoft/vscode-python. In the meantime, there are known related issues that will be fixed the Feb release. One in particular, was caused by the backend jupyter server inexplicably taking a long time to return us a full completion list. So when you don't see completion pop up, it may not be that it's broken, it may just be taking a long time. We are working around this issue for the February release. Also, if you're having docstring issues, you may find that turning off the python jedi language service in settings will help.
- Plots view. Yes, it is limited and honestly can cause rendering slowdowns. We are working through ways of addressing both problems. A good solution for supporting "proper" zoom will require adding ipywidget support overall, which is coming soon.
- "Smart environment stuff". I'm not following completely. Is this a general python extension issue or does it have to do with the jupyter kernel selection support we have for notebooks? Again, feel free to enter additional details at: https://github.com/Microsoft/vscode-python
- Supporting global editor extensions like "VI mode", isn't something that we can support without help from VS Code itself. One of the main reason's VS Code is pursuing native notebook support is to be able to handle issues exactly like this.
- Saving the notebook weirdness will be addressed in the February release. We were waiting for the custom editor support I referred to above to arrive.
Hope this is useful information!
Awesome!
> - For the language server problem discussed above...
I don’t think that’s Python-specific, I’ve encountered language server seemingly getting “stuck” in some state for almost any language I’ve used for more than a few minutes in VS Code, including Python, C/C++, Go, Rust, etc. I usually need to reload the window more than once per day.
> - "Smart environment stuff".
Probably talking about the general Python-mode Python selector. One can only choose from a list of detected Pythons; no file system dialogue available, and one can’t even paste in a full path unless it’s in the “sanctioned” list. I’ve cursed on this one too.
To balance that: I haven't forgotten the "WOW WHAT IS THIS" feeling I experienced the first time opened an ipynb file from vscode when the notebook support kicked in and asking my colleagues "did you know about this?" as I excitedly started exploring the feature. Since then (and I'm a relative vscode noob), I've never been tempted to go back to working with notebooks through a browser. Despite my gripes, the overall experience is still miles better (in my opinion) than accessing notebooks through a browser. It's the future for sure.
And thanks for all those updates - I can't wait for the Feb release - and it's very reassuring that these issues are not only well recogonized but also are being aggressively addressed.
Regarding some of your feedback to the feedback:
- Re. the language server issue - it could well be caused by a slow down. I'll test that hypothesis the next time I experience it.
- The "smart environments stuff" is, as you say, actually a general python extension issue. But actually, I withdraw that complaint. I've just checked and I can point the extension at any python interpreter by editing settings.json - it's just the GUI which restricts you to selecting from a list of discovered interpreters.
Thanks again for your response and your engineering efforts in general.
I think environment selection would be considered a bug. I theory any python environment that's on your machine should be selectable in the UI. Could you enter an issue at that same location? I can for you, but I'd prefer that the issue comes directly from an outside user (sometimes we filter bugs based on from where they came).
Strangely, it uses a different jupyter notebook implementation to VS Code. I found it avoids some of the annoying bugs mentioned in another comment.
It also introduces a SQL kernel which I’ve found useful for organising my queries.
I've never heard this term before and a brief search for it did not bear any definitions.
Can you expand on what exactly makes an SQL kernel?
I am curious to see the general workflow expanded on. I hesitate to think there is a single correct way to keep a notebook. Seems flexibility is the key feature. And is why text is so well used.
[1https://github.com/millejoh/emacs-ipython-notebook/blob/mast...
The only thing I would like is to be able to use the Python interpreter _within_ the context of a function sometimes, it's annoying when an error occurs inside a function and I can't access the local variables.
I like jupyter notebooks too, but one of the things stopping me is that I can't use emacs features for editing ;)
import IPython; IPython.embed()
Finally, you can:
import code; code.interact()
I have a custom function `embed()` [1] that calls `code.interact()`, but with history and autocomplete configured. This is the closest I got it to looking like a regular Python interpreter at an arbitrary place in the code.
In any case I think there was a way to examine variables from up the stack of a stack trace or something like that.. bit too lazy to look it up right now. Regardless, working with a REPL in emacs is pretty great ;)
I think there is a way to do jupyter from emacs (or at least there was a way to do ipython notebooks), but I haven't used it extensively, just tested it once I think. I guess I'm pretty satisfied with the REPL and haven't found a need to have intermediate output during the running of a program.
What I do like about the matpotlib interactive approach is being able to watch the progress of a loop very easily, which is hard to do from a notebook, although there are some ways, but they are either hacky or require some pretty sophisticated things like custom widgets.
So I just add a try: except around code known to be crashing and call my `embed()` function on except. This has served me well enough that I haven't bothered with anything else.
Plus, if I really want to inspect unplanned crashes, I would want to be able to get them when running, and not just developing, the app. So there might not even be a terminal - there are more complications than just 'not having to declare where you think it will crash', so I don't think I'll bother solving the more specific problem unless it would help me debug unexpected crashes in a broader context.
https://stackoverflow.com/questions/4234612/launch-an-ipytho...
This gives you a pdb prompt not an IPython prompt, but you can examine local variables where the exception was raised, and exit back to IPython with Ctrl-D.
In my work (data engineering and training deep neural nets) I also happen to use both. It's unwieldy, but I can't get rid of either one (pycharms or jupyter lab)
I find myself able to prototype code a lot faster in that setting compared to a Notebook. Even with plots, plotting in Pycharm provides me an interactive plot I can zoom around in, which is not the default behavior in Jupyter. The only situations where I ever use a Notebook is when I need to share a Python "document" that isn't just a python script + results, but rather some form of extensive self-documenting code that could benefit from Markdown integration and co-location of output plots, etc. The other situation is some of their built in animation/interactive widgets (perhaps that is the interactivity you are referring to?). In most other cases, I've found it quite limiting compared to a good IDE like Pycharm/Code.
That being said, I think the efforts on Pycharm and Vscode's part to bridge this gap is commendable and quite interesting.
See:
Not sure about Delphi though.
However, I would also point to Java, Smalltalk, Eiffel as another set of examples.
Personally, I use Atom with Hydrogen backend, but I develop the files in sch a way that they tend to end up as working for both interactivity and for scripts. So say my train_keras_script.py works both to train when it's deployed on our GPU server, and works as my primary debugging script.
I tend to throw most of my results into a .png output files and .csv's, the latter of which I use for R analysis scripts (and yes I know R Studio has markdown, but I'm just stubborn I suppose ).
In terms of IDE bells and whistles, I think #1 is certainly autocomplete. I will admit that I can be somewhat of a crummy speller. When first learning to program, it caused me a lot of headache trying to debug. Simple syntax checking and autocomplete of variables (especially long variable names that are meant to be descriptive) made me really embrace programming professionally.
My approach is to combine the strengths of both IDE and notebooks. I put my code into a package that I edit with PyCharm. I import that code with the `%autoreload 2` magic in Jupyther to load data and test my functions. This way I get data persistency and plotting capabilities of the notebook and editing, etc capabilities of the IDE.
While most of my code is in my modules, I still like having the actual notebook versioned.
Ultimately, Matlab is a better fit for the things I want to do in technical computing. My employer offers it and Simulink with a wide variety of toolboxes and blocksets so I can get farther faster for the problems that interest me or that I am asked to solve. The new table datatype and the steadily improving Livescript make exploration and notetaking very easy. I've also been able to rely on Mathworks documentation to get up-to-speed in new areas.
It's an approach that I think has somes pros and cons relative to integrating these features into an IDE. The main argument for moving them to a separate app, is that IDEs are already really complicated.
I'd like to support Linux and Windows eventually too, but it's easier to start with supporting just one platform while I'm just testing out whether there's other people interested in the idea.
(No "data science" here either, but into repl based instant feedback development & collaboratiove programming tools)
(Jupyter supports "kernels" and there are also more non-Python language kernels that you can use)
I use primarily for doing stepwise development. I also use them for a decent way to see the output of my code, usually tables and charts.
I have unit tests for the various modules and classes created.