Spyder – a free and open source scientific environment written in Python | Hacker News Reader

Spyder – a free and open source scientific environment written in Python

spyder-ide.org

386 pointsjka160 comments

It often causes a double-take for me when I see projects on HN with the same name as projects I once worked on.

Way back in time, maybe around 1998, a friend an I worked on a steganography tool that we called Spyder. I don't remember where it was published or under what license (probably freeware), but the algorithm was incorporated into Hide4PGP (http://www.heinz-repp.onlinehome.de/Hide4PGP.htm), and we stopped work on it.

This post just caused a bit of nostalgia. I can still remember my co-developer taking a break jamming on a bass guitar while we tried to figure out the striping problem (the solution was to compress the data to increase entropy first).

There's also a recent video game named Spyder

https://www.spyderthegame.com/

I like spyder a lot for the variable explorer, which makes it feel more like Matlab. However the constant bothering about installing Kite got annoying. Unfortunately there isn't a way to use Kite and have it not constantly annoy you about upgrading to the enterprise edition. If Kite had a no ads plan for like ~ $10 a year it would make a lot more sense to integrate it in spyder.

Not to be trite, but there is a github repo and the license is MIT. I'd guess it's possible to find the relevant nag code (possibly in its own github commit), strip it out, and recompile.

Edit. maybe this is what you're talking about? https://github.com/spyder-ide/spyder/commit/6b602af50f632cf6...

but why is that even there? I guess that's just another reason to stay with emacs...

Sponsoring/buying things in exchange for kite promotion seems to be their MO - there was quite a bit of noise about the more brazen instances of it a while back when the used existing editor extensions to push their product or telemetry code (see e.g. https://theoutline.com/post/1953/how-a-vc-funded-company-is-... and HN discussions about "kite")

Well, this is the thing: code completion in Python is not a a simple task and we've struggled for years to improve it. Now we support the Language Server Protocol (designed by Microsoft for VSCode), but still things are not great, i.e. there are many things that can't be completed by Jedi, specially for scientific libraries.

So, thanks to Kite's software, we implemented a client for it and now lightly promote it to our users, to offer them a possibility that works better, but it's completely optional.

We also consulted our community before adding Kite integration and they were favorable to it, that's why we decided to proceed with the idea: https://twitter.com/Spyder_IDE/status/1138466325532217346

> 147 Votes in 606 days

At the time we did that poll we had ~1000 followers on Twitter, so that's 15% of our audience, which is pretty good for an online poll.

But more important than that, we receive ~200 issues per month on Github (which makes us one of the projects in the PyData stack with higher values on this metric) and I don't remember a single one since we rolled out this feature that argued strongly to remove it. So either people dismiss our suggestion, or try it and decide if they like or not.

Like a fellow poster said, that's easily manipulated, especially if the marketing of a VC-funded startup jumps on the train. My concern with this would be that you've arguably turned a nice free (as in freedom!), open-source-tool into a vehicle to push more or less proprietary and user-tracking code onto naive, unknowing users (though I guess that is not too uncommon in the anaconda-ecosystem you target - spack user and guix/nix-sympathiser here). Why make it a nagscreen (not sure about the extent of that), if you could just make it an option or a status message.

For me you're now just competing with every proprietary python IDE, which basically drops you of my list of tools to use or recommend (one can also use free PyCharm or VSCode or RStudio then, where you know, what you get...)

> My concern with this would be that you've arguably turned a nice free (as in freedom!), open-source-tool into a vehicle to push more or less proprietary and user-tracking code onto naive, unknowing users

I respectfully disagree. First, the decision was not only mine, but from our team and community (see my post just above yours). Second, you are not losing any freedom by simply dismissing our message to take a look at Kite. And third, Kite doesn't track our users.

> Why make it a nagscreen (not sure about the extent of that), if you could just make it an option or a status message.

This is quite common in Hacker News: you're criticizing us without knowing how this works. We only show a message about Kite once. Then we show another message in the editor *if* our code completion machinery can't provide completions but Kite does. Both can be dismissed with a single click, that's it. These features are annoying for the user that started this thread because their settings are reset every day.

> For me you're now just competing with every proprietary python IDE

We are effectively competing with other editors and IDEs, with or without Kite. If giving the chance to (not forcing) our users to have better code completions makes us more competitive, I'm all for it.

I can attest to that kite was only promoted once to me through anaconda... However, kite itself has gone above and beyond to try and nag me into buying their enterprise edition - it's only just made me hate the software.

It will suggest "" completions at the top of the list, even for incredibly mundane completions, which is apparently a "premium completion" which almost immediately results in a nagging popup that requests that we purchase the enterprise edition because "you are out of your daily premium completions". If anything, I wish the spyder team would completely disallow kite to push their product in that kind of way, because it completely goes against what open-source projects stand for.

I would say the average person using spyder would have no idea how to do this. They program as if they are in R and not Python, so it’s short procedures in certain packages.

(Spyder maintainer here) Kite is completely optional for Spyder. If you don't like it, you can uninstall it and Spyder will continue working without any issues.

Is there an option to not install Kite in the first place?

I have installed Spyder both from its own repo, and using Anaconda, and I don't have Kite. I don't recall doing anything special during installation.

Thanks for the confirmation @buo. There's really nothing special about installing Spyder, i.e. we don't install Kite behind the scenes or anything like that.

I don't understand your question. Kite doesn't come pre-installed with Spyder. We only show a dialog once informing users that Spyder can use Kite to improve code completions in our editor.

After that, if a completion can't be provided by Jedi but it could by Kite, we show another message, which can dismissed entirely.

Both things don't force you to install Kite, but also show a button in case you want to do it within Spyder.

The issue isn't it installing Kite, it is the messages about installing Kite that bothered me.

As I said, it's a two times thing, but if it bothers you so much, we could a command line option to disable those messages completely. Would that be ok?

That would be nice, my dev environment gets wiped every night, so I see the message everyday. That and the one time I did install Kite, Kite kept bothering me about buying some enterprise plan.

I do appreciate the work you do on Spyder though!

Great! It's too late for us to include it in our next version (4.2.2), but we'll definitely consider it for Spyder 5.0, to be released in April.

You are amazing, thank you for being so responsive!

Awesome! I also suffer from this -- the public terminals at my university always start from a fresh install and these sort of nags are everywhere.

I really wish more software companies would re-think the startup nags all together. It is easy to say that 'oh we only show it once' but honestly who only installs a piece of software once in their life? I have probably clicked through the onboarding for microsofts browsers literally a thousand times now and I don't even work in IT. A typical user probably at least a dozen if not more, per piece of software...

Thank you for replying to posts on this topic and for all your work on Spyder.

The following is simply an FYI, not a complaint - I am very thankful for all that the open source community does on projects like Spyder.

I've used Spyder for quick tasks for many years but only use it rarely now due to the string formatting in the console.

Just this morning I copied a lot of text from the supplementary data of a paper, pasted it into Spyder, processed it, and output it to a nice tab separated table in the console. In other IDEs, I can copy that output and put it straight into my file and it works perfectly but Spyder now replaces all the tabs with spaces. That might be OK if the output were all numbers but the output has descriptions and then numbers in different columns so I end up having to output to a file instead of the console to get something usable. A minor issue, but it negates the value of a lightweight IDE.

In the past I had a lot of issues with unicode in Spyder where the exact same code worked well in command line python and in other IDEs.

Because of these types of issues I find that I'm better off just going to my heavyweight IDE instead of Spyder even for simple tasks if I don't want to do them from the command line.

You're probably using an old version of Spyder because this problem was fixed a long time ago:

https://github.com/jupyter/qtconsole/pull/397

I’m not sure if it’s really fair to complain about seeing a two time post install message all the time because you effectively uninstall and reinstall the software everyday, or at the least not without disclosing that your environment isn’t what most people use.

I mean, presumably GP has a reason. Adding a command line option, as seems to be happening, strikes me as a perfectly logical solution.

Constant? I only got asked once, on first use. I said no. I still consider it a bit rude in an open source program though.

They funded us to implement this feature: https://news.ycombinator.com/item?id=26042582

But we also consider it a nice suggestion to our users, for a feature they've asked for years to make more reliable and robust.

Also, never forget that they've bought out Atom extensions in the past to insert promotions for Kite.

https://github.com/atom-minimap/minimap/issues/588#issuecomm...

https://github.com/autocomplete-python/autocomplete-python/i...

Edit: Related HN thread with ~400 comments. https://news.ycombinator.com/item?id=14836653

We were aware of that and very careful to not allow something like it for Spyder users.

I mean, the point isn't that Kite can't succeed at integrating with third party tools in less user-hostlile ways. The point is that I'm not touching Kite with a thousand-foot pole after what they've done in the past. The trust has been lost. And the idea that it "nags" you to install Kite seems to indicate that money has traded hands somewhere, as in this is an advertisement. It just seems like much the same as before.

Thanks for the info. I stopped using Spyder over the annoying and distracting promotions. This confirms my feelings about it.

It is also time consuming to properly install from source, via GitHub, versus Anaconda. It does have a learning curve in that sense, even for someone pretty experienced.

I am currently using Atom. I may switch to Sublime Text and add extensions to make it a Python IDE.

Can I get the following behavior from Spyder (last time I tried I couldn't find a way) - for some keyboard shortcut like Cmd/Ctrl + Enter:

- the shortcut sends the selected code to the terminal (this is possible)

- if I hit the shortcut when nothing is selected, the line where the cursor currently is gets sent to the terminal and the cursor moves to the next line of code

- if the cursor is at the beginning of a block of code like an if-statement or a loop, the whole statement is sent to the terminal and the cursor moves to after the statement

RStudio has this behaviour by default, I find it very convenient and I haven't been able to replicate it in other environments.

That sort of workflow sounds a lot like what you would get from a notebook like Jupyter, albeit you would have explicit block of code ("cells") to be executed together.

It looks like Spyder now features Jupyter Notebook integration as well: https://github.com/spyder-ide/spyder-notebook

You can define code cells in Spyder and execute them with a shortcut. I’m on an older version, but for me it’s “# In[]” to start a code block, then shift-enter to execute. -Edit: to clarify, that string starts a code block, which continues until end of file or you designate another block.

Executing the current line or the selected lines is just F9.

As a sibling points out, I use F9.

I prefer the control enter because I am used to enter to execute, but it is fine.

RStudio has this behaviour by default, I find it very convenient and I haven't been able to replicate it in other environments.

Certainly you can replicate that in Emacs?!

I think that's how ESS behaves.

This is how Emacs/ESS behaves, like one commenter said. You can also get this behavior in python mode by installing python-x.el

As someone who has used spyder everyday for years, it’s really getting in my way lately.

Important caveat: I’ve just recently upgraded from some really old versions to the latest spyder.

Now it throws off my work flow by continuously tripping code hints or suggestions or whatever. When I try and click into the interpreter I find now I’d better click two or three times instead of one — to subvert the type hint that is sure to push cursor focus back to the code. If I don’t do that I am suddenly typing into the code window that which was intended for the interpreter. It sucks and I need to find out how to fix it. Surely there is something I can do in settings to chill the hinting/or-whatever out.

I've been having that issue with vscode; sometimes the tooltips get in the way and I have to click several times to get my cursor in the code. I think the tooltips should be click through.

Yes, you can disable hovers and hints completely. For that you need to go to the menu

    Tools > Preferences

and then

    Completion and linting > Introspection

and disable there "Show calltips" and "Enable hover hints".

Thanks! I’ll try turning off “everything” and see how it goes.

— I am hoping to find some middle ground where I can still make use of some features. I’ll play around with it some more. I’d gotten discouraged by stack overflow discussion of similar issues where turning off various features wouldn’t stick.

Replying to my own comment because there’s more to it. I’m paying more attention to the issue, and sometimes Spyder just context switches, quite often with very annoying timing, and often for no reason I can pinpoint. It just wants to be in one window right after I click in a different window. This does not make easy to diagnose or complain about, but it is driving me away rapidly.

If curious see also!

2018 (a bit) https://news.ycombinator.com/item?id=16232940

2013 https://news.ycombinator.com/item?id=6622156

2014 https://news.ycombinator.com/item?id=7042369

2009 https://news.ycombinator.com/item?id=889615

If you regularly use Spyder, what is it specifically that you like about it?

I had pretty much thought it was just a tool for people who normally work in R and Matlab to ease into python. Which I totally understand, but is there more to it? It is purely for scientific programming, or do people use it for other things too? Like would you ever write a Django app with it?

What am I missing out on by not using it? For my normal use, I am not seeing anything I can't do in VSCode. Is it worth checking out though?

I haven't used it, but I immediately decided to try it out, just in the hope that it will make managing matplotlib windows a little more convenient.

Outside of scientific computing, I typically go to Spyder on one-off scripts and smaller projects.

I like Spyder over PyCharm for smaller projects because there's less startup cost and the code cells functionality is handy for the "edit and run this part until it does what I want" style of coding, which makes the bulk of my time in small projects.

For bigger projects, PyCharm's refactoring, commit diffing, code regions, and formatting are enough better to be worth the fuss. As a rule of thumb, if I'm going to bother to set up a dedicated environment / git repo / file structure / test suite, I'll typically use PyCharm, else Spyder.

> and the code cells functionality is handy for the "edit and run this part until it does what I want" style of coding, which makes the bulk of my time in small projects.

Yes, I love this „quick and dirty” interactive style within Spyder too, but switch to PyCharm when I need to refactor, work with larger project or test. Sometimes I'd switch multiple times per day for the same project, so having a more Spyder-like DS tool under development at JetBrains seems to be good news.

I admit that I use it mainly because of all the nice code checking. I've been programming for 40 years, mostly "scientific," but have never made effective use of an IDE. So this is a new experience for me. With the guidance of a couple of colleagues, I'm bring myself up to date on writing quality code.

Thanks for your remarks! I think they are a testament to what we're really trying to: provide a comfortable environment to do scientific computing that also has the nice features you can find in IDEs designed for professional software developers.

I use the Anaconda-packaged version of Spyder on a daily basis. Great product, although I can never seem to get the IDE updates to consistently work within that environment.

Nevertheless it is my favorite IDE for Python.

You could consider using our new installers, if you're on Windows or macOS, to avoid that: https://github.com/spyder-ide/spyder/releases/tag/v4.2.1

Then you can connect them to your conda envs, as described here: http://docs.spyder-ide.org/current/faq.html#using-existing-e...

Thanks!

Does the installer work with WinPython?

Also, thanks for a great tool!

Sure, it works with any Python installation/environment that has spyder-kernels installed.

Thanks. So far, pip install --upgrade spyder has worked with no hiccups on WinPython, but it's good to know there's an alternative.

I'm not much of an Anaconda fan, but installing Spyder as a user package (pip install -U spyder) has always worked out well for me. After an upgrade, all I've ever needed to do was to upgrade to the spyder-kernels in my venv.

Spyder is such a great tool and much more useful than Jupyter Notebooks for explorative DS work, imho. Thank a lot for maintaining it, Carlos. I don't mind the kite installer - but I've tried it out and found the completions pretty useless.

Thanks a lot for your kind words! And I'm glad to know Spyder works smoothly for you with pip (it's not usually the case).

Too bad you had a bad experience with Kite, but (as I said several times here), it's completely optional.

How does it compare to PyCharm?

I'm a huge fan of Spyder. I have tried PyCharm and found myself much less productive. I think it's the integration of the variable explorer, and how you can easily look into tabular data or lists. I wish the updating process were more seamless, without requiring installers, similar to how VSCode or Chrome handle updates.

Thanks for the feedback. That's planned for this year, i.e. to have auto-updating and signed installers (at least on Mac). So stay tuned.

The RStudio equivalent for Python. Last used this so many years ago, and from the screenshots little seems to have changed.

RStudio supports Python these days. I use it daily and it’s really enjoyable.

I think they ought to call it DataStudio bc it’s multilanguage.

https://rstudio.com/solutions/r-and-python/

This is a better link: https://rstudio.github.io/reticulate/

Spyder is great. With all the complexity of python dev environments (virtual env managers, package managers and then the IDE), it is nice to at least be able to provide users with a well functioning self-contained IDE that just works out of the box.

I couldn't agree more. While I have moved on from Spyder onto Pycharm for many years now, Spyder will always have a soft spot in my heart for being a great first IDE that brought so many great quality-of-life features and replicated and surpassed the MATLAB experience of prototyping and developing code/algorithms in an interactive manner.

Also, I know ccordoba is here on HN. If you read this, know that you have immense respect from me for tirelessly maintaining Spyder for all these years (I started using it in 2009)

Thanks a lot your kind words! I'm really glad you enjoyed using Spyder and have fond memories of it.

The variable explorer is much better than the one PyCharm has, for example.

Of course, Pycharm has many other tools and a much better interface (Spyders scaling is grgrgrgrgr).

But for scientific data work? Spyder is great!

Oh boy, I can't even get spyder to move between two of my monitors because it always complains about scaling and then I can't get the popup to consistently not show up

Would you prefer to hide it forever? We could add an option for that, but scaling could fail in some circumstances.

I would 100% rather hide it forever

Eh, Streamlit and PyCharm is a much more powerful combination IMO. I can’t quite put my finger on it, but something about variable explorer always felt like training wheels to me.

Because it is super manly to guess at what the 20th element of an eigenvector is? Or you like super long printouts of data where it is hard to figure out what the index is? Matlab has one for a reason, and it is because very often you want a quick look at your often gigantic data.

Lol, that got a chuckle out of me. Yeah, it boggles my mind that someone would be disdainful of something as useful as a good variable viewer in an IDE. While it is certainly pretty essential when you are working with data or prototyping algorithms, I also do a lot of software engineering and library development in Python and I still find it *extremely* useful in Pycharm when prototyping code, and especially when debugging.

Like sure, I could drop into pdb and tediously work through printing things out (and good luck if you are printing out ndarrays or large dictionaries or wanting to look through instance variables for an instance in your local execution context) and take 4X longer to figure out why something isn't working, or just in a quick glance figure out exactly what the issue is. I guess I'll continue being a n00b developer happily relying on a variable explorer whenever available :o).

I respectfully disagree. The variable explorer allows to check what variables you have declared and how they change with changes in your code.

This is critical for scientific computing because you're constantly interacting with data (i.e. reading, cleaning, plotting and analyzing it), so having a dedicated pane for it is a great advantage.

I've found that people who aren't working on large data cubes or time series don't see as much value in variable explorers.

Anyone who's dealt with huge data arrays appreciate it.

Also, just wanna say Spyder looks like it's come a long way in the last decade. Congrats on some fine work!

Spyder got me off Matlab nearly a decade ago. I had just learned Python and had trouble getting Matlab access in my lab (I was an undergrad working on radio astro data) and after many weeks fighting with our dept. IT over a license, I started using Spyder, and it was pretty awesome. So many thanks to everyone who worked on Spyder!

> Also, just wanna say Spyder looks like it's come a long way in the last decade. Congrats on some fine work!

Oh yeah, this is so true! Ten years ago, when I joined the project, it was just Pierre (Spyder's original author) and me. Then, in 2016, Anaconda gave us the resources to hire three developers part-time to work on the project, which was a huge boost (between 2012 and 2016 it was mostly me and three or four volunteers).

Now we have a team of five people working part or full time on Spyder, all hired by Quansight: https://www.quansight.com/

So you can expect lots more good things to come in the future.

That's good news.

Maybe you could let Quansight know their homepage is a bit broken (at least on Firefox on Android): The "READ MORE" link shows up on top of the text.

The variable explorer comes from Matlab I think. It is handy to quickly and interactively check out what is going on in a matrix or ndarray, especially with the color coding.

PyCharm also have a variable explorer; one which I prefer over Spyder's :)

>Eh, Streamlit and PyCharm is a much more powerful combination IMO.

What is you PyCharm + Streamlit use case and workflow? Just curious because I have just learned about Streamlit from your comment.

Basically that you can have a notebook-like workflow, without the cumbersome interface or unpredictable statefulness. And when you need to wrap it up for batch jobs, you can basically leave the code as is, since it can just take the widget default values in headless mode.

Okay, so maybe a variable viewer isn’t a bad thing. I just know from personal experience that people get in the habit of scrolling through large arrays, instead of learning numpy’s slicing and shape concepts, or functions that detect e.g. the index of NaNs... I think that was mainly my point.

I love the idea of Spyder. Particularly as a replacement/alternative to Jupyter notebook style workflows. The last time I tried to use Spyder to connect to a remote kernel, there was no way to interrupt a remote kernel. I understand that this was an inherent issue with the kernel APIs (Jupyter has it's own watchdog process that is responsible for sending signals to the kernels it monitors). Is this workflow better supported now? (ie. using Jupyer to connect to a remote kernel that has GPUs?, and having an easy way to interrupt / restart the remote kernel if it hangs?

Unfortunately we haven't had time to address that, but it's high in our TODO list. We hope to do it this year.

The last time I tried Spyder, I think around 2015-2016, it really struggled with large datasets. A lot of pausing and hanging that wouldn't happen at an IPython prompt. Is that better now?

Yes, we definitely improved on that front. Spyder should be more responsive now.

You should give it a try with our new installers for Windows and macOS, which save you the trouble of installing Anaconda: https://github.com/spyder-ide/spyder/releases/tag/v4.2.1

I understand why Spyder. But they could also have called it spIDEr

pIDE PYper (just thought of that!)

Haha, the original project Spyder is based on was called Pydee

Spyder is great. More minimalist than Pycharm, but it actually allows you to run multiple Python consoles in parallel, which is strangely absent from Pycharm.

I generally find it really hard to use full on IDEs like PyCharm for doing more investigative/research kinda work in Python, which is where Jupyter/Colab/Spyder are great at. It's just a very different flow, creating a large application versus doing analysis/research.

Pycharm Professional edition has a Scientific project type, which I was hoping someone has experience with.

I like to do a bit of a mix: write the library functions in Pycharm, then import and call them in Jupyter Lab. I used to try to do implementation too in Jupyter Lab but that got messy very quickly.

Personally, I can't stand Jupyter for research because the introspection capabilities are so poor and limited compared to working in something like Spyder, Pycharm or VsCode. Moreover, I much prefer having interactive matplotlib plots for data exploration in a Qt/Gtk window rather than the inline matplotlib plots in Jupyter (though you can sometimes get it work to pop up windows for plots, but you still have a more interactive plotting experience in Pycharm/Spyder/VsCode)

As someone who loves Spyder and used it a lot many years back, I've had no trouble doing all my scientific/algorithm prototyping in Pycharm while benefiting immensely from the astoundingly superior code intelligence/auto-complete/type-hinting in Pycharm. There is also a "Code Cell" plugin you can download for Pycharm that basically gets you to parity with Code-cells in Spyder/Matlab.

For me, the excellent Python shell in Pycharm that supports multiline copy-paste, tab completion, and function-signature overlays makes prototyping algorithms and ideas much faster. That paired with the excellent history browser in Pycharm lets me easily grab relevant bits of code and promote them into a script or function and go from there. And then if you are trying to debug any algorithm, Pycharm is far superior on that front and I am far more efficient at debugging and fixing problems in Pycharm than in something like Vscode (though I do recall Spyder's debugger being pretty decent since it supports integration with IPython)

There is a steep learning curve with Pycharm initially, but imo it works just fine (and for me at least, much better) in comparison to other things like Spyder/Jupyter.

I would definitely recommend Spyder to most non-software engineer Python users, especially if they are in Data science. Nothing by <3 for Spyder's awesome data viewer widgets.

> Personally, I can't stand Jupyter for research because the introspection capabilities are so poor and limited compared

definetely agree. I prefer to work with code cells and prefer Spyder for being less bloated and more responsive than Pycharm, for having multiple iPython consoles and a nicer Plot window when doing DS work, though.

To me, PyCharms only shines when refactoring and testing code to make it production ready.

In fact you can do this, at least in the Professional edition of Pycharm. How to do it isn't especially obvious though. On the left hand side of your python console with be a +. Click that and it will give you a new one.

I may be misunderstanding, but Pycharm has supported multiple Python consoles (and running multiple files) for a while now.

Always enjoyed spyder for its variable explorer.

I tried to use spyder on some projects, but the constant crashing without restoration of my work got old very quickly.

This was fixed in Spyder 4 by providing autosave functionality to our editor. You should give it a try again.

I've been using Spyder for my whole DS master and rarely experiences crashes on Ubuntu, unless I've managed to go OOM. That being said, I've been using Spyder 4 via pip since the 4.0.0 release while Spyder 3 was still the default on conda. Autosave is definitely useful.

Saw this thread and instantly installed Spyder on my Windows PC. My display is 4k 15 inch with 250% scale. The fontsize of UI didn't scale and is too small. Well, like many other Windows apps that can't handle high DPI well, I'll check later when it's polished.

I use spyder on my 4k 15 inch display. It works 100% fine. I think I may have set scaling in the options to "auto" or "hidpi".

Thank you! I only opened the software once and didn't check settings. Enabled high DPI and it worked. Wondering why they doesn't enable it by default.

Because it'd make everything look huge in non-high DPI displays. It's a trade off, but most people still use them.

I loved using this until before I tried using PyCharm and VSCode and switched to them instead due to irregular updates while using Spyder within Anaconda on my Mac. Also, the UI started feeling a bit in my way but that's just me I think.

Yeah, updates in Anaconda can be bumpy. That's why we decided to create our own installers for Windows and Mac: https://github.com/spyder-ide/spyder/releases/tag/v4.2.1

The link to the 'editor' page 404s. All the other stuff in here sounds pretty nice, but I'm curious how the editor experience is (or if I can use something like vim instead)

It's okay. It's a lot like RStudio and Matlab. I consider a bridge between Notebook work and IDE work.

A vim plugin exists, but it's very limited.

Right, we haven't had resources to invest on it, so it's been developed by the community.

It'd be nice if we could use Neovim for text manipulations and show what comes back from it in our editor. If only days had 36 hours...

That would be amazing. I like Spyder a lot, but for big/complex editing tasks I sometimes have to switch to Vim.

> The link to the 'editor' page 404s

We just fixed this. Sorry for the inconvenience.

I really like spyder for it's Matlab like IDE but I wish they handled virtual environments better. I also don't like that I have to install their kernel in the venv to use it.

Hey, thanks for the feedback! We plan to improve our support for virtual environments in the future. Also, it's really necessary to install spyder-kernels in every env to make the variable explorer and debugger work for it. However, that could be by us handling virtual envs and installing spyder-kernels on behalf of the users.

That would work too. Thank you

I love Spyder's UI but it's way too slow to load on a computer with an HDD.

We're working to improve that

Has anybody used Pycharm's Scientific project type for data analysis?

Hey, a PM of PyCharm here.

Justin case someone will find it useful and relevant. Our team is currently working on a brand new IDE (based on PyCharm) for data scientists.

A big part of what we're doing is convenient support for Jupyter notebooks within the IDE. This new IDE is currently under development but already provides the support for notebooks similar to how the native web Jupyter notebooks work – cells and outputs under each cell. The IDE's functionality will include SQL, Jupyter, Python, and R support.

If you'd like to try the early EAP builds of this new IDE with the new notebooks support, please fill out this short form: https://pages.jetbrains.com/pycharm-data-science-insiders

Once you’ve confirmed your participation, you’ll get a detailed email with instructions on how to download the early builds and details about how to share your feedback.

Will be also happy to answer questions if any.

Interesting!

Curious as to whether you'd say the approach you are taking is closer to Jupyter's binary cell approach or RStudio's Notebook's text-based approach.

Thanks!

Our current approach to is very similar to the native Jupyter experience where notebook contains multiple separate cells. You can manipulate these cells with shortcuts or mouse. It looks very similar to Jupyter or JupyterLab. At the same time, if you open a Python script, you'll be able to use cell delimiters (aka `#%%`).

Good to know, I'll always prefer working with cell delimiters like in Spyder over working with Jupyter Notebooks.

I've also tried out PyCharm Professional, but the experience with cells and iPython was just worse for me than in Spyder - so I'm only using PyCharm for refactoring right now. I think the main problem was that errors/results from cmdstanpy would simply cause PyCharm to crash/hang instead of getting handled properly like in Spyder.

Would you happen to know if Spyder renders interactive Plotly plots in console or inline? I know that I can open plots in a browser window, but it is not as convenient.

The top right pane has Variable Explorer,Help, and Plots tabs. Plotly plots using the png or svg renderers will be displayed there. Not interactive though. VS Code on the other hand can do what you're looking for if you specify the "plotly_mimetype" renderer for your plots.

Thanks! This is a bit of a deal-breaker for me because I use Plotly a lot for EDA.

My ideal IDE would combine PyCharm's variable explorer and interactive debugger with Jupyter Lab's plotting, widgets, and the ease of rearranging cells. Oh, and I really need a duplicate line shortcut, which PyCharm, Spyder, and RStudio all have, but Jupyter doesn't for some reason.

I know many people use VS Code for data science, but I haven't used it much. If you have time, would you share your VS Code setup?

Cheers!

As a professional data scientist and daily user of PyCharm Pro at work, super interested in this. Just signed up!

For what it's worth, our team is generally not a huge fan of notebooks, but I get why people like them.

Thank you! Will send all the info within a couple of days.

Awesome! I know myself and a few team members would be happy to share feedback. PyCharm is amazing, but we've always wished we could have something a little bit more suited to our needs.

As a daily Jetbrains user it sounds great!

But - no Julia support?

Julia is on the radar. I'd love some time in the future to include it into scope.

Just submitted my application. I use PyCharm and Jupyter on a daily basis.

Cool, I see the request! Will send you all the info the next week already.

Hey, a big fan of PyCharm here!

I played with PyCharm's implementation of notebooks in 2019. At that time, it felt like you didn't do any UX research and testing with professional data scientists, including data scientists across a range of age groups. For this reason, I stopped using PyCharm for analytical data science work.

I am mentioning it here, just because I would love for your new project to succeed. Good luck!

Thank you so much. I totally understand what you mean. And this is exactly what we try to improve. Anyway, always happy to hear feedback and pass it over to the team.

Is this written in Python? What does it use for the display?

Yes, it's written in PyQt and we integrate with a lot of libraries in the Scientific Python ecosystem (Pandas, Sympy, IPython, Matplotlib, Scipy, Cython, etc).

Every link at the bottom is invalid (404)...

Fixed now, sorry for the inconvenience.

I'm sorry, I'm sure Spyder has lots of merits (and I'm sure the devs are wonderful people), but I've spent so much time fixing fellow academics' Python problems that were in the end rooted in "I don't know what I'm doing, so I installed this Spyder thing and it has now taken over every aspect of Python on my computer" that I feel a visceral negative reaction to the name in my body. Not Spyder's fault, of course, but nonetheless.

(Spyder maintainer here) I don't understand your comment. Spyder is like any other regular Python package, so it can't take over your computer. We have some customizations to run code, but that's to provide a better experience to users.

Things can get very messy when installing Spyder via pip and then trying to upgrade stuff. Distutils tends to screw these updates up. I've nuked my local Python packages recently due to that mess and made a clean reinstall, but I guess that's not what you'd like to do if you have many Python packages other than Spyder installed.

Agreed, installing Spyder with pip can lead to a lot of headaches. That's why we don't recommend to use it and suggest Anaconda/conda instead. But that solution also became problematic during the last couple of years (conda is too slow or throws back a huge dependency errors).

That's why we decided to create our own, self-contained installers for Windows and macOS: https://github.com/spyder-ide/spyder/releases/tag/v4.2.1

Since they are completely isolated and don't come with pip, they can't be broken. And you can connect them to other Python environments by following these instructions: http://docs.spyder-ide.org/current/faq.html#using-existing-e...

Please use carefully if you have aracnopyhopia.

The naming seems pretty poor for me given that I'd assume that spyder would be a webcrawler written in python.

If you use this, you are a junior dev - embarrassing

Sounds like a junior dev attitude.

If you say things like this, you don't belong on HN - embarrassing

Guess I’m junior then. I learned how to code via scientific coding so I love Spyder for its simplicity. When I want to work deeply on a difficult problem and want to quickly flesh out ideas it’s my go to because it sandboxes really well and I can iterate much faster in it.

I use more advanced IDE when integrating the code I developed in Spyder into a larger code base. I’m sure many will criticize my workflow but I’ve tried doing other ways and this works best for me and I write less buggy code this way.

Which advanced IDE? Eclipse?

I am really liking VSCode as my IDE in part because I can almost use it like I use Spyder.

I used Eclipse years ago and am neutral on it. I can use it if I need to. Just wouldn’t be my go to. I can say same for PyCharm; it’s very powerful but me and PyCharm never really jelled (lol!).

Ignore anyone criticizing your workflow. Tools like Spyder and Jupyter can be great when working with datasets in a way that IDEs aren't quite suited for.

Variable explorers and integrated charting/plotting can be fantastic for visualizing and understanding your data.

Cells in Jupyter are great for quickly iterating on data cleaning code.

When I was part of a scientific applications team at a past employer (state agency), we used Jupyter heavily while developing the core code for the data processing pipelines.

We were reading and processing data from instruments based on (bad) docs and notes from a former employee who reverse-engineered the format, so there was a lot of edge-case finding and testing. The notebooks fleshed out with comments and notes in Markdown upon completion and were saved within our repo as interactive documentation, demonstrating the more complicated processing steps in a far more approachable fashion.

What wisdom! Thank you senior dev person

Aww snap - guess I'll go back to programming everything in nano then.

This comment is embarrassing. The tools a person uses is not an indicator of their skill level and thinking so is only holding you back, not the person using them.

This is more for using python as a science calculator, not for writing software.

Some of us drop out of the womb as senior devs and tech leads I guess.

Move right along, folks, don't feed the troll.

If you judge someone by the tools they use, you are a junior dev. Embarrassing.

Stop and consider that there are different tools for different purposes. Exactly the same way Word, Publisher, Photoshop, and Sublime or Vim are different from each other, so are things like PyCharm, VS Code, Jupyter, and Spyder.

A "full" IDE like PyCharm is designed for application development, including deep introspection of your code, deployment automation, tool integration for things like preprocessors (such as sass/less), and integrated support for VMs and Docker. This is overkill if you're writing a simple script, but helps for larger projects.

Something like VS Code is closer to PyCharm, but might not be as deeply integrated for any given language. It does provide a more consistent environment if you tend to work in multiple languages, which can help you avoid some of the context switching between langs.

Jupyter and Spyder serve a different purpose. They aren't the kind of thing you'd (usually) write a full application in, but it's exactly the kind of thing that would be useful to data scientist, or someone trying to explore a library. The re-executable cells in Jupyter are quite useful for quickly iterating on data cleaning code, since you can re-write and rerun a series of cells without rerunning expensive calculations or data loading.

I haven't used spyder in years (2008/9?), but it was a great way to put my fledgling python skills to use back then. I was performing a lot of data analysis on radio astro data in Matlab, but we only had so many 'roaming' licenses for our lab. Using numpy and spyder allowed me to practice my python and get my data analysis done outside of the lab.

So consider the fact that not all tools are suitable for the same thing, and that a 'senior' dev considers what tool is right for the job at hand.

Reply on news.ycombinator.com