The second thing is the presentation of the plots: it's really handy to have multiple interactive plots laid out right next to the code block that produced them.
There's nothing that requires that this is done in a web interface, but no-one seems interested in making a non-web version (IIRC there was a QT version in the very very early days of ipython notebook but it's long dead). One big thing that any replacement will need is the easy ability for the code to actually be running on a remote machine, something that's trivial for a web interface but requires active effort for a native client (and makes a lot of interactive chart work more difficult: certainly there's currently no other matplotlib backend which can do it).
I'm fairly sure a better interface than jupyter is possible for data exploration, but I haven't found one yet since there's not really anything else that gives you the two features above.
(one thing people often miss about notebooks and the kind of work that go into them is that the code itself isn't actually really hard code. It's 100% about interacting with the data and the algorithms running on it. Working in one isn't much like software engineering)