Python Tools for Visual Studio
hanselman.com
hanselman.com
Disclaimer: I used to work on IronPython.
That's really nice. I wish there would be something comparable for debugging JNI/Java code.
It is mentioned.
1: http://www.youtube.com/watch?v=wvJaKQ94lBY&feature=c4-overvi...
To accomplish that we require that we have symbols for the Python interpreter (which are typically available on python.org). The debug engine uses a combination of the native equivalent of sys.settrace, strategic breakpoints, and using the symbols to walk core Python data structures. There's a bunch of fun tricks for evaluating code when stopped at a Python frame and creating strings, ints, etc... when users type them into the watch window.
If you'd like to dig in more the source is at http://pytools.codeplex.com/SourceControl/latest under Python\Product\Debugger\DkmDebugger.
Here's a screenshot of a debug session with a custom host process written in C# running Python code that in turn uses a C++ extension module: http://i.imgur.com/IDPsWUu.png
Stack walking works by rewriting the native stack. Basically we just let the native stack walker do its job, then strip out all python##.dll frames except for PyEval_EvalFrameEx. For that one we read the "f" argument and then parse the PyFrameObject to which it points, and replace the frame with our own using data from that frame object.
We parse the data structures ourselves, but use symbols to do so, which is why it'll work on any CPython build with symbols. This is why you can inspect objects even when process is stopped somewhere in native code, and it's impossible to eval using the interpreter.
We still use sys.settrace for breakpoints and Python stepping. Python-to-native stepping works by setting native breakpoints on code paths inside Python interpreter that can potentially invoke some user code (e.g. type_call invokes tp_init, which may point to a user function). When those breakpoints are hit, we eval the function pointer and see if it's outside of python##.dll, and if so, then set another breakpoint there. When it's hit, your step is complete.
So no, there are no special hooks added to the interpreter. We do end up relying quite a bit on internal implementation details, since we need to do things such as set breakpoints inside static functions, and read their arguments. However, because we use symbols for that, this works on any CPython build, including debug ones, and even customized ones so long as you don't rename anything that we rely on. E.g. adding a new field to PyObject is kosher, but renaming ob_type to something else would break us.
You guys deserve some recognition. I'm not too frequent a user lately, but I'm continually impressed.
But i have to admit, nothing beats visual studio in awesomeness / completeness
6 months into it, I all but stopped caring, and overall, I felt like it actually created problems for me. But I'd like to understand what problems others feel it solved for them.
Running my CL environment on Gentoo feels like the future; a not-quite-there-yet-but-still-awesome future; running VS on Windows feels like kiddie toys that haven't gotten around to growing up yet.
And that's a weird feeling when you realize the dollars, time, and research poured into each environment. :-)
Regular text editors just feel so clunky next to the cohesiveness of VS/Resharper or Intellij.
At some point in time, Emacs is going to have get overhauled to join the 90s, much less the 21st century.
And I understand that CL is a special case with Emacs, but it's still clunky as hell compared to what could have been really awesome environments (think Smalltalk environments), but with lots of resources pumped into the development.
You've kind of hit the key here, Visual Studio alone is decent, but Resharper is really what makes the experience for me. I don't know a single .net developer who could live without it.
It stinks on our 1Mloc C# project to the point you have to disable it or it crashes VS every 5 minutes.
Both Smalltalk and VS+Resharper have great refactoring tools. Both have great debugging tools (and in Smalltalk I can even do a lot of development right inside the debugger and watch how the code is reacting while I'm writing it). In the CL/Emacs setup debugging is so bad most people recommend not to even bother; just use tracing. And I can't understand why it needs to be like this, CL is closer to Smalltalk than it is C# (that is, CL is a "live" language as Smalltalk is or at least it very much can be. Even a compile-only CL like SBCL).
The other issue is project management. In Smalltalk it doesn't really come into play as much since the image is your "exe", but despite this Smalltalk has great tools for organizing classes, methods and so on (and Dolphin's way of letting your image be a "solution" and you make individual exes from that that drop everything they didn't use was absolutely brilliant). In VS, I create my assemblies, make dependencies and so on and the IDE just handles all this for me.
Contrast that with CL/Emacs: CL has ASDF files that can manage all your dependencies for you (people complain about it, but I've used them for several years and never had any problems) but you have to manage all this stuff manually. If you want a nice way of saying "create new package" or "create a new file that is in this package" you'll need to write all that handling, UI, etc. yourself. And after you write it all, what will you have? The interface will be hideous no matter how you do it because you're restricted to just text. In Visual Studio you can have the disapearable side bars approach, the "old school mac" approach of different info windows floating all over the desktop, etc. With emacs you'd be restricted to doing that stuff via ascii art or popping up more windows with ugly ascii art in them.
The only way I could see CL/Emacs as the future is if we're talking post-apocalyptic.
I think that lies at the heart of our disagreement. I don't find text restrictive, I find visuals and GUIs restrictive.
What specifically do you miss about debugging?
With visual studio, you develop for websites (MVC), apps (XAML based - WP, WPF, Silverlight, ...) and that makes it great.
I recently got back to android because of Android Builder and i'm still struggling with it (1 day or so). The visual studio experience is way smoother.
I have to admit, my little node.js experience with github and heroku felt awesome.. But it lacks a good IDE-experience and that is where Visual Studio really excels.
But command line tools can be integrated as wel in VS ;-), Azure is awesome to if you experimented with it.
With .Net you can integrate almost any language and Python is one of them. I recently talked to someone who does his PHP work solely on Azure because of the platform.
I guess you have to get in the .Net platform fully (Linq / Entity Framework / DDD (Repository / Unity Of Work / ...) to really comprehend what i'm saying.
Perhaps for some, i added a new threat on https://news.ycombinator.com/item?id=5981257 for more information on "Enterprisy" (can't name it better) apps on .Net.
http://visualstudiogallery.msdn.microsoft.com/07d54d12-7133-...
At the moment, I use PyCharm for python projects because I'm usually working on OSX (and prefer the local unix-y environment instead of the Windows-virtual-machine thing.) However, I started with PTVS years ago and really like it.
MSFT adding IDE support for Python is a good thing -- as developers, we need more options, not less.
They're never gonna go it. They stand to win nothing from that.
Another possible cause can be extensions. Not that VS doesn't have its own problems of 'doing too much work on the UI thread', but there are TONS of callback points that third parties can hook into, and lots of times people do very ill-advised things on the UI thread, leading to poor responsiveness.
We monitor it actively (via PerfWatson[0], you should install it if you want your perf problems brought to the attention of Microsoft, it should have low overhead at runtime, only kicking in when the UI thread is already unresponsive), but of course with a product as large and complex as VS, and with said product being extensible, inevitably customers run into stuff we never see internally, so we rely on getting repros/data from them to understand what is going wrong.
[0] http://visualstudiogallery.msdn.microsoft.com/fa85b17d-3df2-...
Thanks for the link. I'm happy to add it.
But as you and the parent pointed out, I've also seen huge improvements in build times with VS2012.
And with VS2013 we finally get edit-n-continue on x64.
/sarcasm
Making VS on Mac would earn them no money directly and would only encourage the creation of more Mac apps. At best, it might increase the takeup of .NET/Mono on Mac, which might mean more .NET apps, but no money since .NET runtime is free. I'm not seeing the incentive.
At this point both XCode and VS are pretty equal but XCode I also like and have no problem with. Both are unique to the OS though. Both are also extremely bloated but needed for C/C++ dev. Eclipse is also a bit cludgy. That python to native screenshot on the site looks intruiging.
I do use MonoDevelop on Mac for Unity dev and it does support debugging but would love to use VS on Mac one day.
All in all when I do python dev I like less in my IDE at times, just a good autocomplete/intellisense and I usually despise integrated source control into IDEs preferring to do it with outside tools such as terminal/cmd, sourcetree or tortoises. SCM integration in IDEs always seems to kill performance and gets out of sync, especially in the latest MonoDevelop (startup indexing kills the usefulness - http://answers.unity3d.com/questions/356129/monodevelop-perf...).
ButI agree. VS is an awesome IDE and the more operating systems it would be available on the better.
(I've had MVPs yell at me for discussing data analysis with Python, even on Windows.)
The setup.exe is both the installer and the package manager.
I prefer Windows for a dev platform (really, I do), but I've only once written a webapp that was hosted on Windows. I'd much rather prefer unixy servers.
How does it compare to VS Express? Why would there be two different free Visual Studio offerings...just to keep things locked down and rule out support for add-ins?
Anyway microsoft doesn't provide support for their core languages (C#/C++/VB) for the shell, so the main consequence is that, in the shell you basically get all the languages microsoft /doesn't/ directly support in visual studio, but nothing else. Whereas in the express editions you have an IDE you can't extend, but each edition supports one of microsoft's core languages.
Our PTVS + integrated/isolated shell basically gives you a virtual "Python Express". However recently msft has moved away from the "Express" model to "Scenario" models. ie, VS for Phone, VS for Desktop, VS for Web, etc. and each includes all the tools and multiple languages you might need for that workload. We're hoping to include PTVS into the desktop & Web scenarios in the future.
[disclaimer: project lead]
Yup, try PyCharm, PyDev, Wing, Komodo, Spyder, ... I did a comparison last year (which I can't unfortunately share :( I can ask) -- bottom line, all pretty darn good, each has its strengths and weaknesses. Take away: it's a great time to be a Python dev on Windows (finally?).
WRT to PTVS itself, the key feature is being VS based; the mixed-mode debug for example takes advantage of support in VS. But at the same time, if you're looking for a cross-platform, very very very lightweight tool, this isn't the IDE you're looking for.
In other areas like code editing, the differences are often qualitative. For example all Python IDEs have code completion, but try entering this in your favorite one:
def f(x):
def g(y):
return x + y
return g
a = f(1)(2)
b = f(3.0)(a)
c = f(u'a')(str(b))
d = (a, b, c)[input()]
And then see what completions it gives you for a., b. etc.It would be interesting if some independent third party would do a detailed point by point study and published it.
As far as I can remember, there was some resistance to going through that route before (v1.5 didn't have this, I think).
(esc;qgrep -rn thing .) is faster for me than reaching for the mouse.
Have you tried it in VS 2010+? You might be surprised. I've seen it correctly handle polymorphic Boost lambdas, and that's a rather impressive feat of TMP. Also, I've just tried the scenario that you've described (if I understand it correctly - "Find all references" on a class template), and it does list all the explicit instantiations in the project.
(On another note, Xcode. I'm normally pretty negative about Xcode, because it's mostly awful. But I believe its code navigation and code completion is done in a similar way - only using LLVM, of course - and that part does work extremely well. If anything, it's actually slightly better than Visual Studio, because its navigation shortcut is much better about giving you a popup list of all possible reasonable destinations, e.g., if you try to go from a member function call to its definition when that function is virtual.)
To be specific, what they did was take the EDG C++ front-end, and use it to drive all code intelligence features (code completion etc) - see http://blogs.msdn.com/b/vcblog/archive/2009/05/27/rebuilding.... That's why it can correctly handle all templates, macros etc.
Here's a nasty trick that I love to play on unsuspecting C++ IDEs:
template<int = sizeof(void*)> struct foo;
template<> struct foo<4> {
template<int> struct bar {};
};
template<> struct foo<8> {
enum { bar = 0 };
};
float baz;
int main() {
foo<>::bar < 123 > baz;
}
The nastiness here is that the line of code inside main() should be parsed differently depending on pointer size on the target architecture. If it uses foo<4>, then it is a declaration of a local variable named baz, of type foo<4>::bar<123>. If it uses foo<8>, then it's an expression involving two comparison operators: ((foo<>::bar < 123) > baz) - referencing the global float baz.Now, if you try the above in VS (2010+), you'll see that it will parse this correctly and give you the proper type info for baz (and "Find References" etc), which will change if you switch project configuration from "Win32" to "x64" and back.
VS2008 C++ build system with CMake.
As an example, the Zeus IDE automatically generates ctags information for all files in the current workspace and it has this feature.
The ctags approach offers and additional bonus in that you can also do a wildcard search.
NOTE: I'm the author of the Zeus IDE.
Get all the editing capabilities of Vim, but don't lose integrated debugging, code completion, code nav, type info popups, etc. that an IDE provides.
Can somebody point me in the right direction?
Integrated package:
https://pytools.codeplex.com/downloads/get/697389
Isolated Shell downloads:
http://www.microsoft.com/en-au/download/details.aspx?id=3066... http://www.microsoft.com/en-au/download/details.aspx?id=3067...
good luck
Using RubyMine now as a Rails dev, after years of Visual Studio use as a ASP.NET dev, and having the familiar ReSharper-like tools and interface in RubyMine is awesome.
RubyMine is an incredibly powerful IDE (like most of the JetBrains IDEs), with lots of Ruby specific goodness.
Never used it myself though, so I can't speak for it.
:)