Benchmark: C++ vs C# | Hacker News Reader

Benchmark: C++ vs C#

codeproject.com

78 pointsprog35 comments

The results seem conclusive: if you're developing software for the desktop, and (like me) you are equally skilled in C# and C++, it's safe to take advantage of the simpler syntax, rich standard libraries, stellar IntelliSense and reduced development effort that C# offers.

With the glaring exception of numeric code, which is the most crucial thing for a lot of apps that are now written in C++. You'd be crazy to write a native CRUD app in C++ now but most of the big C++ apps I can think of do heavy work in numerics (Photoshop, Ableton, Maya, etc).

For these the best approach is to write the whole app in some higher-than-C++ level language with performance critical parts (usually less than 10 percent) in C/C++.

That's often a good approach, but sometimes the overhead of marshaling data between the layers can kill you. For instance, Google warns people about assuming the apps written using the Android NDK will be faster, partly because of JNI overhead.

http://www.getpaint.net/

I'm not saying paint.net is a replacement for photoshop. However, for me, it's in the "good enough" category.

http://en.wikipedia.org/wiki/Paint.NET

"Paint.NET is primarily programmed in the C# programming language, with small amounts of C++ used for installation and shell-integration related functions."

Again this is good enough for a lot of peoples image manipulation. That area of "good enough" is getting larger all the time.

Paint.NET is fine for small images, but performance really falls off a cliff when images get larger, whereas Photoshop hardly misses a beat. Which I guess is kind of your point, for doing 'normal' tasks under 'normal' loads C# is fine, but if you have to go beyond normal, then you really need the control and speed of something C-like.

I don't know, to me it seems there's a likely chance that Photoshop's implementation of handling large files is just better, and if Paint.NET took a similar approach maybe the performance might be more comparable. By this I mean stuff like not having to go through the whole file to generate a zoomed view of it, etc. Of course this is just an assumption based on the fact Paint.NET isn't in the same class of application as Photoshop, so I wouldn't expect handling massive files to be a priority to the authors. That and Photoshop has been around a lot longer and Adobe probably has more resources to throw at potential problems like this. Either way, these are just guesses to consider before assuming the difference is caused by the language.

Certainly C++'s niche is getter smaller every year. It just happens to be a niche occupied by the most interesting (IMO) native apps.

The flood of low-powered mobile devices out there seems to have given it some new legs though.

The poor guy is taking a beating in the code-project comments which is a pity since he obviously spent some time putting it together. In a comparison like this, you are always going to get people on either side of the fence claiming the results are invalid because there was some compiler switch or code construct that was missed.

The language-agnostic version of this article: you can write slow code in any language.

In the article he illustrated that with a Microsoft C++ library example (a heap?). In the benchmark he fixed it by writing his own. You can do that in C++

I have heard that you can expect a 10-15% difference in most cases. However, it seems that there are just some arenas that C# wont quite measure up. I'm thinking multimedia applications would not scale well. Mainly because there is no way via Microsoft's .NET implementation to access SSE or other processor intrinsics. I know that mono announced that this a year or two ago, anyone have experience with it?

You're thinking of Mono.Simd. I've toyed with it a bit, and it seemed pretty fast, but I haven't yet had the chance to write something "real-world" with it.

Miguel de Icaza wrote an article with some preliminary benchmarks when they announced the release a few years back: http://tirania.org/blog/archive/2008/Nov-03.html

We've been aware of how slow CF is on Windows CE for a while. Here are some benchmarks we have run:

CuWin 3500, CE5, NS Basic/CE 8 NS Basic/CE: 26529 loops/second Microsoft .NET: 133 loops/second

Acer beTouch E101 Windows Mobile 6.5, 528 mhz, NS Basic/CE 8.0 NS Basic/CE: NS Basic/CE: 16669 loops/second Microsoft .NET: 3060 loops/second

CuWin 5500, CE6, NS Basic/CE 8 NS Basic/CE: 14394 loops/second Microsoft .NET: 764 loops/second

i-mate 400mhz, WM5, NS Basic/CE 8.0 NS Basic/CE: 12632 loops/second Microsoft .NET: 1290 loops/second

Running the same test under NS Basic/App Studio, an iPhone 4 scores 282,674, Nexus One is 403,000 and the Motorola Xoom is 718,484. Plainly, Windows CE.NET has a long way to go!

He did not say if optimizations were turned on for the debug builds, and I'm not familiar enough with Visual C++ to know what the defaults are. You can still have DEBUG defined so that asserts are compiled in when optimizations are on. However, looking at the results, I suspect optimizations were turned off, which makes those results meaningless.

The benchmark numbers for the Compact Framework on ARM make this a lot more interesting. I used it on PowerPC (the XBox 360 .NET runtime used by XNA is based on the Compact Framework), and I was uncertain whether the issues were related to the PowerPC architecture. It looks like the Compact Framework is just bad.

I wonder if this is the same thing that's used in WP7. Bad news if it is, I guess...

The compact framework has traditionally been horrible and kind of a joke. I hope MS has either ported the proper .NET framework or greatly improved the compact framework for WP7, if not it makes the choice of no native code support even worse.

I think C# is in the category of "fast enough".

When I was working through project Euler (http://projecteuler.net/) with various languages, the only one I really had to optimize was Python (vs C/C#/Haskell).

I really like Python, and for most stuff it's fast enough. Just saying the real penalty (that I noticed) is a scripting language, _if_ it's heavily arithmetically constrained.

Some (non-trivial) games in C#: http://www.arcengames.com/w/index.php/aiwar-alienbundle

http://murudai.com/solar/

However, it's possible and even practical to also write games in Python. See the pygame project.

This article raises interesting questions. He has done low-level tests which do not read much from the data or instruction pipelines, so these tests show compiled MSIL can be better optimized than C++. C++ is prone to accidental performance deficits from problems like aliasing, so that makes sense.

Now I want to see higher-level tests, where algorithms reading significant data or application-level benchmarks are run. The faster the CPU core, the less penalty there will be from C# compilation. The problem is that it would take lots of time to translate something like WORF, GAMESS, or other standard numerical benchmarks.

I think he hit the nail on the head referring to c# programmers in general not being as "performance conscious" which leads the misconception that c# is a much slower language to use. I've worked in many places and have come across the same thing. As a c++ programmer I have also worked with c# coders and am often shocked at their slap-dash approach to software engineering. I'm not saying they're all like that but it seems to be a common trend in my opinion!

Do you think those programmers would suddenly become "performance conscious" if they switched to c++? What I'm saying is that I don't see the point in referring to them as "c# coders" they could be using any language just as badly.

Yeah I'd not label it a language issue but rather as you've already described it, as an implementation issue.

Maybe that's true, but I'm working on a Windows Phone 7 app right now that struggling to hit its performance targets. The major culprit? Silverlight's data-binding. Microsoft shipped the framework without shipping high performance controls necessary to meet the certification requirements of their app store (ListBox, I'm looking at you), so while our code might return the results of the list in <20ms, it takes another 800-1500ms for the results to be rendered. It's frustrating.

Have you tried using the virtualizing listbox?

From my research, it looks like the Listbox enables UI virtualization by default. It didn't help. I ended up finding some helpful soul's NavigationListControl and limiting the number of results until the user taps a "show all" button. It helped a little. I think the final solution will be pre-allocating the first N data items with empty strings so that the initial cost of the Listbox is incurred at initial-page-layout-time rather than at keypress-time.

I was solving similar problem but with a datagrid. I ended up adding my own scrollbar (so that I have better knowledge of what to preload) and populating the datagrid with an array of references to a single "Loading" object. The reduction in allocations sped it up a lot as well, I guess.

I think one of the big reasons is that most C# developers are ASP.NET CRUD developers, where, most the time the bottlenecks are in HTTP i/o and in waiting for the database results. Optimizing C# for these applications is a waste of time.

You will find that a greater proportion of C#/Winforms, console apps, windows services etc. developers are performance conscious.

2 things:

1): Most projects that I've used c# for just aren't performance constrained. What's more important for us is the productivity dimension. We have _lots_ of hardware. We have a few programmers.

I've actually just finished the _only_ c# program that had a performance aspect. It was/is an in-house search engine. I profiled it, fixed the hotspots and it's fast enough.

2: MSFT Marketing. Linq|dynamic are heavily sold by Microsoft. You'll have to read Joe Duffy's blog for actual performance advice: http://www.bluebytesoftware.com/blog/2010/09/06/ThePremature...

True enough in general, but that cuts both ways -- I've worked with a number of longtime C++ guys that tend to optimize early, write long unreadable methods and seem positively allergic to standard OOP architectures, much less design patterns.

This benchmark is totally wrong as you can read in the comments of the page.

Basically he didn't turn on all optimizations on the C++ program and used the "slow but secure" version of the Microsoft STL.

Also his GenericSum uses array in C++ and lists in C#, so the C++ has to reallocate at each append.

I could go on for long, I appreciate the intent, but all you need to know is that Microsoft gave up on using C# everywhere because it's damn too slow.

>> but all you need to know is that Microsoft gave up on using C# everywhere because it's damn too slow.

Got any proof of that?

Not proof but, fwiw I read an article saying that.

When they started Vista they wanted to go .NET all the way and had to go back using more native code.

>I could go on for long, I appreciate the intent, but all you need to know is that Microsoft gave up on using C# everywhere because it's damn too slow.

As a data point, a friend of mine works for Microsoft on one of the Blend products. He's using C# for at least some of his work.

I don't know much C#, but it seems to me there are so many ways of writing "numeric" code in C++. Using the STL algorithms is one, using C-style looks on POD arrays another, and using an expression template math library like blitz yet another. You can't really say that any one of them, except possibly the expression template method, is uniquely C++.

Even though the benchmark is incorrect, the flawed conclusion should lead you to see that Java is just as good a solution as C# (possibly better since it runs officially on more platforms).

Reply on news.ycombinator.com