C# 11 Preview Updates – Raw string literals, UTF-8 and more | Hacker News Reader

C# 11 Preview Updates – Raw string literals, UTF-8 and more

devblogs.microsoft.com

125 pointsphilonoist88 comments

I have been working with .NET since the heady, early days when the web framework was being called ASP-NG (next gen) and C# had not yet been given it's official name. I've written millions of lines of C# but sometimes I think it would behoove me to start over with C# as though I had never seen it before because there are things that have been added to the language in the last few years that I had no clue were there. Is there a course or book or blog post series along the lines of "C# for people who are writing C# like it's 2010"?

I was in a similar position, worked with C# from the beginning and recently switched to non-MS stacks. I found official documentation [1] easy enough to follow - I started with "What's new in C# 8.0" to the latest version.

[1] https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csh...

I'd start with just learning about all the new features by reading the updates in reverse chronological order.

John Skeet's "C# In Depth" series presents how the language changes from one version to the next with code examples.

C# In Depth is indeed a good read. Unfortunately, the C# language is getting more and more complex, especially when you compare it to Java.

Sure, but it's backward compatible too. You can write C# 4 style code while in the latest version of .net/visual studio.

Java is catching up, and what appears easy at the language level, gets extra knobs at the runtime and lower APIs.

I don't know if they have videos beyond 8 (the highest available to me in unity) but I founds Mads Torgersen own walkthroughs very good and succinct. https://m.youtube.com/watch?v=VdC0aoa7ung

Edit - sorry c#9 seems to be available in unity now

Resharper helps a lot with that, making useful suggestions.

I finally started using ReSharper for the first time and was blown away by the suggested refactorings/changes. I had no idea some of this stuff was even possible in C# -- and then more astonished to learn it had been there since 2012.

I’ve been using C# for about 12 years. I look at the new features and think of places where it would clean things up or how I could build a new feature. An example: I used readonly structs to replace regular structs that were being passed in by reference (boxing) and the new ref for readonly is a one time cost, both speeding up the system and reducing garbage generated.

I recently wrote a little cross platform command line app to tag my audio files [1] with C# (dotnet 6) and I must say: This language is awesome on macOS, Linux AND windows. I personally use Rider as IDE and it feels pretty natural.

The new features will make my C# life even easier.

[1] https://github.com/sandreas/tone

Found out the other day the nintendo switch emulator ryujinx is written in C#. It's the first I've heard of a performant emulator written with a gc language and it's amazing. It's as fast as its competitor written C++ which goes to show that C# can be just as fast as language without a runtime with minimal gc pauses.

C# has a lot of performance oreinted features that are rare to find in GC languages. Value types and real Generics, of course, which helped during the dark years when the JVM was clearly better. There's relatively efficient and convenient interop with unmanaged function pointers and other helpers. Ref structs like Span make it easy to efficiently and safely pass around references to buffers rather than copies. In, out, and ref parameters and readonly structs help reduce unnecessary copies in function arguments. The .NET team has been optimizing the standard library using C# SIMD intrinsics. It's a great language for writing convenient code most of the time, and hyper optimizing where it counts.

Having done a fair bit of C/C++ interop stuff with C#, I would say this is definitely its standout compared to Java.

The things you mention are huge, along with "UnmanagedCallersOnly", the ability to write structs and mark them as meant to be used in interop code (see "Blittable/Unmanaged Types" proposal that was recently adopted), and the seamless "DllInterop" annotations for pulling in native external methods.

https://devblogs.microsoft.com/dotnet/improvements-in-native...

https://github.com/jkoritzinsky/runtime/blob/a5048c2fe5068e9...

The team that works on the native/interop stuff is top-notch, and they're constantly pushing boundaries and making neat improvements.

I'm more of a JVM guy, but credit is well due here.

Not familiar at all with this codebase, but one thing to note is that it does make heavy use of the unsafe keyword^.

https://github.com/Ryujinx/Ryujinx/search?q=unsafe

I really like the removal of the !! operator from being included in the language. The syntax was awkward and the introduction of branching/exceptions into parameter spec felt wrong.

> The feedback and the wide range of insight we gained from this feedback led us to reconsider this as a C# 11 feature. We do not have sufficient confidence that this is the right feature design for C# and are removing it from C# 11. We may return to this area again at a later date.

But Microsoft bad, right?

Yeah they tried for like 2-3 releases but I think they are not happy with adding it. Language design is brutal: you make a mistake which you can never revert. And this group is mature enough to realize it.

Like DateTime. We are stuck with it.

At least they finally added DateOnly and TimeOnly structs, though support for those is lacking for RDBMS conversions.

DT, Offset, and the TZInfo helpers, is better than many languages get. It's braindead simple to convert UTC to a specific TZ and to do date arithmetic.

What's wrong with DateTime? Seriously curious

I have spent weeks of work time trying to track down DateTimes, work out their intended semantics, replace them with a user-defined type, and fix the resulting fallout when it turned out that people were relying on bugs. Unclear who thought it would be a good idea to have the same type mean all of "I don't know whether I have time zone relevance", "I am in UTC", and "I am in local time" depending on some runtime enum it contains.

File systems need backward compatibility. 40 years ago, it wasn’t clear people gonna invent Internet and start moving files across time zones. Keeping file modification time according to local clock was reasonable idea at that time.

The FAT file system (1977) stores local time, and no time zones. CDFS a.k.a. ISO 9660 (designed in 1986) was better than FAT but still less than ideal, it keeps local time, and offset in 15 minutes intervals between local time and GMT. That’s not generally convertible to UTC either.

Windows NT came with the proper file system which keeps time in UTC (also high-resolution, 100 nanoseconds versus 2 second in FAT) but most people only upgraded to NT and NTFS in 2002 with WinXP, and various external drives were still using older file systems without UTC timestamps.

.NET only dropped the support of Windows 9.x in .NET 3.0 (2006), and even afterwards they probably wanted to support all these external drives. Including timestamps of the files on these drives.

I guess I'm just more willing to accept solutions of the form "recognise the mistake we made, and create a whole new API that is more correct" than they were.

1) The API is incredibly error prone. To the extent that even today EF6 Core can do random things to your dates if you’re not careful.

2) It’s the wrong model for representing dates and times so you end up with stuff like the midnight convention (which would be fine if it wasn’t for point 1)

In short: in 99 instances out of 100, you would be better off using DateTimeOffset, DateOnly or TimeOnly.

It should have been two types (UtcDateTime and LocalDateTime), and Unspecified should never have been a thing.

Awesome. Our company is considering moving scientific applications from Python to .NET (C# or F#), mostly for stability reasons and better backwards compatibility.

We need to wrap C libraries though. Does anyone have experience with or reliable benchmarks for the C interface?

I can't give you any benchmarks, but in my experience (from wrapping a big C++ game engine in generated C#), modern .NET interop overhead is basically unnoticeable.

.NET 7 finally also ships with NativeAOT which compiles your C# to native code that you can expose as a C library.

The Kestrel HTTP server included in ASP.NET was using libuv (async event loop implemention used also in nodejs) for a while (it is removed now and replaced with the managed `Socket` implementation. Since libuv is native code, they had a lot of PInvokes/DllImport in their glue code. The performance was way ahead of what they had in their IIS implementations, so I think you will be good performance wise.

It’s pretty fast. Likely reason for that, MS designed both language and runtime this way since version 1.0. They needed that for their Windows Forms which consumes huge chunk of WinAPI.

I benchmarked a while ago when testing this library https://github.com/Const-me/ComLightInterop#performance On the computer I was using at that time (probably Ryzen 5 3600 CPU) the overhead was 15-20 nanoseconds per call.

I just turn my head to look at my bookshelf, and sure enough, there it is, my C# 6.0 & .NET Framework 4.6 book. That was the last version I worked.

Nice to read about all the changes in C#, I've been thinking a lot about going back to .NET, but at this point I would probably need some training. There are things like Blazor or .NET Core that I'm absolutely unfamiliar with.

I kind of miss Visual Studio for some reason, after what feels like a lifetime of web development in other open source tech stacks.

Oh indeed. The language and the normal coding style evolved significantly, the base class library evolved and the whole base promise of the language switched from supporting UI apps and Windows services to be a top contender for any program type you currently can imagine.

Learn .NET Core (nowadays called just .NET). It is a fresh breeze to your memory. So easy, so rewarding compared to the complexity of Java or Node dependency trees.

> So easy, so rewarding compared to the complexity of Java or Node dependency trees.

How do Java dependencies differ much from C#?

Starting with the question maven vs Gradle etc. Maybe that is a bit more build system madness than package management madness.

Build system madness? Basically all projects are either build with Maven or Gradle?

I think .net 3.1-6 has settled enough now that the confusion is more or less gone.

If you want to come back and dip your toe in the water, Visual Studio 2022 is better than ever. But Rider is a great IDE and I’d recommend starting an evaluation of that.

I feel like 2022 is the slowest edition I have used. My 2017 instance is still significantly faster. I've also had a hell of time with "Hot Reloading" and even the debugger in general.

Absolutely; the language moves fast. Around 2015 I was using C# at work. Last year I did some code review on a C# project and at first I had no idea what I was looking at. The new features and design patterns used made it like reading a language I had never seen before.

It’s pretty easy to pick back up and get in the swing of things - and very fun too. Lambda expressions and async programming might take the most study and practice to really feel comfortable using. Blazor server side is a new but very accessible paradigm, particularly with free component libraries like Radzen and MudBlazor. Dapper helps as an alternative for those not willing to go full EF for data access. Visual Studio 2022 is a nice, refreshingly automated environment. Azure has a ton of ridiculously fun tools like devops, key vault, functions, app service, etc.

I’m perhaps in a similar camp, years of experience during the .NET 4 era, a lot of TypeScript and Rust after that.

Recently, I went back to C# for a smaller project. The .NET ecosystem certainly has its advantages compared to the “wild west” of NPM, which includes Visual Studio/JetBrains Rider.

My main problem was that the language feels so dated at this point. Why can’t I make local variables immutable? Why are there still no algebraic types? Why do the new nullability rules feel so inconsistent and tacked on, with “String?” and “DateTime?” behaving completely differently?

I realize not everybody cares about these “modern” features as much as I do, but once you’ve wired your brain to think about program composition in a way that simply feels better to you, it’s very frustrating to go “back”.

Edit: It’s not like the C# team is unaware of or in any way against these proposals. There just seems to be little visible progress:

Algebraic types: https://github.com/dotnet/csharplang/issues/113

Const locals: https://github.com/dotnet/csharplang/issues/188

>Why can’t I make local variables immutable?

How it'd work in such a case:

You declare immutable variable "a" which's an instance of a class

and then you execute "DoSomething" method which mutates under the hood?

Compilation error? runtime error?

Anyway have you tried records?

I’d be fine with semantics similar to readonly fields (and incidentally, “const” in JavaScript): you can’t reassign the variable, but the contained object can do what it wants.

To your last question: Yes, records are really nice syntactic sugar for classes that want to behave like values.

It's usually on a syntactical level, to disable reassigning a value. For objects this means the reference is still mutable, value types are locked.

Js has const, C has const, Java final.

Imho it'd be awesome if mutability were not the default and we would have to mark mutability, but until then any variable that doesn't need it, is final. Which also helps skimming code

>Js has const, C has const, Java final.

C# has readonly and const

>Imho it'd be awesome if mutability were not the default

If they decided one day that - "let's make immutable by default", then it'd be performance hit - wouldn't it?

> If they decided one day that - "let's make immutable by default", then it'd be performance hit - wouldn't it?

Not really. It's just a syntactic default, reassigning local variables should be discouraged from except for iterators etc. It just fosters bad habits.

For example, A list can still be mutated like usual, the reference however may not change. This usually allows compilers to do optimizations in multithreaded situations. But that's not why I code that way, it reduces the cognitive burden of keeping track which variables are actual moving parts as opposed to context/names etc.

> C# has readonly and const

Haven't written C# in a few years, GP was talking about variables as far as I can tell and their lack of immutability.

C# does not (as far as I know) have readonly or const for local variables, only to fields. You can even have consts inside method bodies, but they are only actual compile-time constants.

codeflo means to declare a local variable that cannot be reassigned later.

    void Foo() {
        // this is possible in C# today, but only for compile-time constants
        const int Bar = 42;

        // this is what codeflo wants: variables that cannot be changed after declaration, but can hold runtime values
        readonly string text = $"Blah: {Bar + System.Environment.ProcessorCount}";

        // with such a variable, you wouldn't be able to reassign it later:
        text = "Some other text";
        // the line above would be a compile-time error, just like trying to reassign to a readonly field.
    }

I am sure you have heard this before, but F# sounds like it would be right up your alley. It has sane defaults (non-nullability, algebraic data types, etc.) while still targeting the CLR, enabling the usage of any normal C#/.NET library to be used within your application. Granted, there's _some_ friction going between C# and F#, but that is a small price to pay (IMO).

For me, another gripe is: “Why do I have to manage a csproj file?”

That’s one of the things that tangibly improved IMO: they removed the file list, so except still being ugly XML, the csproj is now very comparable to package.json or Cargo.toml.

A lot… A LOT better than package.json…

And can be edited without unloading the project first.

Better in what sense? Syntax aside, I don’t see a substantial difference in terms of file content. I do see a substantial difference in terms of tool capabilities (like .NET/NuGet’s inability to resolve conflicting diamond dependencies).

As much as I'm not a fan of the XML based project files, the csproj format is much more flexible/capable than package.json files. For example, you may want to have different versions of a package reference based on some condition. Would it be better w/JSON than XML? I think it's a toss-up.

That’s a good point. Rust’s cargo files have that feature as well. And I agree that surface syntax isn’t the main issue.

Lack of comment is a serious flaw for package.json syntax.

Csproj is one of the better build config setups, I'd say. In practice, the IDE handles it all if you want. What would you prefer instead?

csproj is fine these days. sln on the other hand...

Fun thing is the current explaination regards readonly locals: https://github.com/dotnet/csharplang/issues/188#issuecomment...

It is basically: if we introduce it, it gives us a tiny benefit for a gigantic amount of noise during every single code review. It boils down to comparing developer productivity vs. language safety.

I find the argument a bit weak since code review guidelines can end this conversation AND C# has already quite some code analysis code fixes to auto-fix language usage stuff (like file scoped namespaces or nullability checks). And that case of const and readonly can be algorithmically checked.

It does seem like a weak argument when it can be enforced/fixed by linters e.g. this is already solved in JS with eslint prefer-const rule

I suppose you can emulate it today with variable prefixes/suffixes and check the AST with a custom linter yourself

Especially since its not really a language safety feature. As a reader of code I much prefer knowing what I need to pay much attention and what not. As a writer I want to be reminded when I promised myself to not touch a variable.

Some of the comments in that thread must come from people who are lucky enough to have never had to read or refactor any large, convoluted methods. Knowing that this particular bool flag isn’t being flipped thrice, 100 lines down, is a godsend.

That is indeed a problem every once in a while. These people develop a framework and 3 languages. They do not built large systems.

But there are other groups at Microsoft which keep them on track with their real needs.

Echoing the sentiment, I was a visual studio user for the best part of a decade, until Rider appeared on the scene. Rider/intellij is the new bar for developer tools today IMO.

but why?

I'm asking because I do use extensions like Roslynator for refactorings I'm not sure what I'm missing, what fancy Rider has?

Rider has all of ReSharper but without the performance hit that you get in VS + ReSharper.

Ok, but I've been talking about Roslynator (free refactoring extensions), not Resharper

I haven't tried Roslynator (since I stopped using VS years ago), and to be fair you said "extensions like Roslynator". Just trying to give a bit of an elevator pitch for Rider. The analyzers in Roslynator would definitely still work in Rider (that's just a Roslyn compiler feature), but not any extension-only refactorings, that's correct.

Caveat: I'm writing C++, not C# but, Rider does out of the box what I used extensions for for the best part of my time working in Rider to start with. The IDE is faster to start up, faster to use, the tooling works better than the visual studio tooling, the licensing is sane, the source control integration is an order of magnitude better than VS's. The code completion for intellij tools is what you'd expect, for VS it's... varied depending on whether or not VS has parsed your project yet (and good luck figuring out when that is).

Code formatting tool support was substantially better (although still not ideal) in rider than VS, and VCS backed project settings mean that we can have developers auto format locally rather than building out tooling to enforce it.

Cross platform support, with remote editing is also a huge one for me. I have a windows workstation and a Mac as a secondary workstation. Before, I had to remote into the mac and use XCode to debug/build issues on the mac, now I use rider with remote support for 95% of my mac needs, and on occasion when I do need to drop back to the mac, I'm using the same tool rather than xcode.

Until 2022 (which granted I haven't used), VS was 32 bit which despite MS's frequent assurances wasn't a problem, was. Running OOM was a regular occurance on large projects for me over the last 10 years.

Roslynator is a collection of refactoring enhancements for VS. Resharper/Rider add a ton of things beyond refactoring. Its unit test runner is way better (IMO) than VS's native test runner. Support for Unity & Unreal Engine development for game developers. Rider works on Linux, Mac and obviously Windows.

I've been a happy R# customer since 2006. Since then I converted to JetBrain's all products subscription - DataGrip is a really nice database management GUI that supports nearly every database you could ever want (from SQL Server to Postgres to MongoDb!) If you do any frontend development Webstorm is IMO a much better experience than VSCode.

Biggest chance in dot net core is IoC by default and syntax sugar ( eg. Tuples)

Since .net 6 things are changing more. Eg. The first time with a minimal API needs a bit of exploration. In general it's explained by global static functionality.

Blazer is c# and a different way of writing websites.

ASP.NET, Blazer and Maui are IoC by default because they are complex franeworks, .NET is not.

Almost all projects are using those frameworks.

Asp.net MVC 4.5/webapi and before did not have IoC build-in.

Right… so before people had to manually add ioc for all those projects then might ass have a default one, which is swappable if you don't like it.

I think you should reread what I said.

It wasn't there before, it is now. I didn't say anything about if it should be there.

Eg. I know an older developer who still doesn't see the benefit. So it's useful to mention that it's in there now.

Install ReSharper, write code your way, then click on the suggestions. It'll rewrite a lot fo code for you to the new style. Do that for a week and you'll get back.

I'm not spending as much time as I like coding C# nowadays but keep the R# subscription up, it's invaluable when I need to get back up and running.

Why oh why does C# still not have sum types? It is by far the most useful feature still missing from C# and one of the key reasons why I prefer F# over C#.

Useless features. What C# really needs is the following:

- Discriminated unions

- Switch expressions as standalone statements

- Type providers, so it will verify the content of the text, provide syntax highlight, autocomplete, etc for many languages and data structures:

var sql = <SQL>"""

   SELECT Id, Name FROM Users

"""

- Pipe operator to avoid nested method calls (similar to F#)

Did I miss something in the Framework -> Core transition where strings aren't UTF-16 internally anymore?

No, they're still UTF-16.

No, the internal storage is still UTF-16.

What the "UTF-8 literals" feature is really about the conversion of strings to byte[] (and similar).

And more specifically the static initialisation of byte[]: initialising a byte[] from a string literal will store the UTF8 encoding of the literal as a bytes array and can be performed from static data (a bytes constant in the binary), before that you had to write the "new byte[]" with individual integral byte value by hand e.g.

    byte[] thing = new byte[] { 0x77, 0x6f, 0x72, 0x6c, 0x64 };

can now be written

    byte[] thing = "world";

Objects of type `string` are internally encoded as UTF-16. Byte arrays or spans can contain anything, including strings encoded as UTF-8 (or whatever encoding you like, or raw binary files, or random bytes). The new feature does exactly what's written in the blog post: if a string literal contains only UTF-8 characters and you assign it to a byte array or span, it gets encoded as UTF-8. It's just syntactical sugar.

This is a post about C#11 the language, not the framework or the runtime. It's only telling you about what the compiler does when it encounters some syntax. Under the hood, it probably calls some encoders from the standard library.

> if a string literal contains only UTF-8 characters and you assign it to a byte array or span, it gets encoded as UTF-8.

I write a bunch of C# for my job, but am far from an expert in the language. My reading of this statement is redundant, which means I feel sure it's trying to communicate something the authors thought was "obvious" and is not.

* A string literal - so, realistically some Unicode text, right? All the other encodings anybody was actually using can transliterate to Unicode, so, they are just Unicode (with a different encoding)

* contains only UTF-8 characters - UTF-8 is an encoding of Unicode, so, this just means Unicode again

I'm guessing actually C# can write something that's not Unicode in a String for some reason? But what that might be is unexplained:

Can you... emit arbitrary bytes? But how when your native encoding (UTF-16) isn't even byte oriented? What does that mean?

Maybe you can emit the rare Unicode "non-characters" like U+FFFF ? But, you can express those just fine in UTF-8 so who cares?

Or perhaps it's as simple as C# lets you write literals which are sequences of 16-bit code units but aren't UTF-16 ?

The proposal is linked right there in the blog post. You could read it and save some time. https://github.com/dotnet/csharplang/blob/main/proposals/utf...

> The language will allow conversions between string constants and byte sequences where the text is converted into the equivalent UTF8 byte representation. Specifically the compiler will allow string_constant_to_UTF8_byte_representation_conversion - implicit conversions from string constants to byte[], Span<byte>, and ReadOnlySpan<byte>. A new bullet point will be added to the implicit conversions §10.2 section. This conversion is not a standard conversion §10.4.

    byte[] array = "hello";             // new byte[] { 0x68, 0x65, 0x6c, 0x6c, 0x6f }
    Span<byte> span = "dog";            // new byte[] { 0x64, 0x6f, 0x67 }
    ReadOnlySpan<byte> span = "cat";    // new byte[] { 0x63, 0x61, 0x74 }

> When the input text for the conversion is a malformed UTF16 string then the language will emit an error:

    const string text = "hello \uD801\uD802";
    byte[] bytes = text; // Error: the input string is not valid UTF16

I like C#, but the language authors have a tendency to overcomplicate with unintuitive semantics like significant spaces at the closing of the raw string literal (otherwise a very welcome feature) or struct vs record struct etc.

Hey there! Language designer of Raw Strings here.

The use of whitespace to affect the meaning of strings was actually a big ask from the community and was due to a lot of confusion and frustration over the years from customers who didn't like that if they had a multiline string that it would then not indent properly with all the rest of their code and would cause things to look cluttered and unpleasant.

Users participated in the design here and we got feedback from thousands in multiple venues about this design. The behavior around whitespace here was viewed as being both very beneficial and intuitive.

Finally, to help out here, the VS ide will also draw a line showing what part of the code is content and what is not.

You feel this is overcomplicated, but this was the result of a lot of cooperative design with a lot of the user base to something that was near universally felt to be desirable over not having this behavior. :-)

Thanks for Raw Strings! That was one of my annoyances and now it will be much easier.

>Finally, to help out here, the VS ide will also draw a line showing what part of the code is content and what is not.

This will help a lot, especially people who don't expect this behaviour. Practically answers my problem with this (when using VS at least). Still not my preferred solution, but I guess I got outvoted.

I appreciate the inclusion of raw strings. Together with the work on interpolated strings api in previous language / base class library, there might be some nice innovations in the templating corner coming to C#.

Reply on news.ycombinator.com