Language Server Protocol implementation for Swift and C-based languages | Hacker News Reader

Language Server Protocol implementation for Swift and C-based languages

github.com

232 pointsKryptoshi117 comments

For anyone interested, here is their official announcement back in April 2018 on the LLVM mailing list about their vision and plans for the clangd updates. As a C++ dev, I'm excited to see this tooling come to fruition. It's going to make the IDE experience with Clang based tooling far superior on all platforms.

https://lists.llvm.org/pipermail/cfe-dev/2018-April/057668.h...

Related: It's an interesting time for C++ tooling in general. Based on conversations with a member of the Visual Studio team, I've come to understand they're responding to increasing competition from CLion. This was the driving force behind the decision to prioritize first-class CMake support in VS 2017. I also think they're taking notice of what's working well with VSCode and taking that into account for VS 2019 updates.

I've moved away from it for my C++ needs since it's just so complex. The vim extension works well, but beyond that I can't master any of the keyboard shortcuts as they're so different than any other editor I've used in the past. These days I use VSCode with VSCodeVim, cmake tools and clang-query (which is fantastic). I've been really loving working with this setup. I fire up VS 2017 with `devenv my_proj.sln` when I'm lacking any advanced performance profiling / debugging features, but this is increasingly rare.

Overall, with the clang developments in the last few years, you can entirely replace the MSVC tooling. PDB debugging, ABI-compat, etc. Now clang-query and this work by Apple will ensure the IDE experience is taken care of too. The competition is welcome.

Clion still has lots of catchup to do in mixed language debugging, GPGPU, multi-threaded debugging, DirectX debugging, COM/UWP development.

From what I see on online discussions, clang still isn't as good as VC++ in incremental compilation and linking support.

Then lets also see who gets modules first.

Interestingly we had Energize C++, Visual Age for C++ v4 (earlier versions were different) and C++ Builder. Had they managed to win a sizeable marketshare and such C++ nice productivity tooling would have been a thing much sooner.

I was wondering whether that means that Xcode will adopt LSP as well and found this article: https://nshipster.com/language-server-protocol/

"Getting Xcode to use our new LSP service should make it viable to use other LSP services as well, and it’s something that we are interested in, but we don’t have specific plans to announce at this moment."

It would be great if this becomes reality. I currently hate switching editors/IDEs for a project using several technologies, because e.g. Xcode is great at C++ debugging but terrible at CUDA editing.

Maybe https://forums.swift.org/t/new-lsp-language-service-supporti...

> We are also going to make the necessary changes for Xcode to use this new LSP-based language service. Similar to how you can load an OSS toolchain 397 in Xcode to try it out now, in the future you will be able to load a build of the OSS LSP service in your installed Xcode and test any fixes or enhancements you make.

The other interesting protocol in this space I only recently became familiar with is the Debug Adapter Protocol or DAP [0]. Curiously it doesnt't seem like there's a standard for test running though? Unless that's supposed to be an LSP action of some sort.

[0] https://microsoft.github.io/debug-adapter-protocol/

>Curiously it doesnt't seem like there's a standard for test running though?

The code lens API allows you to register commands with code spans of your choice. There's no specific built-in command for running tests, so it's ad-hoc.

https://microsoft.github.io/language-server-protocol/specifi...

Eg for the Rust LSP server: https://github.com/rust-lang/rls/blob/ea7a0c36aa5f45846b06f8... + https://github.com/rust-lang/rls/blob/ea7a0c36aa5f45846b06f8...

That's great news to me. I was hoping they do the same for debugging.

> felt like something we all should of figured out years and years ago

Before language server there was YouCompleteMe, a plugin for Vim that had the same ideas. It was eventually ported to other editors, but LSP wasn't far behind and eventually took over.

I'm no good with Google Trends, but here they are: https://trends.google.com/trends/explore?date=all&q=youcompl...

YouCompleteMe used fuzzy finding to create something like IntelliSense.

LSP is an API that talks to compilers, and returns actual relevant data about the code you are working with, and in conjunction with the Debug Adapter Protocol allows things like setting breakpoints in a standardized way.

EDIT: looks like YouCompleteMe did try to use OmniSharp and racer to provide better support for certain languages, but it still is different than a standardized protocol that any language could add support for without needing to be included in a swiss army knife like YCM.

> LSP is an API that talks to compilers, and returns actual relevant data about the code you are working with

That's what YCM did as well. Well, compilers or other intellisense-adjacent tools (like Jedi for Python, which Python's language server also used, or Tern for JS).

YCM includes various non-standardized static analyzers and such, and glues them together to be a swiss army knife. LSP is a standardized API that compilers can target, and that means there is no need for a middleware component that does work.

I still use YouCompleteMe and even standard ctags because they fill additional functionality that an LSP can't provide. In a big C++ project you'll have several different configurations that can be built, none of which touch all files. In that case the LSP can't give you completions and diagnostics for a file that isn't built, but YCM can often deal with it by compiling with include paths similar to surrounding files. So this way I can safely edit and browse files of other configurations. Similarly if you're building for different OSes, you have chunks of code behind #ifdef linux/WIN32/osx, so they just never get compiled and the only thing that can index e.g. structs defined within those ifdefs is good old dumb ctags. I strongly feel that the union of these different tools that operate at different levels of integration with the build system results in a much more comprehensive solution for real-world systems than would be possible with just one of them.

YouCompleteMe provided an API that abstracted away what compilers/tools were on the backend and gave you information that an editor could use. Just like LSP.

Yea, but LSP gives an API target for the compilers not just for the editors.

whoops this was supposed to be a reply to https://news.ycombinator.com/item?id=18883088, mods feel free to move it.

Apple implementing a spec designed by Microsoft!?!?! Joking, I love that Microsoft came out with LangServ though, felt like something we all should of figured out years and years ago, good thing they wanted to make a cross platform text editor :) I gotta give Microsoft their props, they hire some really smart people who do some great things for the software industry.

Props to Apple for doing this too, I am looking forward towards the next stages for the LangServ v4 spec, not sure what more they could do to make it awesome, but I'm sure there's plenty that could happen in the future in this space.

Yeah it seems so obvious in hindsight: of course a language should have ONE editor-optimized compiler/analyzer for IDE like features, which can be reused by all editors out there instead of building half-baked implementations for each IDE.

But many things seem obvious in hindsight.

IMO the reason this rather obvious idea took so long to be implemented was because on one hand the closed source vendors didn't really have a strong incentive to provide something like that (you support your own ecosystem and let the others deal with theirs) and on the other the open source community was spread across many compilers and interpreters which made standardizing difficult. The only project I can think of with enough clout to impose a system unilaterally was GCC, which was notoriously hard to extend and modularize (especially before LLVM started shaking things up a ~10 years ago).

So overall the effort to create a protocol and then getting everybody to accept it was rather huge and the chances of success rather slim. It's great that MS finally decided to do it and the huge popularity of VScode (especially with younger coders) seems to have helped a lot.

I recall the lack of extensibility in gcc being deliberate, according to RMS -- he wnated everything around gcc to be free, and plugins would allow for non-GPL or even closed source code.

Deliberately harming your users to own the corps. Jokes aside, open source is great, but RMS’s perspective is extreme. I don’t think it’s an exaggeration to say that the overall quantity and quality of open source software is much greater because of closed source software because companies fund open source initiatives with profits from closed source initiatives and then the second order effects of maintaining a huge pool of professional programmers, some of whom use their skills to contribute to OSS.

I think you have to put his principles in a historical perspective. Sure it sounds somewhat extreme today when OSS dominates many layers of our business stacks. But assuming that the industry would have naturally seen the OSS light is far from a given IMHO. I think copyleft licenses where a very important springboard to start the virtuous OSS cycle.

No one should make money selling software. Beg for donations or sell ads if you need to pay your bills. Or get a cushy university appointment like me, and spend your days polemicizing about freedom.

That's your moral view, but you missed my point which is that it's incompatible with a florishing ecosystem of open source software.

Open source software existed before closed-source software was even a thing. Software used to come in source-code and freely-redistributable form; it was basically funded as a loss leader for things like hardware and support, that could be billed for. Many companies today which fund FLOSS have very similar models, only with other sorts of value-added services being added to the toolset. Others have even managed to pay for FLOSS directly via crowd-funding.

To be clear, "OSS predates CSS" is not a refutation for "CSS makes the OSS ecosystem larger and more robust". Apologies if that's not the point you were trying to make.

the funny thing is that gcc was pretty easy to extend before the eggs split. unfortunately, it was also pretty stagnant and nowhere near as good of a compiler suite as it is today.

The key element was Andreas Heijlsberg work on the Roslyn C# compiler which exposed all stages as APIs (e.g. AST etc). He calls the dragon book as significant outdated when it comes to modern compilers (Channel9 video).

That then transferred to TypeScript, which again influenced VS Code. OmniSharp was then externalizing it. In VS Code I think Erich Gamma took it over and put it into standardized a protocol.

The Tiger book is a better reference nowadays.

Another good alternatives are "Compiling with Continuations" and "Lisp in Small Pieces".

I wish there was a book that teaches how to create a programming language based on the Roslyn idea.

Well that's great to know. I just bought the dragon book. Is there a better book out there?

It is still a pretty damm book to read about compilers, if not anything else, at least from historical perspective.

The Tiger book, "Compiling with Continuations" and "Lisp in Small Pieces" are some of my favourite ones.

Then if you want to learn about language semantics validation, "Formal Syntax and Semantics of Programming Languages: A Laboratory Based Approach".

> many things seem obvious in hindsight.

The best ideas seem obvious in hindsight. I've had the experience of reading some books that explain groundbreaking ideas so well that you feel "shit, that's obvious. I should have thought of that". I think Language Server protocol is similar.

It sounds pretty much like Steve Yegge's Grok project (now "Kythe") from over a decade ago. They just started at a different use case.

> something we should have figured out years ago

I actually blame Stallman for that. He insisted that GCC should not dump any AST or provide embeddable functionality so that actors like Apple wouldn't be able to build nonfree software on top of it. However, that meant you also can't build any free tools on top of it, so everyone was forced to make their own ad-hoc C++ parsers and analysers. Once clang appeared there were much higher quality tools available. There's rtags that's a sort of proto-langserv implementation for C++ from before Langserv was a thing.

I suspect Stallman's decision was right at the time, but it did hold back free tools.

Do you have a link to such a proposal being put down by rms? Curious about this.

For me the takeaway is that Apple is _really_ serious about supporting the use of Swift outside their own ecosystem. After all XCode won't be using this, at least for a while, so clearly it's aimed at making it easier to use Swift outside their own tool chain.

Nah, Swift still does not support Windows on their official builds.

Fair point, but nobody’s writing backend software to support iOS apps on Windows servers so not something Apple cares about.

But then backends for iOS apps are part of Apples ecosystem, if peripherally, so I stand corrected.

Nor anyone is writing anything big in Swift on GNU/Linux, which even with builds available, is still pretty much WIP.

Vapor, Perfect, and Kitura are no match for the offerings in Java, .NET, Go, OCaml, Haskell, Erlang.

Of course, but then comparatively few people are writing iOS apps in Swift either. It's still a very young and still evolving language ecosystem, so building out the tooling around it is incredibly important in order to support it's growth.

It seems to be a very un-Apple thing to do. Do you know why they would want to push it?

Get more people using Swift serverside perhaps?

I can imagine that having a swift backend for your swift mobile app would be a lot nicer to work with than converting back and forth from a bunch of stringly-typed JSON.

Yes, this is it precisely. Swift already has decent momentum for serverside usage with Vapor, Perfect, and Kitura among others. By adding LSP support, Apple is adding to that momentum and pushing it closer to critical mass.

I'd love to use Swift on the backend, but I don't want to use XCode.

I only develop part-time, as my main job is financial in nature, but god, it is really hard to find good learning material for serverside without depending on XCode. In particular, learning the server frameworks is almost not worth it. By using one language, I can keep my productivity acceptable.

If anyone has any references, I'd be super thankful!

> stringly-typed JSON

Off topic but I love this, definitely stealing it. Did you coin this?

I learned this term from a 2012 blog post, which summarized a (now deleted) Stack Overflow question:

https://blog.codinghorror.com/new-programming-jargon/

My other favorite from that list is "baklava code" -- code that has too many layers.

This is great, thanks!

As the bald guy said: "Developers, developers, developers"

Now if they'd only let me compile my iOS app on Linux ;)

What if this is because a lot of internal Apple developers are using VSCode instead of XCode, and they wanted better support?

Not impossible, but unlikely. For Mac, iOS, tvOS, and watchOS projects at Apple, Objective-C is still the dominant language due to all pre-existing projects being written in it. Swift is used in a few places — most notably the macOS Dock has been written in Swift for at least a couple of iterations now — but bulk adoption won’t come about until Swift achieves ABI stability with version 5.

> the macOS Dock has been written in Swift for at least a couple of iterations now

Dock is only partially written in Swift. A significant portion is still Objective-C.

That isn't what they announced at WWDC 2017, during the state of the union.

When did they mention this? I can't find this in the video.

Wrong year and video.

It was during WWDC 2016, "What's New in Swift" session.

https://developer.apple.com/videos/play/wwdc2016/402/

From the transcript. More on the video.

"I want to talk about a particular case study, and that's the Dock. And the Dock is interesting for two reasons. The first reason is it's a lot more than people think. It's not just the bar at the bottom of your desktop. It's actually a large amount of the macOS Windows, you know, management experience. So things like Mission Control, LaunchPad, Spaces, this is all what we consider part of the Dock. The second reason why Dock is interesting is it's actually been a Swift adopter for two releases. It started adopting Swift in El Capitan. So we have two releases to look at and to compare in how it was using Swift. So a lot of changes happened to the Dock in Sierra. First, almost all of Mission Control was rewritten in Swift, and the accessibility engine was rewritten in Swift. Why did we do this? We just didn't rewrite code just for the sake of rewriting code. The code had kind of grown organically over time as features had been added and it was time to refactor it. But also some new functionality needed to be implemented, and so it was the appropriate time to make this, you know, to do this rewrite. Now, what does this look like at the project level? So to give you an idea, Dock is about 200,000 lines of code with tens of thousands of lines written in Swift. And between El Capitan and Sierra, the amount of Swift code in the project more than doubled. And this actually, because this is part of a rewrite, this replaced a significant amount of C and Objective-C code. And the effects here are somewhat non-linear, because there's some refactoring and some new features being added, but we saw there was about 15 percent less code to replace the existing C and Objective-C code in Swift, and this is with new features being added at the same time. And the net result is this project is actually slightly smaller in Sierra than it was in El Capitan. Now, there's a lot you can read into statistics. I think the most important thing here is the engineers working on the Dock love using Swift. They are not mandated to use it. They volunteered."

So one would expect that during the last two years, the amount of Swift code has only increased, specially with the tiny spot Objective-C now gets at WWDC.

Ok, so that matches my experience of hacking with Dock pretty well. In OS X El Capitan, Dock moved a small part of their code to Swift, and in Sierra a much larger portion was migrated. What I said is still true, though: a major (but slowly shrinking) portion is Objective-C. But to say that Dock is written in Swift is misleading: from my estimates, I'd guess that Objective-C still makes up a sizable portion of the codebase, probably around half or so.

> Apple implementing a spec designed by Microsoft!?!?!

Not new at all. They support AJAX in Safari. Macs use SMB for file sharing.

I did say I was joking, but yeah, I love that AJAX was basically reverse engineered from IE.

And they support MS Office files formats in Pages and Numbers...

We did figure it out years ago; it's called Swank (for SLIME). While it was intended for Lisp dialects, there's also an implementation for JavaScript.

LSP is Swank for other kinds of languages, and is certainly welcome, but neither the idea nor successful implementations of that idea is new.

Kinda like Google dropping an in-house language to use TypeScript -- designed by Microsoft -- as the language for building their JavaScript UI framework. :-)

That's one thing I always find a bit funny. There's swarms of developers who love TypeScript over regular JS. Personally I rather use whatever is default first, but that's just me. If you wanna pay me to do TypeScript cool I'll use it.

I am guessing OmniSharp [0] is based on this LangServ protocol? I tried OmniSharp last year and a few years prior to that, since I wanted to develop in C# using SublimeText. I can only say that both times it's been a depressing experiment. It seemed the OmniSharp server was really buggy and kept crashing.

---

[0]: http://www.omnisharp.net

I think it’s the other way around: LSP was based on ideas prototyped by omnisharp.

I used it years before LSP was announced.

>Apple implementing a spec designed by Microsoft!?!?!

Mind you, the current recommended file sharing protocol on macOS is SMB. (since Apple have deprecated AFP)

Really? Then why doesn’t it work?

They’ve added some Apple-specific extensions to SMB, which can result in suboptimal performance when browsing non-Apple hosts.

https://lists.samba.org/archive/samba-technical/2014-Septemb...

Clojure community pretty much had it solved for clj & cljs with nRepl & cider-nrepl & co.

Did that help someone who wanted to work on clj or cljs inside Vim for instance? It's not really a comparable solution until it's multi-language, multi-editor, cross-platform, and supported by many different teams and communities.

> Did that help someone who wanted to work on clj or cljs inside Vim for instance?

Yes, it did.

from https://github.com/clojure-emacs/cider-nrepl#editor-agnostic

> Projects like vim-fireplace and vim-replant are making use of cider-nrepl already.

Swift is a really great language and I'm very bullish on it becoming popular for server-side projects in the next few years, once they improve a few things (namely performance gotchas and concurrency).

I see two kinds of programming languages in the world. There are a couple which were designed with a great concurrency model from the start, and there are all the rest which weren’t. In these, concurrency remains awkward at best, and seriously problematic at worst.

I have never seen any programming language retcon a good concurrency model onto version 6. The actor proposal I’ve seen looks horrendously complex.

I try to be open minded about something I haven’t used yet but I really don’t think this is something you can just bolt on.

I do agree with your point, however, I think Swift is in a unique situation since they intentionally deferred on making concurrency first-class. Now the language is in a much better position to implement a modern system.

What about the proposal is horrendously complex? The target audience is meant to be language designers, not the average developer. The actual suggested API changes looked to be a few added keywords and some new classes/protocols to use.

Hasn't the introduction of async/await to C# been rather successful?

One of the most important things they should fix ASAP, IMHO, is how cumbersome it is to install and get up and running on Linux in general, it totally needs something like Rust's rustup before or first party support from major distros before I can recommend it to anyone.

Language Server implementation for Swift... meaning that writing, compiling, and debugging Swift (and Obj-C) in Visual Studio Code is just around the corner? What other editors/IDEs use Language Server?

Doesn't the Language Server only cover writing code with things like auto-completion, documentation lookup and refactoring? Compiling doesn't need editor support beyond running arbitrary programs and debugging uses a different protocol, as far as I remember.

LSP doesn't really have anything to do with the latter two, compiling and debugging.

After Googling I see the list include Eclipse, IntelliJ, vim, and Sublime Text and VS Code with Atom and emacs in progress.

https://langserver.org/

I feel like that page might not give the full picture. I see language servers that are green on all 6 columns but don't work very well for even medium sized projects. Still, I'm super excited about the work being done here. It's only going to get better.

That table only shows that the languge server supports the feature, Whether it's stable or not depends on the language server implementation. You should report to your language server implementor about the instability

I've been using langserver in NeoVim for the past year and it's been great.

Which plugin are you using? Looking at https://langserver.org/#implementations-client there are at least four of them, and IIRC the neovim folks are also working on a native implementation.

I'm using autozimu/LanguageClient-neovim [1]

From my .config/nvim/init.vim:

call plug#begin('~/.local/share/nvim/plugged') Plug 'autozimu/LanguageClient-neovim', { 'do': ':UpdateRemotePlugins' } call plug#end()

let g:LanguageClient_serverCommands = { \ 'javascript': ['node','/Users/jrousseau/javascript-typescript-langserver/lib/language-server-stdio.js'], \ }

" Automatically start language servers. let g:LanguageClient_autoStart = 1

[1] https://github.com/autozimu/LanguageClient-neovim

I love the momentum around Swift these days, and the LSP looks very interesting.

Thanks to Apple for making this open-source. Looking forward to integrating it into Sourcegraph! https://github.com/sourcegraph/sourcegraph/issues/1557

I finally installed ALE in Vim this summer and have been so happy with it: https://github.com/w0rp/ale. Great even for linters for JS. Always much easier than expected. Shouldn’t have waited so long...

This is amazing, I was just wondering why this wasn't a thing a few weeks ago! Can't wait for Swift to be a great general purpose language for me.

This is very exciting! Does this mean it will be possible for users to write swift using vim in a more painless way vs xcode?

I don’t believe vim is an IDE, so it won’t be on the same level, but I’ve been writing Swift for websites in VScode for over a year now and using the command line to compile the code.

Reply on news.ycombinator.com