How does Windows decide whether your computer has full Internet access?
devblogs.microsoft.com
devblogs.microsoft.com
It wouldn't be that bad if it was just the status indicator, but several apps refuse to work properly if "internet" isn't detected. The workaround is to disable every other adapter in the system until NCSI is happy. I ended up binary patching the connection test service in memory to get it to always return true.
I had my share of troubles with it, but most of the times I found a way to force it work as I need.
Well, this escalated quickly.
How is this executed? I would not know how to patch binaries in memory. Is this a common way to fix bugs under Windows so they have tools for that?
I'm used to just have the source, so I can recompile if ever needed.
For me it's very bad code smell when the developers try to check some sort of a global flag ("has internet") instead of just perform the action (eg send http request) and fail gracefully
Why maintain code to duplicate functionality that is provided by the OS? Can I be more reliable than Microsoft?
Absolutely.
Writing all these cozie Windows trivia blog posts. That I like.
Why is he not smashed by some manager? Could everyone else do this if they liked?
https://www.slashgear.com/1102367/microsoft-is-reviving-clip...
But granted, MS Research is not M$…
I would take Scala.js anytime instead. (If I would need to do front-end ever again).
People love this programs so much that MS added them to their dreaded Teams. Now you can have truly productive meetings!
https://www.theverge.com/2022/11/16/23462041/microsoft-teams...
PowerShell is excellent as a scripting language, and has deservedly been a huge influence on shells which have come after it.
WSL is extremely frustrating because it has so many bugs and gotchas, but I'd say it's pretty popular with its intended audience.
I understand why WSL2 is now just a HyperV'd Linux, but WSL1 is amazing despite its limitations. I'm honestly hoping that they re-consider deploying a properly-developed Unix personality again, but that ship has sailed.
Agreed. WSL1 was incredibly ambitious, and that alone made it exciting. And what it does manage to do, even in its unfinished state, is also impressive in its own right.
(FWIW, I actually had WSL2 in mind when I was talking about bugs. Switching to a VM-based approach solved some compatibility issues but WSL2 definitely still has problem.)
My best guess after turning my head 270 degrees, closing one eye, and squinting the other is that the noun is Microsoft/Windows, and the verb is running the ncsi daemon and having it fail the check.
It’s definitely not an OS I would like to make my daily driver. I just hope Apple won’t go down that road with macOS.
What is the secret keyboard combination?
How on earth did Microsoft okay releasing a Professional version of their OS that offers no suggestion on how to finish the install when no network devices are detected?
A simple Google found me the answer but it is piss poor UX to offer zero options when it knows there is no network interface to enable.
People like to joke that you need the Terminal in Linux still and yet I couldn't even install the brand new Windows 11 on a computer without needing to open a command prompt using a keyboard shortcut and enter some cryptic command which rebooted my machine and enabled some hidden option.
When you get to that step, press Shift + F10 to bring up the terminal, then type "OOBE\BYPASSNRO" and press enter.
The installer will restart but this time will have a "I don't have Internet" button that you can use to bypass this nonsense.
Created an MS account (because I want this machine to be as normal-user as possible), set up a PIN. Signed in with a PIN to the desktop, run 'Remote desktop settings' and fip the 'Remote Desktop' toggle to on and affirm the prompt that asks if you really want to do this.
After that no issue RDPing to the machine by IP or hostname from another machine on the same LAN. Username and password is the same as the MS account I first signed in with.
(For reference, Windows 11 22H2 running on an HP Prodesk 600 G5, RDPing from macOS using Microsoft Remote Desktop 10.7.10 installed via App Store.)
But yeah, I'm pretty sure the domain join is only an option on the pro and enterprise editions.
I've also found out that the domain join is only offered if it can contact the internet. I installed this on a brand-new laptop the other day, and it didn't detect the wifi card and it had no wired network. It absolutely refused to go past the "let me connect to the internet" phase until I went through the "hidden terminal" dance.
This is in a stark contrast with current linux desktop distributions, which do allow domain join straight from their OOBE.
Then the outrage comes in full force, as a kind of culture shock, whenever some external situation requires you to do something like set up a proprietary desktop operating system.
> I hope there's a future (or alternate universe) where dark patterns, such as this, result in economic loss rather than economic gain.
I agree. I wonder whether that can actually be achieved through end-user savviness alone.
Sure there are probably was a way to skip most of the requirements (except iTunes itself) but around that time I helped a friend to activate an iPad.
I was forced to register an Apple account AND give it a credit card to activate that iPad.
You can blame MS all you want, but MS IS LATE FOR THE PARTY WITH ALL THAT SHIT.
> Please don't use uppercase for emphasis. If you want to emphasize a word or phrase, put asterisks around it and it will get italicized.
And just because other manufacturers do it, isn't a reason not to blame MS for what MS did. Nobody forced them to do this and it's not even common practice. The OS wasn't "built with online accounts in mind". It's something nobody asked for and everyone lived happily without before.
Like, my point is that these devices are fully functional out of the box even without an account(but yes, it sucks that you need one to unlock it fully)
Windows 11: lay-users can do literally nothing of value with their computer without signing in to an account they probably don't want
MacOS: lay-users can use the internet and download software provided outside of the app store without signing into an account that they probably don't want
Calling these equivalent would be quite an exaggeration.
Welcome to microsoft ecosystem
> So it's not malicious
I would have to firmly disagree there. Microsoft are pushing the notion that an online account with some identity provider is a necessity for home computing by hiding the (perfectly functional) option to create a local account.
This is just frog in a boiling pot. Windows (and iOS) is becoming an OS as a service.
Pretty soon you won't be able to use it without paying for something.
- This is My procedure (with Lan) to obtain a local Account on W10
Install Windows 10 and go through the OOBE
- Select Region
- Select Keyboard Layout
- SKIP Secondary keyboard layout
- Network connection
- ENABLE Allow PC to be discoverable
- Setup = For personal use
- Account = Offline account
- like old times - a completely standalone PC
- Sign in = Limited experience
- ignore the nudging to make an online account
- User account = <my initials>
- as this gets used for the name of the home folder and I don't want my full name for that
- No password
- to avoid the nonsense socalled "security questions". Password will be set later *after* completing the installation
- Location usage = No
- Find my device = No
- Diagnostic data = Required only
- Improve inking = No
- Tailored experience = No
- Advertising ID = No
- Customise experience = Skip
- Set a password for the accountSo it's not malicious -it's just malicious.
Gotcha!
Ahem. You literally just described a very clear-cut, textbook example of malicious. Yes it is absolutely malicious.
Certain powers in this world want there to be less regulation for the moneymakers in this world. Centralized wealth has been a cancer on humanity for millennia, and we're nowhere near putting the proper amount of shackles on capitalism.
[1] https://www.windowslatest.com/2022/09/28/spotify-app-is-auto...
I don't know if this changed with the 22h2 upgrade, but it's the behavior I'd noticed before.
The weird CPU spec thing is that Microsoft doesn’t let you sell a high end desktop computer with “regular” Windows Pro, and requires instead that it comes with Workstation. But you can either version on any grade of CPU as long as it runs Windows in the first place
(Source: Windows OS admin/engineer for large company where we /do/ manage all of this.)
I used to use it to test connectivity on my own apps because Apple is probably better at making sure a url reachable than I am.
http://neverssl.com also works
And https://1.1.1.1 breaks people's brains if they know enough
A bit harsh I thought!
Android and Chrome do the same sort of thing to detect internet access; this is how Android pops the notification to sign in to the network.
Then, at least on Windows, the results of NCSI flow down into WinHTTP and a ton of other things so apps can know the status of the network.
It's also possible, via Group Policy, to configure a different URL for NCSI. This is useful in enterprises which may not have the NCSI URL available to unauthenticated things (eg: the OS) but still has internet access via proxies.
It's also possible to disable NCSI, captive portal detection, etc, which is useful on some closed network boxes (eg: some enterprises) but this will cause problems if the machines are ever used on public/walled garden/captive portal networks.
The biggest problem I've seen with this comes about where captive portal detection is disabled, a user ends up on a captive portal, tries to hit a website to satisfy the portal, but due to most sites that normal users will try being https these days can't get their session redirected in order to display the portal, so they think "the internet is broken". The NCSI/captive portal detection makes a point of using HTTP so captive portal redirection can work properly.
Not if they parse the payload of the response
Yes if they only use the http status
Edit: an update (3.0.0) to the Switch changed it to http://ctest.cdn.nintendo.net/ but I can't seem to access it from a web browser.
Apple (iOS, MacOS) checks for https://captive.apple.com/hotspot-detect.html and expects a basic HTML page with the body containing "Success".
I have a problem at your math though, I'm guessing with the overhead and the pretend-IE headers it could be a whole Ethernet packet (which can be up to 1,500TB/day, but realistically it could be around 500TB).
But I guess you could say the content length header would be 1 byte longer in the double digits length contents (which it is).
curl -is http://www.msftconnecttest.com/connecttest.txt | wc -c
520
Can anyone give any tips on either of these?
Note you need to disable tamper protection and reboot first otherwise it silently reenables itself.
Is there a nice description / workflow / tutorial / script / community where I can learn how to do that?
I did not find any recommended workflow for this by Microsoft itself, but maybe I was searching for the wrong things - windows updates are generally a bad thing to research anything related for. I expected to find some standard workflow description plus tools on some MS website, but no success. Does that exist?
Thank you very much!
Then, in WSUS console, you set up approvals for updates and then the updates will be offered to clients only once you approve them. You can divide the clients into groups and manage the approvals for these groups individually, so you can have a separate testing group.
Being shafted like this every now and then has eroded my trust for Windows' updates.
Remember that security vulnerabilities in Windows are discovered all the time, so it's dangerous to use Windows without installing the updates. If you (rightfully) don't want to install the updates, then you should switch to an OS that actually respects your freedom instead, like Linux.
(I already do all my important work on Linux, since like 20 years)
https://download.wsusoffline.net/
will download Windows updates and create an installer for them.
I used thousadands of Windows machines in last decade, this is typical. You can ignore this "feature" almost entirelly.
If you introduce proxy in your system, then you can be certain that it will not work, including Windows updates. You have to masssage your system with net commands and learn about WinHTTP proxy (that nobody heard about) sfor it to sometimes work.
To deal with this and other nuisances I made 2 functions in PowerShell:
Update-Proxy https://github.com/majkinetor/posh/blob/master/MM_Network/Up...
Update-CLIProxy https://github.com/majkinetor/posh/blob/master/MM_Network/Up...
I am currently on basic OS install without anything in between and it doesn't work. I just switched from home router to my phone's hotspot and its the same.
It seems its not only Windows problem. Viber desktop has exclamation icon and it will persist until it is restarted, Mattermost works, Signal works etc.
Check it out: https://i.imgur.com/Oi5gmw4.png
BTW, didnt know proxy is exotic, its literrary the norm in the company.
If they don't load for you you probably modified your install with shutup10, block microsoft domains using your firewall or /etc/hosts or something else and you're out of support.
irm http://www.msftncsi.com/ncsi.txt
Microsoft NCSI
irm http://www.msftconnecttest.com/connecttest.txt
Microsoft Connect Test
Keep in mind, that these do require Hyper-V nowadays. Especially if your Windows 10/11 has virtualization-based security enabled (mandatory in 11); then using Hyper-V is the only way to virtualize anything.
Hyper-V is also requirement for WSL2.
So given this, Hyper-V might be enabled on a good chunk of these billion PCs.
It's not that Hyper-V would be my first choice either. But it is not exotic configuration at all, and the fact that it is a first party product which breaks this makes it even weirder.
It might be bubble compared to entire planet, but its still pretty big bubble. Once you have hundreed K users, you should have responsibility to deliver.
Especially becasue people in that bubble are those moving the others forward.
Is it because I've also created a Mobile Hotspot with the same SSID on a spare mobile phone I have, so my family can use their iPads out and about without having to connect to a new Wifi network? (i.e. I just works)
Is it because for some devices on my network, I DHCP them a different DNS server so they get adblocking via AdGuard home?
Who knows. It's so annoying. The fix if you want to update iOS on my home network is to connect to the Wifi Network called "F*kApple" which is exactly the same network as normal, but with a different SSID. Because that works just fine.
Also, F*k Apple.
PS: Also F Google because trying to search this problem just gives me the most infuriatingly childish "How to fix!" articles.
You should be able to set wifi network priority so "home network" is first but if not available it automatically goes to "phone network".
Apple caches a lot of info about networks it connects to. So it's probably caching that it received this option and "knows" that network to be metered.
Best solution for this is to have either more control over your mobile network so it's not sending that option, or more easily, name the mobile network something else from your home one.
Curiosly, Microsoft does respect it: https://learn.microsoft.com/en-us/openspecs/windows_protocol..., though it has to be matched to 'MSFT 5.0' clients.
If that spare mobile phone is an iphone, probably yes.
Wonder what kind of uptime this service has and what sort of footprint is needed?
Here is a list of Azure operated locations: https://learn.microsoft.com/en-us/azure/cdn/microsoft-pop-ab...
and what about the origin behind the cdn? seems like that it’s still rather critical and would have migrated multiple physical hosts over the past 30 years.
This sort of thing is fascinating and I do something similar at a much smaller scale at day job, keeping a simple service running that enables the rest of the stack to survive.
I doubt there is any origin at all. Since the response is so small and never changes it is likely hard coded into the config. Maybe it is a file on a shared object storage.
I'm in a situation where I needed to modify my Registry https://learn.microsoft.com/en-us/windows/win32/cimwin32prov... in order to convince the Metro UI that I was actually connected, whereas everything else was working fine.
I can't remember exactly what I did now, but it's been working great.
Also, sucks how Spotify wont even try to play songs when Windows displays no internet
But you have to point it at a server with the right file present. It doesn't just do a ping.
[1] https://www.ghacks.net/2014/02/07/disable-customize-windows-...
Who's
IT'S WINDOWS FILE PROTECTION
for spotify, offline playback is premium feature.
I like that one because it provides NAT + DHCP, creating another switch doesn't.
In my experience it is a selectable option when creating a new virtual switch but is not modifiable for the default.
I have my machine directly connected to my underpowered Remote Desktop client via ethernet, and it always picks that connection instead of the WiFi that actually has internet until I disable ethernet (or maybe the virtual adapter created on top of ethernet, I forget every month) and reboot.
In Windows 2000-8.1 the control panel GUI was the standard way of accomplishing this, but in modern Windows 10/11 I doubt Microsoft has the setting still accessible. There are still guides out there with screenshots, though: https://www.windowscentral.com/how-change-priority-order-net...
The automatic metric detection system bases its decision on network speed (https://learn.microsoft.com/en-us/troubleshoot/windows-serve...) so virtual 10gbps adapters can cause problems if you use a common network adapter and the custom settings for Hyper-V and such get messed up.
Note that as with the Windows version, the protocol is HTTP, not HTTPS – because captive portals completely break TLS, but plaintext HTTP will result in a clean redirect to the portal, allowing the network service to detect the presence of the portal and to bring up a browser window to let the user authenticate.
(https://devblogs.microsoft.com/oldnewthing/20221115-00/?p=10...)
I can see no reason why HTTPS is needed in any event. It's a single purpose domain that serves a static text file which everyone knows the content of.
Mayeb that's the problem?
It still wouldn’t find http filtering, but it would work better than I initially gave it credit. (I still doubt it would give a contextually correct answer for an airplane wifi connection [where DNS May very well work but few other services do if not paid].)
I mean really, what does it mean to you, or to Windows, that I have "full" access to the Internet?
For me personally, I only visit a few walled gardens, so as long as I had my Google, my Wikipedia, and my work-related sites, I wouldn't miss 99.99% of the Internet anyway.
But what if your ISP blocks a whole bunch of ports? What if Adware has taken over 33% of your DNS space? What if you're behind a Great Firewall of <Dictatorship>? What if there's some sort of Balkanization or segmentation of your side of the 'net and you can't reach a lot of stuff? What if Cloudflare's down again?
However that doesn't tell you about the presence of a portal or if HTTP traffic is actually possible.
This can be a problem if there's some kind of critical vulnerability in the Microsoft HTTP stack, but I don't think this attack vector is all that relevant.
Same with other captive portal detection endpoints, there's very little actual parsing going on with these requests.
Thus: Hijacking WU to download malicious content takes far, far more than just DNS hijacking. You'd also need to subvert the WU signing system. (This is more nation-state level stuff.)
It should check if you are behind NAT and then say "Your computer doesn't have full Internet access but can reach some services via a gateway"
(And if you want something like UPnP to let programs automatically punch holes themselves anyway, again it doesn't matter much whether we're talking about NAT or "just" a plain firewall.)
The true evilness of NAT only really comes in when it's done by some third party outside of your control (CGNAT and friends), but I think that compared to home routers doing NAT the latter is a slightly more recent phenomenon that only got widespread traction when the IPv4 shortage became more acute.
For example port forwarding dosn't help evolution of new internet protocols. Iit prevents replacing TCP with SCTP due to this, or deployment end-to-end IP level encryption (like IPSEC attempted). Or a myriad of other decentralized or security enhancing inventions that depended on the end-to-end nature of the internet architecture that now have never gotten off the drawing board because they are not NAT-compatible.
(And of course the majority of users behind NAT are in fact behind third party controlled NATs)
IPv6 doesn't require NAT, but my bog standard home router still firewalls it, and I need to manually allow inbound connections (or give up and just use UPnP).
But also the whole posture of a home network and consumer os might have been different without NAT, maybe the host based firewall would have won out, who knows. In the alternative universes we can't assume other things remain the same.
And even if you somehow have a non-NAT, non-CGNAT, no-ISP-filtering home connection, do you have full Internet access if the server behind NowhereNews.com refuses all your connections because you’re in Europe?
Today most home networks have NAT for for v4, and then NATless IPv6 (or no IPv6 as the case may be).
Trivia: NAT is not routing, the normative router requirements RFC actually specifically forbids tampering with the IP source or destination address fields.
Anyway, won't be my problem much longer.
Anyway, won't be a trouble for me much longer. Retiring all my Windows instances, had enough.
Micro$oft is trying to define what is and is not 'internet' access, eg. they turn the definition of 'internet' to mean unfettered access to the micro$oft servers is and you're ability to send personal information over to them.
The Redmond you’re thinking of is in Washington, BTW.
Disagree.
http://connectivity-check.ubuntu.com/, https://nmcheck.gnome.org/check_network_status.txt, http://captive.apple.com/hotspot-detect.html, http://connectivitycheck.gstatic.com/generate_204, http://www.archlinux.org/check_network_status.txt, http://networkcheck.kde.org/
You can disable the checks if you want. Any application that uses the operating system API/DBus to evaluate network connectivity rather than building its own bespoke online check will probably break if you do, though.
Because all of these have to use HTTP, you can also easily override the standard network addresses in your hosts file and pick your own server if that's what you prefer.
For some people Facebook is the internet and for others it is Outlook. Somehow a twiddler at MS has decided that a file on a web server at some wanky location is the internet and that's the final answer.
"I cn haz a file" is fine lovely internets! No it bloody well isn't. You should be constructing a response to a challenge from the other end, not a response to a simple GET like it's 1999.
I run quite a few systems/sites that have multiple internet connections - deciding whether the internet is available is quite a nuanced thing and that Windows internet detector is bloody stupid, naive and a fucking hindrance.
Define "internet" and then have a crack at defining "internet accessibility". Those things are quite specific to individuals. orgs and so on. Connectivity is way too complicated for a simple, naive check.
Seems like it's the same in many linux distros as well as android and chrome os. It's right there in the article. So it's not really MS specific at all.
This is asking for the true Scotsman. This is designed for the 99% of users where internet access is a ternary "yes, no, needs credentials". The "naive" check is enough for them.
Also, as far as I know only Windows has defined settings to disable this check and assume that there's a connection (https://learn.microsoft.com/en-us/troubleshoot/windows-clien...). At least you could disable this if you need to, because Android don't have a similar setting.
wot?
I'm not following this, where is the nuance? What problems does this cause and why are they so difficult to solve? Even if behind a strict firewall that allows only a few IPs or ranges (arguable that would still be the "internet" as commonly understood), couldn't you just override DNS to return the same file from your server?
I've been annoyed by iOS disconnecting from wifi when no access is detected and I'm just trying to stream something from local network, but not Windows.
For example how do you tell traffic to go via WAN2 (or 3 or whatever) instead of WAN1 if is really down (define really down). So you create a rule that says that all inbound on LAN is routed via a failover thing. That's fine but now you've broken RFC1918 routing. You try to connect to a remote site via 192.168.lol and its fucked.
So you now create a rule that forces 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8 to be routed via the usual routing table and after that you have a rule that worries about internets and multi WAN. Simples.
No of course it isn't that simple but it is quite close and good enough mostly!
There are several problems in search of a solution here. Is a WAN down? Usually you ping something. What do you do if the thing being pinged is down but the link is actually available and how do you deal with that? It gets to charts of risk/reward at this point.
Also, I hope you’re not relying on ICMP to tell you meaningful things about your relationship with the internet. It lies.
> You should be constructing a response to a challenge from the other end, not a response to a simple GET like it's 1999.
Really? Isn't simple better here, why are you making this sound so crazy? Also you can turn it off.
The author also had a good response for spoofing question imo: "So what if somebody spoofs it? Congratulations, you tricked Windows into showing a “full internet access” icon, and then when the user tries to go to a web site, they get an error."
You sound like a typical engineer who cannot see the bigger picture of business decisions.
I've been a Managing Director for 22 years. Make of that what you will, me old fruit.
I used to be Chartered (Stick n Bricks n that) ...
If you need some complicated algorithm, write a quick simple web server that runs on 127.123.45.67 and does all of these checks for you when the magical portal URL is requested. Then update your registry to point to that IP (or use hacks like editing your hosts file) and you've just added your special logic to every WinHTTP application on your computer. You can even point Windows to an endpoint only reachable over VPN if you want so the Internet check becomes "is my VPN operational", though that may break the VPN software itself.
Microsoft did a good enough job for all normal use cases of the Internet. Bespoke use cases need bespoke solutions, and they provide the ability to set that up without hacks if you want to change the standard behaviour.