Is there a maximum size for Windows clipboard data? | Hacker News Reader

Is there a maximum size for Windows clipboard data?

devblogs.microsoft.com

269 pointskencausey166 comments

> over 300,000 rows

Any process you ever enable in code will be pushed to limits you never dreamed possible. This is exactly where discoverability can come into play - someone starts by copying 50 rows and using that to get their job done, and may never realize there are other ways (CSV export, etc).

If code can "recognize" when something like this is being done and provide documentation on other ways, it can help people learn new methods.

> If code can "recognize" when something like this is being done and provide documentation on other ways, it can help people learn new methods.

Now I'm imaging Clippy popping up when you copy a ludicrous amount of data to the clipboard. "You appear to be using a cludgy method, may I suggest something better?"

Of course, I'm guilty of things like this. I have one excel macro that splits data from a giant stream* (one monolithic CSV, actually) into discrete units, graphs them, and then sorts the graphs for printing. It makes a lot of use of the clipboard, and god help you if you try to copy/paste something while the macro is running.

*Giant enough that at one point it grew to where I overran my Int and had to upgrade some of the code to Longs :V

There are some exotic scenarios, where copy-paste is the good(fast) solution. For example, one of my clients can be accessed via a VPN. Once I'm on the VPN, I have to start their VDI. Inside the VDI, I need to start another VPN, that then allows me to open a Putty session to a bastion host, that I can use to reach my destination cluster. Believe it or not, they don't want to make it simpler.

Copy/paste works flawlessly through the VDI into the putty session, so I usually just uuencode my blob, and copy it to destination cluster and uudecode there, instead of uploading the file to the VDI, then pscp to the bastion host, and then scp to the destination cluster. (I know of zmodem, port forwarding and such, none of this is allowed...)

That sounds horrific. I wonder if anyone has built a TCP tunnel that works over shared clipboard? :)

Not sure but I’ve made something more ugly.

Once upon a time at $workplace we had a production and development LAN and we had two PC’s sitting on our desks, one for prod and one for dev.

Now there was a special SMB share to move stuff between the two networks but trying to copy something from one Pc to the other was tedious.

So of course I wrote a magic clipboard utility that used a file on that SMB share as the clipboard “storage”. Running the utility on both PCs allowed me to use copy-paste between the two PCs…

I have once worked for a customer where other freelancer had developed a special proxy. The customer required working in a hermetically closed vpn environment even to access the test systems. Whitelisting a single web page cost an unreasonable amount of budget and took days to weeks. Therefore someone had decided to run the vpn inside a virtual machine, and developed an http proxy that consisted of two processes, one running inside the VM and one outside. Now, you gan guess how they exchanged requests and responses. That whole setup was surprisingly usable.

copy/paste really is a great tool. I bought a Windows gaming PC during the pandemic, and my other systems are Macs. I started to use barrier (a multi-platform software KVM-like keyboard and mouse sharing thing) and was pleasantly surprised to learn that it shares the clipboard across all of my PCs, and my Macs in turn also share the clipboard to my iphone. This works in both directions. It's convenient to the point that using multiple PCs with multiple operating systems is barely different than using one PC with 2 monitors. I travel a lot for work and this feature also lets me use my personal laptop as a slightly limited second screen for my work laptop. Total gamechanger for free.

Have you looked into powerpivot in Excel? I learned how to use it, then eventually moved into using Power BI, and now that's what I do for a living. Powerpivot unlocks so much potential in Excel, and it allows you to have millions of rows of data and work with it seamlessly. It also allows you to create relationships between tables instead of using vlookups. If you haven't tried it, look at some introductory videos on youtube. It will blow your mind.

I haven't! If I need to revive this particular bit of shameful spaghetti code, I'll look into it, thanks!

I always wanted Clippy to pop-up when excessive use of VLOOKUP is detected with a message: "It looks like you're trying to implement a relational database in a spreadsheet. May I suggest you get professional help?"

People complain a lot about MS Access, but a easy-to-use relational database is actually a very necessary tool. It's too bad that Microsoft hasn't figured out how to make it easy to make an access-like system in Excel itself.

For sure, re: Access and a relational database tool being important. If somebody ever manages to make a web-based Microsoft Access clone (that actually works) they'll make billions.

It's fun help a "savvy" Excel user understand how a relational database can reduce their byzantine VLOOKUP-based fever dreams into simple joins. Typically if somebody can grok VLOOKUP-fueled nightmares they can grok SQL pretty quickly and will actually appreciate what it can do for them.

>If somebody ever manages to make a web-based Microsoft Access clone (that actually works) they'll make billions.

They did, they called it Salesforce.com

I think Salesforce is fine for large companies with teams of sales reps and a well staffed IT department, but it's overkill for most small to medium sized companies with just a few sales reps. If you can run on a creaky set of excel sheets, access would have been a far better and much more flexible solution. With minimal training it could be set up and run by a competent excel user.

Salesforce admin is a pain in the ass. There's mandatory updates 3 times a year that require low level software development expertise in order for the admin to understand what will break, building new functionality essentially requires tens of thousands of dollars of consultant development, and god help you if you want to run a report that involves objects that don't have a direct parent-child relationship.

I'm an analyst, and because I know how a database works, Salesforce stuff gets dumped on my desk all the time. I actively seek out alternative systems for everything it does, and would dearly love to see us drop this eye-watering expense.

> There's mandatory updates 3 times a year that require low level software development expertise in order for the admin to understand what will break

If you're veering from defaults, you're doing Salesforce wrong. Even Salesforce's own Services team these days refuse to write custom code that'll break beyond default features.

Airtable?

There was Access Services in SharePoint but it was depreciated a few years ago. Unfortunately there are some very large companies that use Access in a shared drive instead. They make daily backups just in case there is data corruption.

'just in case there is data corruption'

In my experience that was guaranteed if any of the clients connected other than by ethernet.

I've not done it but I understand that it's not difficult to have an Access front end for a SQL Server backend rather than the default mdb or accdb backend.

All being Microsoft products the migration to SQL server is easy as I understand it.

I find it hard to understand why large companies would not adopt this approach.

Access files are definitely super-fragile. That's not isolated to Access, though. Any time a file sharing protocol is used to host a database-- particularly with multi-user locking semantics-- I get the willies.

I've done more "development" of Access-based applications over the years than I'd care to admit and, as a menu-driven front end for interaction with a client/server RDBMS (SQL Server, Postgres, etc) I think it's good cheap solution. Visual BASIC for Applications (VBA) can even do some powerful (and arguably ill-advised) stuff like directly calling Win32 APIs. It's an awesome prototyping tool in my experience.

The typical Access solution I see in large corporate environments is something knocked-together by non-IT people. Once it gets to the level of needing to be upsized to a client/server RDBMS back-end the "real" IT department comes in and demands a purpose-built solution. Usually bureaucracy, disdain for Access, licensing costs for the client/server RDMS, taking away file servers, etc, end up killing it.

Application Packaging was in my past; can confirm the disdain for Access. There was a definite gap in IT Support though.

Upon upgrading from Windows XP, Access was removed from the environment AND existing access databases were shoved into a database farm.

But any new business needs for something similar IE "Hey IT, we want a little app just to store XYZ business info" fell on deaf ears - the response was always "Hire IT Company to build it and hand it over." which of course would cost tens of thousands rather than a handful and thus priced it out of existence.

Which meant of course the business people started finding solutions OUTSIDE of IT Support purview....

> They make daily backups just in case

Doesn't everyone make daily backups in production environments?

Also, aren't all backups made "just in case"? No sysadmin ever arrives at the office, thinking "Today that accountant is going to mess up the database, I can just feel it in my bones, better make a backup this time" (unless they think that every day...).

In this case the production and development "environments" are probably the same exact file lol.

You are right on the backup thing. It's just funny to me because access databases on shared drives corrupt very regularly. It's less of it "just in case", and more of a monthly necessity

>If somebody ever manages to make a web-based Microsoft Access clone (that actually works) they'll make billions.

Microsoft already has built that web-based access data-layer, it's called Dataverse - http://aka.ms/dataverse https://www.youtube.com/watch?v=iSKVxx_ghbw

Snowflake

"Easy" is tough when it comes to relational databases. Even Access requires a decent amount of technical know-how when compared to the people who use Excel heavily. In Excel Power Query and the M language are quite powerful. You can combine tables from both your sheets and/or a variety of external sources and create relational views into them. You can then display these views on a sheet.

When I worked for a company where corporate used it, the problem was that people would come to me for help. M is pretty easy to learn so it wasn't a big deal for me. But I am a software dev with database experience, so I'm not sure who the target market is here--it's not the pure Excel users. Like, they wanted tables from a shared folder appended on each other and joined to another table. In pure Power Query you can do this without too much trouble, but you need to know what you are doing. These people have trouble with VLOOKUP already (I mean the string/int problem is unintuitive, so I'm not talking about that), so this is an impossible task for them.

It's confusing that Excel supports relational database operations at all. If you want to query the data in Excel it should seamlessly push you into Access rather than partially recreating the same functions.

> It's confusing that Excel supports relational database operations at all

I'd say it's for the better: sometimes you want to stay in the same tool, and it's the tool's job to interpret your intent.

Imagine you are editing a markdown file and have a picture in your Windows clipboard.

IMHO, the proper action would be to output base64 encoded jpeg data, even if it's "partially recreating the same functions", as it makes a copy-paste of screenshots from say the snipping tool (Win Shift S) seamless.

BTW, github doesn't support base64 encoded jpeg or png data, under the excuse that SVG could have security implications (which is true but not applicable for png/jpg) so don't bother trying it over there...

Airtable is so much better at this than anything Microsoft has tried to make.

Oh you must have seen my accounting department's 'transaction register' which is a...

checks latest

64mb spreadsheet that amounts to one gigantic, ever slower pivot table.

One department at my agency shrunk an ~80MB Excel sheet down to ~1-2MB by using the .xlsb (Excel Binary) file format, and it sped up quite a bit. But point taken, once this is the single source of truth for biz-critical data, Excel is not the right storage format.

“Yeah Frank I got rid of that annoying pop up telling me to get professional help by mixing vlookup and xlookup in my spreadsheets”

> excessive use of VLOOKUP

I feel called out. Maximum number of rows in a VLOOKUP in personal spreadsheet is like an array of 2x1000. In an earlier job, it was 13x39000. Excel automatically increases the range when you add rows in the middle. Then I split my data per year, & it was like 13x5000.

I have a colleague who had a dedicated PC that had to be on at all times. The company bought a UPS for it, and it was on patch exception lists to prevent automatic reboot.

One day, I asked him what it was and he showed me. He was running a series of excel macros that took over 96 hours to run, and fed a critical business process that was worth a few million dollars.

I sat down and took a look at it, and in about 3 days implemented a prototype in SQLite and a small Perl script that ran in about 8 minutes. Basically, it was a series of sql joins and some regex. My colleague, a project manager, literally hugged me and was near tears.

Some devs later took that prototype and refactored it to run on a normal production platform.

My team often does the reverse of this. We need something built and it goes into project planning for the next year (we plan out bigger features a year out, with flexibility to squeeze in smaller features every month). Sometimes those features get punted out from the next year.

So we end up building the feature in Excel. With large (for Excel, say 100K-500K rows) data sets, it gets pretty slow. Or it'll get too complex, so we'll put it into our own Postgres DB.

Eventually, it'll get big/complex enough that we prioritize using development resources to automate the feature.

To an extent, this has become a standard part of our rigor for deciding whether or not to build features. You say you need X, but you aren't doing X today. Can't do you do that with an SQL pull and then doing all the business logic in Excel, and then creating the entries by hand? In some cases, the answer is no, we need it to be part of a larger pipeline, etc., but for a lot of things we do, building it by hand proves that the feature is needed, and also helps us work through the edge cases and makes writing the stories/BRD much easier. It also takes pressure off the dev team to try and intuit the true requirements.

The problem with turning an excel solution that is maintained by a subject matter expert into a properly programmed one (even just a python script) is that it’s not clear who will maintain the solution once u leave. This can be especially difficult for consultants - yes I can “fix” the hack now to and reduce costs, but if it means hiring some dev/sysadmin etc. to maintain the solution, it may not end up being cheaper.

Hold my beer.

  Private Sub Workbook_SheetChange(ByVal Sh As Object, ByVal Target As Excel.Range)
      Dim intRetval As Integer
      ' TODO maybe respect some of the properties on Assistant to let the user disable this
      If InStr(Target.Formula, "VLOOKUP") = 0 Then
          Exit Sub
      End If
      ' TODO is one use really "excessive"?
      Assistant.Visible = True
      With Assistant.NewBalloon
          ' TODO I should probably do ".Mode = msoModeModeless" but I don't feel like writing a callback
          .Text = "It looks like you're trying to implement a relational database in a spreadsheet."
          .Labels(1).Text = "Get professional help"
          .Labels(2).Text = "Just let me generate excessive technical debt without help"
          .CheckBoxes(1).Text = "Don't show me this tip again"
          .Button = msoButtonSetNone
          .Animation = msoAnimationGetAttentionMinor
          intRetval = .Show
          ' TODO make the user's choices actually do something
      End With
  End Sub

You'll need to load Office 97 or some other ancient version in a VM if you want to actually try this, since Microsoft finally killed off Clippit completely in Office 2007.

My chief memory of clippy was that he was modal and that he couldn't answer the question 'make clippy go away', so I could actually do something.

>(one monolithic CSV, actually)

might i introduce you to sed and awk?

If you can also convince work to issue me a non-Windows machine, sure!

Honest question though, as I have no experience with utilities like that: Could those tools together manipulate e.g. the data on the left to the graphs on the right?

https://i.imgur.com/DXhMlE5.png

if it is in any kind of easily formatted manner comma, tab, pipes, etc, then yes.

depending on your windows, you might be able to take advantage of WSL or as a last resort run cygwin. maybe there are other methods of using awk/sed/grep type utilities as well, but I am not familiar.

i still remember the day when someone watching me struggling with large text files and took pity on me by showing me the light of the holy power of awk/sed/grep. it was taking minutes to make each search/replace edits and I had several of them. it took mere seconds for the shell commands to process the same thing.

Looks like I have some research to do, thank you!

Just paying it forward! Standing on the shoulder of giants.

This is a chicken and egg problem. This limit seems impossibly high because users lack the expectation they can perform this action, because designers of software don't expect users to perform this action, because users lack the expectation they can...

I disagree with sibling comments that copy/paste is a hacky solution. It's a very natural one for lots of users and there's no actual fundamental reason not to be able to accommodate it.

There's no actual fundamental reason that opening and accessing workbook data from the command line should be so hard, either. Particularly from PowerShell.

This is a "Microsoft refuses to make composable software" problem.

The ImportExcel (https://github.com/dfinke/ImportExcel) PowerShell module is quite useful.

Some video games have tutorial systems similar to this. They will recognize when players are struggling with a certain mechanic or system, and pop up a relevant tutorial. i.e. if a user is trying and failing to jump across a gap, they probably forgot to use their hover boots.

This can get annoying though if the user is deliberately doing something that ends up triggering these messages.

“I bet I could get across this if I had those hover boots” as a voice line from the character can be somewhat less annoying. If it’s implemented well. I think it was the new Horizon Zero Dawn game that I’ve heard constantly does that to an annoying amount? Haven’t played that one.

Filed a bug report once. App crashes when font is more then 13,000 points.

There was a long discussion on memory management before they just stuck a max size on it.

When I was a dumb kid, with my friends, we'd inject 100k point unicode fish into the chats of online games and crash every single client. Eventually they encrypted their chat protocol instead of just removing HTML support and doing sanitation. We eventually figured it out but it was exam season so we dropped it

We (Wireshark) ended up doing this when someone tried to load a capture with 54 million packets: https://gitlab.com/wireshark/wireshark/-/issues/16908

Visual Studio used to (or maybe still does) have an issue where it wouldn't hit breakpoints after the 65,536th line in a source file. It wasn't something I ever encountered, I just came across the issue in the docs somewhere, but I pity the person that ran into it.

> Any process you ever enable in code will be pushed to limits you never dreamed possible.

Not me. And that’s not a brag. It is probably not a productive way to think using modern toolsets. But in my world, if you can copy a few things, you should definitely expect someone to copy a shitload of things or at the very least, document clearly and with rich error messages what went wrong.

Likewise, I tend to get nervous when I can’t explicitly handle out of memory conditions, and the problem is I’m a Go programmer. It kind of bothers me that there’s no standard way to deal with that on my end with built-in data structures such as maps and strings. This is probably sheer paranoia because most modern systems have plenty of memory… But it still leaves me uneasy.

> This is known as clipboard delay-rendering

Aha, so that's why Word[1] just explicitly asked me if I want to save the current copied image before quitting. It's probably asking before loading something big into memory?.

[1] https://i.imgur.com/sNfE3Be.png

Yep, Excel does something similar where it asks you "you have a large selection copied [(marching ants)], do you want to keep it available after closing Excel?"

wow, that's a big selection. after all, it's just text, right? or am i selling it short? I understand when Adobe notifies that the clipboard is large since it's rasterized (typically) image data. what in the world is in the buffer for Excel that size is pushing limts?

A lot of the time it's not just text, but rich text. For instance I'm pretty sure the font is preserved if you paste into a font-aware app.

okay, but I have way more RAM than even the largest of font's I have encountered (and I've seen a few). also, what an absolute bozo way of doing things to include the font rather than the path to the font

The Rich Text Format specifies the font of each word, etc. That is what the GP meant.

I don't think there is any implication that font files are being included with the data.

Okay but then that's still text to describe how to write more text. How big can that be?

no, it's that what you believe to be in your clipboard is not actually in your clipboard, and Word is asking if you want to have that data after it closes and can no longer populate the clipboard.

it has nothing to do with clearing memory.. clearing memory for anticipated use isn't really a thing ever since virtual memory became a thing. (memory isn't prepared for use until after an application actually tries to use it.)

Side note/feel good moment: the author Raymond Chen is an OG beyond-super-engineer at Microsoft, and is an incredible person to work with. He assisted me with my intern project many years ago, and was amazingly eager to share his wealth of experience with a green-eared intern from outside his org.

Thanks Raymond! I decided to go the startup route instead of returning to MSFT, but you were an inspiration all the same.

His "people send us dumb vulnerability reports" series seems quite arrogant to me, with new severe hole being found in Windows every other week.

The Windows Clipboard, for all of its problems, remains one of the easiest program-to-program communication methods to use in modern GUIs.

I can take a screenshot of my Firefox with "PrtScn" button (moving a bitmap of the page into the Clipboard), paste it into Microsoft Paint, edit it a bit, and then copy/paste that into my Discord chat screen for commentary.

--------

Yes, Linux / *Nix users have pipes and great text processing tools. Yes, Powershell has interesting objects that can be passed round. But practically speaking? Most of my "inter-process communications" is over this silly "Clipboard" Windows has, and it works way better than it probably should.

Even better, you can copy an attachment in a mail in Outlook, and paste it into an application running on another computer via RDP. The file will be copied over without the applications knowing any better.

We use this so users can attach invoices and other documents they've received through mail to orders in our application. Users typically run Outlook on their laptop/PC, but our application as a published application.

We rely on Excel exposing the data in an XML format on the clipboard to reliably import data from Excel. Same for Open/LibreOffice. Without it it's impossible to handle cells which contain both the quote character and newlines, due to the way Excel formats plaintext.

We don't want to import the whole file, because the users typically only need to import parts of the spreadsheet. This saves us from writing a complex UI to select the data to import.

And FWIW, I've used that with >100k rows of data.

Yea this is a super underrated feature - copy and paste through RDP is awesome. Although in some cases you would probably want to of course disable that for security reasons.

It works several layers deep too. My typical WFH scenario these days is:

    home PC -> 
    RDP to work laptop at home -> 
    VNC to work network -> 
    RDP to work desktop -> 
    RDP to other VMs and PCs at work

I can copy text or a file or a screen shot at any level and paste at any other level. Pretty amazing.

It's clipboards all the way down...

I've found the clipboard to be a highly flexible tool, that seems under-used. As an example, I've extended my game engine editor to draw any position in 3d that I have in the clipboard. So when problems happen and are displayed in the logs, I select the log, copy to clipboard, boom, position drawn.

This idea has a lot of legs IMO.

There was a browser based game called "Utopia" in the 00s that was reasonably popular. Someone wrote a tool where you'd copy/paste your kingdom homescreen, and get all sorts of calculations / predictions.

It was one of the easiest-to-use calculators I've ever seen. Just enter your webpage to play the game, Ctrl-A (select-all), Ctrl-C (copy), and bam, the Utopia-program would instantly analyze your kingdom and provide advice.

Just all off of the HTML-copy that Clipboard supports.

------------

Windows developers "know" the clipboard is one of the most important inter-process communication structures of the entire system, and have known for years. Its probably one of the Win32 API's biggest advantages, no joke.

The shear amount of creative solutions I've seen from clipboard manipulations is outstanding.

Since Windows 7 or so, I've noticed more and more instances where you ctrl+C and the clipboard doesn't update. I had a chat with a Windows PM on it opportunistically a while back.

It turns out, more and more applications are locking the clipboard and holding onto the lock inappropriately. Enough that there's a particular telemetry watcher for this condition for MSFT to get an idea on who the guilty parties are.

Everyone has had it happen - I notice it when colleagues are in a rush and send me something they didn't mean to send (because their clipboard didn't update). I don't know who the guilty parties are (since that telemetry is encrypted, and can't be inspected), but I sure hope someone is knocking down some doors to address the problem.

Oooh so that's what that is. I've had it happen numerous times where I was absolutely sure I pressed CTRL+C yet the clipboard was empty. I now use CTRL+X and CTRL+Z just because of that.

That's interesting - that Ctrl+X behaves differently. It shouldn't right?

But you have found that it helps?

It helps because there is visual feedback so you're sure you don't have to go back to the original program and do the copy again. Plus sometimes your original program forgets your selection, with CTRL+X you're sure it's on the clipboard and won't have to return.

One guilty party: some versions of the RealVNC client randomly lock the clipboard.

Genuine question, why is it even possible to lock the clip board?

I have tried to track this down - it seems like at some point it was not possible. My guess is Vista is when the lock was implemented. I've come up blank repeatedly.

I am sure Raymond Chen knows. Looks like comments are allowed on his blog too!

An application doing a paste needs to query the available clipboard formats and then ask for the data from one of them. This wouldn't work reliably if other applications could simultaneously change the clipboard. So the OpenClipboard API locks the clipboard and applications need to remember to call CloseClipboard.

A weird one I've noticed with the clipboard on the Mac, is that the "rich text" clipboard and the "plain text" one seem to be able to get out of sync. This is most noticeable to me when trying to paste text into some dialog boxes in Microsoft Word.

This sounds to me like it's probably a proxy for the rise of GC languages. C#, which I reference because it's popular for windows desktop apps, has IDisposable and using() which helps a lot, but isn't mandatory. If you don't do something like this and there isn't memory pressure, there are low odds of a handle or resource lock being released.

Coming from the other side, most apps don't check the winapi error code when copying text. One app I use (notepad2 maybe?) at least throws up a tiny "Error" dialog box when it fails.

I configured my clipboard manager to make a crinkly paper noise when an app copies, so at least I have some feedback when it fails.

This must be a common issue because now that you say it I realized that I have a habit of always Ctrl-C'ing multiple times. Could also be the keyboard, most of mine had a glass of Coke at some point in time.

IIRC factorio's blueprints are represented, or have one representation, as base64-encoded gzipped JSON on the clipboard (with a version byte as prefix in the b64 form, double-IIRC), meaning you can copy/paste blueprints between the game and anything other that accepts text.

Hearthstone added something similar for decks (even made it human readable):

  ### Starting Party
  # 
  # Cariel Roame
  #   - Tome of Light 4
  # Xyrella
  #   - Robes of Purity 2
  # Rokara
  #   - Helm of Inspiration 1
  # Murky
  #   - Time to Krill 3
  # Millhouse Manastorm
  #   - Arcane Powder 1
  # Deathwing
  #   - Heart of Unliving 3
  # 
  CO+cmQQSDlN0YXJ0aW5nIFBhcnR5GAEiRQoJCBIQiwEYICAACgkIExDKARgjIAAKCQgLEIgBGAsgAAoLCLgCEOMCGLIEIAAKCAgOEB8YFCAACgsIqgIQygIYkgQgACgA
  # 
  # To use this deck, copy it to your clipboard and create a new deck in Hearthstone

I haven't been keeping up with the Windows world. Does Microsoft provide any safeguards or tools for the user to limit access to the clipboard? The clipboard was introduced in the good ol' days when you could basically trust the applications running on your computer. Nowadays, third party (and for those of us more paranoid) first party apps need to be considered attackers. The clipboard is a juicy target that often contains sensitive data I'm sure most apps would like to gobble up and exfiltrate. I know iOS and Android both are starting to introduce restrictions on mobile apps' use of the clipboard. Has this made it into the Windows desktop world?

Windows isn't sandboxed so any per-program restrictions is pretty much pointless because it'll be trivial to bypass.

Regular apps, Windows Store apps are sandboxed. Some of them are quite popular (eg: Netflix).

The rules are completely different if you're running "appstore" apps, which actually have some kind of sandbox.

If it's a regular win32 app, then it has access to all your filesystem (except UAC bits), can read and post keystrokes to other windows, and indeed can just straight up inject code into other processes.

Not true anymore - there are process isolation modes you can enable in Windows 10 as well as kernel isolation where it runs core processes in a VM essentially. And there is also protected folder access.

I've been wondering about this -- I'm making a text editor RPG, and in the medium term I want to make a file picker that builds a procedural dungeon based on your file system. I know such a thing would have been trivially fine in older versions of windows, but I didn't know if going down that road has any foot guns a more educated win developer would know.

Is protected folder access an on-by-default thing, or is it only certain folders?

Quick googling makes me think my plan might trigger windows defender for an exemption request. Hm That's a drag, but understandable I suppose?

You'd trigger Windows Defender's "Controlled Folder Access" only if the user has it on (it's still off by default IIRC due to many, many false positives) and only for write operations. By iterating every file you may trigger its "this might be ransomware" flag but you'll only be stopped when you try to write to a protected folder. (Which is only Documents, Pictures & Videos by default)

But honestly, I've had to whitelist Microsoft's own Windows utilities from CFA in its earlier days, so don't fret about it.

Thank you. I'm deliberately going to be readonly (and most likely only filenames unless the user requests contents) When I'm walking around the filesystem.

Thank you for the info.

What you describe also works on X11 and probably what MacOS uses too.

I'm not sure why it would be impressive or unusual. The ability to store arbitrary data on the clipboard with metadata that describes what data it is, such as an image file, or an audio file, is very old.

> What you describe also works on X11 and probably what MacOS uses too.

Except its no where near as smooth or universal as in Windows. In Linux, you have to worry about KDE programs vs GNOME programs having different clipboards, for example.

Its also why compatible suites of programs (ie: Adobe) can be so useful when those custom formats come into play. Adobe can copy/paste layers from Photoshop and send them over to other programs. If the photoshop layer isn't understood, it seems to become a Bitmap, which works when I'm on a VBulletin forum over in Firefox land.

---------

Copy/paste works on files / file-managers, images, text, HTML. Yes, any OS could do the same thing, but they largely don't in my experience.

The conversion behavior you describe is also what happens in MacOS.

>In Linux, you have to worry about KDE programs vs GNOME programs having different clipboards, for example.

I don't worry about this on linux because it's not the case.

I suppose if I were on windows I would have to worry about legacy programs that only understand UCS-2 not being able to talk to modern programs that only understand UTF-8.

>Its also why compatible suites of programs (ie: Adobe) can be so useful when those custom formats come into play. Adobe can copy/paste layers from Photoshop and send them over to other programs. If the photoshop layer isn't understood, it seems to become a Bitmap, which works when I'm on a VBulletin forum over in Firefox land.

I'm able to paste images from GIMP, Krita, and Thunar into forums just fine.

MacOS clipboard is also pretty good, I'm often pleasantly surprised by it. I don't use Windows enough in the right ways to compare them, I wouldn't be shocked if Windows is even better.

But even if old, it's still notable that such an old concept/technology works so well, and there are few newer inter-process communication methods that work as reliably or with as good UX!

Who invented the "clipboard", Xerox PARC? No, wikipedia just credits two individuals, Pentti Kanerva and Larry Tesler .

They're about equal, MacOS generally is better about letting my copy a thing in one application and paste the whole object in another. Windows however has RDP, which I've used as a file transfer mechanism by base64 encoding files, I thought about writing software that would use it as a half duplex communications channel.

Mac also has something that lets you copy/paste clipboard across machines (including between iOS and MacOS), but I don't know how it works technically and haven't actually used it myself.

https://support.apple.com/guide/mac-help/copy-and-paste-betw...

Nah man, clipboard on Linux, mainly KDE, is really good. KDE splits out selected text, and ctrl+c text, into separate clipboards, allowing for some really good workflows.

Selecting text and middle clicking is so much faster than ctrl+c, ctrl+v. Having the two separate, allows you to essentially copy two separate things at once, and paste them separately, which is great for filling out forms.

Also, KDE provides an absolute gem of a clipboard manager, which stores all recent copies in to a list, from which entries can be browsed in order to recopy some text that you may have copied a while ago. IDK where you got the idea that Gnome apps have a different clipboard, as I have never experienced that. Maybe that is an old thing? Definitely not how it is now, thats for sure.

Copy/paste works on files / file-managers, images, text and HTML. This is not a Windows only experience. In fact, the clipboard manager even has little file icons, and image previews, for copied objects.

> Also, KDE provides an absolute gem of a clipboard manager, which stores all recent copies in to a list

In windows 10 and up, win + v to see (and the first time, enable) the clipboard history selector

Yes I love this Windows 10 feature.

Note that KDE has had it pre Windows 10 though.

> KDE splits out selected text, and ctrl+c text, into separate clipboards, allowing for some really good workflows.

Just for completeness & correctness: that's an X11 feature, not related to KDE, and predates KDE by several years. I can't easily find the history of it, but this Keith Packard paper[0] on it is dated 1990 so it's necessarily older than that…

[0] http://keithp.com/~keithp/talks/selection.ps

(pg. 5, "4.1. Selection names" - PRIMARY, SECONDARY, CLIPBOARD. I don't think SECONDARY is used much these days.)

> Except its no where near as smooth or universal as in Windows. In Linux, you have to worry about KDE programs vs GNOME programs having different clipboards, for example.

I am no sure where you get this idea. The clipboard is an X11 shared resource and any program taking control of it removes what any other had on it before.

> Its also why compatible suites of programs (ie: Adobe) can be so useful when those custom formats come into play. Adobe can copy/paste layers from Photoshop and send them over to other programs. If the photoshop layer isn't understood, it seems to become a Bitmap, which works when I'm on a VBulletin forum over in Firefox land.

The same exists on X11, programs can define custom metadata and define a hierarchy of different metadata and associated data on the clipboard and any application can decide for itself which formats it understands and which it does not.

I've personally built software that puts images on clipboards, and falls back with uris to that image if direct images not be supported by whatever reads it.

It's cute just how old the concept of the Windows Clipboard is.

The basic APIs were there from the beginning in Windows 1.0, including multiple clipboard formats, application specific formats, and the ability to register for clipboard changes. More standard formats have been added over the years, but it still supports some ancient formats, like Software Arts' Data Interchange Format.

I recently looked at the API docs for Windows 1 because I wanted to see if ExitWindows was there from the start (since that naming really only makes sense if Windows has something to exit to). It wasn’t there until Windows 3, but I was surprised to see just how far back many familiar functions go. I expected the API to be far smaller than it was for 1.0

I'd love to get my hands on old Windows API docs. Are they available online somewhere?

Some are, here's the Win 1.0 and 2.0 SDK documentation:

https://www.os2museum.com/wp/windows-history/windows-library...

> "inter-process communications"

Way back in the 90s, it was possible to put any (+) document inside any sufficiently aware app or document with OLE ("object linking and embedding")

The (+) indicates that there were a lot of terms and conditions applying to this, but you could actually just put an excel spreadsheet in a word document, and it would run a small copy of excel inside word. This is why the original word save format is so horrendous - it's serialized binary objects.

I think OLE used the compound file format which was modeled after a FAT file system image.

https://en.wikipedia.org/wiki/Compound_File_Binary_Format

OLE is an awesome tech that I am sad isn't more widely used. The idea of being applications being portable toolboxes that can be embedded inside of other applications is just so cool. I remember embedding, IIRC, Print Shop Pro media inside of Word and when I clicked on the graphics the PSP UI appeared! Well after 10-20 seconds of hard drive thrashing it appeared, and then operated at about 5 fps.

Cool idea though!

On the current macOS, the screenshot can be edited/annotated when it's previewed on the mac or on a nearby iPad where it automagically appears (with edits propagating back to the mac).

Something similar is available in the more modern Windows 'snipping tool' feature (win-shift-s) but without the tablet magic.

For screenshots and edits, I have to say that I love MacOS's tools. CMD-Shift-4 to capture a screen region, click on it in the bottom right, edit it right there, and click 'Done'. Their editing tools are also super intuitive there. Honestly my favorite MacOS tool and I'd love a cross platform equivalent.

FWIW the process is about the same in Windows. The only difference is the keyboard shortcut is Windows-Shift-S.

The only real difference is that Windows introduced the simple way only fairly recently (7 years ago with Windows 10), so lots of people are still used to doing it the old way.

It's also great when you copy some text from Outlook and it pastes it into discord as an image of the text. I believe it has been fixed now but that was happening to me for a while.

I believe Windows did a lot of things right by just keeping things simple for developers. I use the clipboard to transfer large blobs of data from one program to another when doing exploratory programming -- it's a lot quicker than creating file formats for everything. Of course there's the risk of losing it in the process.

Another under-appreciated Windows feature is how they implement drag-and-drop apps. When you drag a file into an app icon, it just stuffs the file path into the command line arguments and runs the app. In contrast, though MacOS introduced drag-and-drop, actually implementing it in System 7 was a huge complex chore.

Unrelated, but you can actually use Firefox's own screenshot function which is far more robust than PrtScn, allowing you crop the image how you want (including snap to element), you can also save to clipboard or download it for further editing or storing. The key-shortcut is CTRL+LSHIFT+S.

I find WIN + SHIFT + S to be superior, because it is not limited to the browser window, automatically copies to the clipboard, and you can choose to annotate it if you want

I mean, if you are in Windows already you should be using Snip at least since Win7. It has cropping, window snap, and basic annotation, in addition to capture save and copy.

On Windows, it's now faster/easier to use the built in Snip+Sketch tool (Win + Shift + S).

It does instant rectangular selection, then pops up a simple editor which you can then copy/paste anywhere you'd like.

I have Win+Shift+S bound to a mouse chord, as well as copy and paste, so I can actually paste an arbitrary rectangle of my screen without touching the keyboard.

Also really handy sometimes is Win+PrintScreen, which puts the screen image on the clipboard, and also saves it to a numbered file in your Pictures\Screenshots folder. If things are happening fast and you don't want to paste and manipulate each screen shot, this lets you save several in a row to go back and work with later.

I've had the pleasure of looking at the Windows source code for the clipboard.

Insanely well engineered. It can perform miracles of computing. It damn well has to.

> I can take a screenshot of my Firefox with "PrtScn" button (moving a bitmap of the page into the Clipboard), paste it into Microsoft Paint, edit it a bit, and then copy/paste that into my Discord chat screen for commentary.

All without you worrying about supported image formats or color space encodings! It is insane! Copy from Firefox and then paste into a program that expects 16bit color and it'll probably work. The clipboard stores data in so many duplicate formats and can convert between so many formats, it is nuts.

I love how when you copy something onto the clipboard, it silently and transparently outputs different things depending on where you paste.

For example, copying from Excel can output:

- to a table (when copied into word)

- to rich text when copied into Wordpad

- to plain text when copied into Notepad

- and even an image when copied into Paint.

All without having to select the desired type at the source.

I do this exact same thing on linux without any issues. I use a presetup desktop environment, it's fine.

If I used arch or some other power user linux, I suspect I would have to jump through quite a few hoops to get this experience setup as nicely. But as someone who daily drives both a linux distribution and a windows distribution, I see no notable differences in clipboard functionality.

The Old New Thing is such a treasure of insights into how Windows works (or used to work).

And Raymond tells the story in a fun and engaging way. I really look forward to a volume 2 of The Old New Thing book (a compilation of his blog entries)

It's interesting how The Old New Thing is clearly a passion project for Raymond, but it's also an essential part of Windows documentation. I've run into a lot of situations where Raymond's blog is way more useful than the official docs.

I love how I read the subject and knew it was a Raymond Chen post.

Raymond's blog always hits a note of morbid fascination. At times it almost feels like schadenfreude. "Neat, but thank God I didn't have to be the one that wrote/discovered that."

The Old New Thing and Dave's Garage (on youtube) have helped me develop an appreciation for the decisions made in Windows development, kinda wish we had this kind of stuff from the Mac and Linux side of things.

I must admit, just by reading the title and the domain on HN's homepage I was dead sure it was a Raymond Chen post

Wouldn’t it be nice if it threw an error, and didn’t just return NULL?

Itd be nice if the proviser were to show a dialog that a copy was requested but couldnt be provided. Altough, dunno what a good message for that would be.

Very old school windows apps like SecureCRT display semi-helpful messages for this. Better than a silent failure anyway.

I don't think low level system APIs when called from an App should be messing with UI components. Can you explain what is better about it?

I don't think it is the system level API, but rather the program receiving a notification that the copy paste didn't succeed where the program can then display an error/warning. At least from my understanding.

If you get NULL, you are supposed to call GetLastError to get information on what happened.

How would you like this C API to throw an error?

The world is run by one giant excel spreadsheet

It’s a spreadsheet world and we’re just living in it ;)

The world's mail definitely is!

https://youtu.be/dHhkNwE7pr8

Follow up post: "How can I wait more than 30 seconds for a delay-rendered clipboard format to become rendered?"

https://devblogs.microsoft.com/oldnewthing/20220609-00/?p=10...

I find the size always seems to run run parallel with my patience at that given moment

> Next time, we’ll see what we can do to extend this timeout.

That, somehow, seems like the wrong conclusion. Why not try to solve the problem of Excel taking longer than 30 seconds to export a RTF table?

Given the range of computers on which Excel is likely to be running that seems like a complete non-starter. There will always be cases that take longer than X no matter what you choose for X.

Yeah, why not display a "Exporting..." dialog then that the user themselves can cancel if they think it's taking too long? Extending the timeout to X won't solve anything.

When you press Ctrl+V, the target window is focused. Forcibly switching back to Excel to show this dialog seems weird and confusing to me.

Then why not show a system dialog similar to copy/move files dialog after e.g. 5 seconds of waiting for a copy-to-clipboard operation?

Actually, now that you point out that this could be done, I’m surprised it wasn’t.

Cool, now it's optimized and it manages to export up to a million lines in 30 seconds. What do you propose to do when the next customer wants to copy 1.5 million lines?

> What do you propose to do when the next customer wants to copy 1.5 million lines?

Thankfully, that can't happen :). XLSX has a row limit of 1,048,576 rows.

In all seriousness, I don't have an answer for you other than that the developer that's looking to shove >1M excel rows around should find a better solution than the clipboard. I've done more than my fair share of work pushing excel spreadsheets around (and I've even worked on Excel add-ins). There are plenty of performant ways of accessing large volumes of Excel data, none of them involve the clipboard. Most of them even work on non-Windows platforms.

The arbitrary row & column limits in Excel (XLS was pretty small, XLSX is quite a bit bigger, but still arbitrary) are annoying and I cannot begin to tell you how many hours over the years I've wasted due to this limitation or Excel silently truncating large CSV files on import.

Indeed - it would be very possible to make a version of Excel which can deal with infinite-sized sheets by leaving most of the data on disk and only loading a few screenfulls at once.

"Very possible" reads to me a bit cheat-y, in that you could do so but you probably don't want or need to. Paging a workbook out to disk in this way necessitates keeping a cell-to-cell relational map for everything and then paging it back in when you then need to pointer-chase through a set of updates--well, you've just reinvented virtual memory but worse, haven't you?

Excel is an intensely business-critical product and has a lot of smart folks working on it. There are tradeoffs you're eliding and I'm confident that they know of them.

People said the same about text editors. But there are quite a few text editors that will happily open and edit a 1Tb text file.

Editing a text file can be done in such a way that editing/adding/removing lines has no effect on any untouched lines. In an Excel spreadsheet, changing a single cell can result in a cascade of updates all over the place, in a contrived scenario it could result in all other cells being updated.

A text editor is not a relational system with cascading updates.

I don't think its realistic to expect a Windows developer to solve a problem in Excel!

I'm actually quite surprised that it takes over 30 seconds to go through only 300000 items and generate some text with them. Unless the story is from ancient times, that is.

300k rows but who knows how many columns.

30 seconds seems awefully long indeed and it's probably not on a modern / 2022 computer, but RichText clipboard data in windows does a lot of processing to render in a way that fit context, eg copy a few excel row, paste them in whatsapp webapp and you have a picture, not text.

As a tech user, you quickly learn to fall in love with "paste as raw text", which skips all that and just give you text data.

You would think that, but try injecting 5K rows table into html document. My experience in Chrome and modern 4GHz CPU:

'6K rows, 300ms to insert into page on paper (console.time(1); console.timeEnd(1)), but in reality browser freezes for 3 seconds (1.8s style, 900ms layout, 300ms update tree). Freezing goes away with position: absolute, but it still takes 3 seconds to show up after .appendChild. I tried replacing Table with Flex divs, even worse speed.'

I imagine I could make Chrome hang for more than 30 seconds with plain old HTML Tables :)

> Next time, we’ll see what we can do to extend this timeout.

I guess I'm impatient... how does one go about this?

I don't know, but I'd except a follow up post from Raymond on this tomorrow. He often does these multipart posts, broken up in rather digestible chunks.

There shouldn't be an arbitrary timeout. Anything that isn't instantly should have a UI showing progress / ability to cancel (or even pause).

I believe there is such an API... If you copy a file out of a rar archive and paste it into Gmail, you get to see the UI pop up and extraction begin...

tl;dr: Yes, but its measured in seconds.

wild.

Reply on news.ycombinator.com