Deep cloning objects in JavaScript
builder.io
builder.io
- https://github.com/tc39/proposal-record-tuple
- https://github.com/tc39/proposal-deep-path-properties-for-re...
CLJS: ¯\_(ツ)_/¯
The goal of #-private properties in JS is to have total runtime isolation, so that there's no way at all to access it from the outside. If you're fine with a compile-time error, use TypeScript's private keyword instead (JS does not have a compiler).
But why can't `otherObj.abc = 3` throw an error if abc is declared as private? Because it would incur a runtime cost on all property accesses to check whether it's public or not, even if private properties aren't used anywhere within the class. Not a sacrifice anyone would be willing to make.
So, the solution is to make sure that private property accesses can be distinguished from public property accesses. You could have had something like `private.abc` instead of `this.#abc`. But I don't think that's better, either.
Honestly, you'll get used to # if you use it for a while.
structuredClone, if I'm not mistaken also allows cloning functions/Maps/Sets, so while there may be some overlap, I'm not sure having Records and Tuples solves the same problems. Or as nerdy engineers are so prone to say, they're "orthogonal".
Map and Set yes, but not functions. Makes sense because there is no reasonable way to serialize functions in JS.
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
edit: I have no idea what actually happens to cloned maps with function values or keys though...
And structuredClone was standardized to serialize objects passed between contexts.
So I don't see what's wrong with teaching people about this as the native way to clone serializable objects.
Record/Tuples are different types and I don't think that "what you really want" is a future proposal when you want a built-in now.
Misuse of deepClone & co is just a very common antipattern.
Or when working with libraries that demand this.
In that spirit, I find MobX pleasant to work with as an alternative.
Although it remains needed to grok reference equality checks in reactivity, there's no way around that I guess.
Fundamentally, when passing messages with structured payloads I'd consider it good style for the payload to be serializable and free from references and mutable state.
As an unrelated aside, recursive proxies would also be extremely useful
[0]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
"instead of learning x, use y library" Sometimes that advise makes sense, yes. But it's constantly used in a dogmatic manner.
I also think the automatic conversion of a `Date` to a string is a bad thing, which again is prohibited in Python.
The issue, if there is one, is that nobody ever got around to implementing JSON.serialize.
> JSON.stringify({a: ()=>{}})
{}
> JSON.stringify({a: /b/})
{"a":{}}But I think in the case of JSON.stringify it’s more about use case. 99% of the time, users of this method are taking some data and serialising it to a JSON compliant form to send to the server. JSON doesn’t support functions, or complex objects like a Date, so I tend to think it’s a reasonable default that functions disappear and Date’s are converted to an ISO standard. To insist that every single user runs a preparation step that strips out unserialisability data and chooses how to handle Date objects sounds laborious, error prone, and ripe for another npm dependency everyone suddenly normalises for every project.
Maybe a “strict mode” of some sort where you could have it throw on anything for cases where you need to guarantee everything is being sent?
OTOH, I have to concede that while this method has silent failures, they then implemented JSON.parse to throw at the slightest issue. So I have to admit there’s consistency even within the API.
You might prefer some well established standard implementation over ad hoc roll your own, but that's a discussion about npm culture, not about stringify.
Now now, it's only 1M weekly downloads: https://www.npmjs.com/package/superjson
The problem I see with this is that whatever you’re sending this to must have knowledge of the meta information superjson produces, so at that point you’re investing in it as a wrapper library. The fact you can extend the types it serialises also complicates things and means the receiver needs further implementation specific knowledge.
I think in my original comment, I was imagining a world where JSON.serialize threw errors on unknown types and we needed a wrapper just get basic JSON out of it.
[1]: https://www.builder.io/blog/structured-clone#why-not-code-cl...
Not because it worked well, but because it was _good enough_. I feel like that in itself is an important lesson about our industry and probably the world.
That implementation eventual became obsolete (although it still runs on basic object types) because of newer features being added to the JavaScript language, and by then other implementations were available. It's great to hear a de facto standard is emerging, even if hasn't made it into ECMAScript standard yet[2]. It sounds like the same edge cases are still causing problems. I think if a language is to ever offer really great support for deep copy it must be designed into the language as a first-class feature from the get go.
[1]: https://www.oranlooney.com/post/deep-copy-javascript/
[2]: https://es.discourse.group/t/structuredclone-as-ecmascript-s...
> VM187:1 Uncaught DOMException: Failed to execute 'structuredClone' on 'Window': () => {} could not be cloned.
Ah, yes. I actually asked this question on StackOverflow 5 years ago [0], and the reason it can't be cloned isn't the worst reason ever... but it basically breaks cloning any non-trivial object. Well, I suppose I'll check back in another 10 years when the next ecmascript proposal around cloning lands...
[0]: https://stackoverflow.com/questions/51939812/how-to-clone-an...
You have a function, and that function could have local bindings to arbitrary local variables.
JSON defines what is possible to do that doesn't involve arbitrary bindings to the rest of the runtime. Short of serializing the world, JSON is the best we can do.
It would be possible to try to serialize, and maybe a function is simple enough to serialize. Do we have existing examples of this? There's hundreds of different serialization libraries. For some reason everyone in this thread is super ignoring what seems like a basic widespread limitation, is being condescending to structuredClone, for not doing what no one else does either. Because there's a very good reason: because here be dragons. I don't get why everyone is so combative & aggressive over what seems so clear.
> I don't like how short you are, how you phrase this as some huge weakness.
I think you may be reading into my comments more than what is actually there. I think it's a reasonable enough compromise, and I'm happy to have structuredClone.
function f() {
f.count ??= 1;
console.log(f.count++);
}
f(); // Outputs: 1
f(); // Outputs: 2
f(); // Outputs: 3https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
> Structured cloning algorithm defines the semantics of copying a well-defined subset of ECMAScript objects between Code Realms. This algorithm is extensible by host enviroment to support cloning of host objects.
https://github.com/dslomov/ecmascript-structured-clone
Eventually, in 2015, it was suggested to expose the `structuredClone` algorithm as an API:
> Has anyone ever proposed exposing the structured clone algorithm directly as an API? Katelyn Gadd was musing about fast deep copy in JS on Twitter, and I proposed a hack to use postMessage to do so[1], which works but it's a little roundabout. Since structured clone is a primitive that the web platform is built on it seems like a primitive that ought to be exposed. I know this exists in other languages (Python has copy.deepcopy[2]) and there's an npm "deepcopy" module[3] with lots of downloads so this is clearly something people use.
https://lists.w3.org/Archives/Public/public-webapps/2015AprJ...
To me, deep clone implies that I will get back an exact copy of what I put in. structuredClone does not make such guarantees.
"Structured clone" is enough to give you pause and question "what could structured mean?" (just as you did), at which point you will read the docs and find out. It succeeds in communicating what it needs to.
What gave me a pause (and would be part of "yet another bump on the road to intuitive use") is the disconnect between the awareness of the better naming and the choice of a worse one (so, in a sense, this hardest problem has already been solved by other computer scientists!)
The actual docs do. This second-party attempt to recreate the docs in their own way that you point to has fallen short, I agree. Granted, Mozilla does a better job than some of the other second-party recreations, like Deno's. There is little question that the Javascript ecosystem is a bit of a disaster.
That said, I clearly see the caveats right there up front and centre, so even still it has successfully managed to provide you with what the function name wants to bring to your attention.
https://github.com/lodash/lodash/blob/main/src/.internal/bas...
For example, the code imports "copyArray", a 9LoC function that does exactly the same as the built-in Array.slice() would do.
Same goes for other imported helpers like "arrayEach", which could be replaced by the built-in Array.every().
It's basically a bunch of unnecessary polyfills for built-ins.
On the other hand, when you consider #privateMembers, you realize cloning a class in JS is basically impossible (and adding them to the language was probably a horrible mistake).
[1] https://developer.mozilla.org/en-US/docs/Web/API/structuredC...
What's the use-case?
But it's not necessarily related to frameworks; if you're working with complex enough data structures and you're following a functional approach, you'll need to do similar things sooner than later.
Why does that require deep clones?
I simply do:
return {
...current
foo: "bar"
};Maybe you're handing over your data to a library developed by someone else and you want to make sure the library cannot mutate your original data, so you opt to pass a deep copy instead. Or maybe you are the author of said library and you want to make sure you preserve the original data so you copy it on input rather than on output.
There are many situations where deep-copying is useful but I agree that you should use the simplest pattern that works for your use-case.
Maybe it's not a situation that comes up often, but it would be fairly hard to debug and guarding yourself against mysterious problems in advance is always neat.
I have never needed to "deep clone" an object in JavaScript. Any time I thought I needed to do that, it was actually a code smell for a different underlying problem.
I was glad to see the end of the "immutable everything" dogmatic obsession wrought by Redux. The code smell in that case was usually related to passing entire objects as props.
Can anyone describe a compelling use case for "deep cloning" objects in JavaScript (keep in mind _everything_ is an object in JavaScript) that isn't dancing around some hidden complexity? IMO, if it can't be (de)serialized via JSON.stringify and JSON.parse, then you should consider why you're trying to serialize it in the first place, and why you're scared to pass around the reference to the object. (That said, structuredClone looks like a better alternative to the JSON method - but note it's still not cloning functions, so you should still be careful where/when you use it.)
Chrome dev tools logs a pointer to the object, so if the object is mutated after logging, but before you view/expand it (especially for nested structures), it can be very confusing! JSON.parse(JSON.stringify()) usually is good enough for this :)
But yes, generally speaking I think a deep clone is a sign of a smell elsewhere. That doesn't mean it never has a place in any code, but just that if I find myself reaching for it, I usually see if there's a mistake being made elsewhere.
So if between two logs it was modified, it reflects the new status on the log before as well.
I think you may have been confused and got how it is, vs how it ought to be the wrong way round.
https://twitter.com/justinvincent/status/1714866433426067573
Yes. That's why you need to use structuredClone. If you don't, and you mutate the object, then you'll see the new values when you expand the object.
Sure. A library accepts an object as function argument (for example ‘params’) and then uses it inside. Modifying users object in-place would be a bad idea.
[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
The full idea is implement a deckbuilding game like magic and use genetic algorithms and montecarlo simulation to balance the card pool. But that requires millions of games with millions of simulations to work. So i'll see if it's even possible.
For inspiration, many chess engines store game state in a "bitboard" [0] which is a 64 bit representation of some (partial) state of the board.
Of course, at this point you might not even want to be using JavaScript... maybe you could write the hot path in Rust, compile it to wasm, and then use JS for orchestration/UI.
The issue is I want to figure out if there are combos that I don't even know exist that are unusually strong. Basically make sure I balance the card pool but automatically.
Yeah :) Definitely better to make it slow but ship it. Then make it fast later.
Deserializing requires paying all the same costs as deepcopying, both in terms of compute time and in terms of memory usage, but it also incurs the extra costs of parsing.
I'm not saying that there wouldn't be benefits to having a serializer for their gamestate (because there would surely be benefits), but I just cannot understand this as an alternative for the problem they described.
First, many "everything must be immutable" libraries provide data structures that give operations that feel like mutations (but actually are not), and many of those can re-use all the old objects that are not mutated. This could save a lot, relative to full naive deepcopies. Or it could save very little, depending on what you're doing.
Second, if you have the ability, you can always mutate before recursion and then revert when the recursion finishes. This could potentially be very tricky, because it's not always easy to revert mutations. As such, it's not a good plan for a prototype, but if you can get it right, this could make your tree search wayyyyy faster.
Communicating with workerthreads.
I so much would like to pass references of big readonly objects towards them - but no, full copy it is, for every thread.
[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
(but I hope I missunderstood something there)
But I need many threads to access the same data, at the same time (readonly, so no racecondition).
You might be mixing ArrayBuffer (which is a transferrable object[0] and gets zero-copy moved) and SharedArrayBuffer.
From [1]:
> The structured clone algorithm accepts SharedArrayBuffer objects and typed arrays mapped onto SharedArrayBuffer objects. In both cases, the SharedArrayBuffer object is transmitted to the receiver resulting in a new, private SharedArrayBuffer object in the receiving agent (just as for ArrayBuffer). However, the shared data block referenced by the two SharedArrayBuffer objects is the same data block, and a side effect to the block in one agent will eventually become visible in the other agent.
[0] https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
No. And I hope to be wrong.
I will give it another try then, but I was using SharedArrayBuffer and I was getting explicit errors about this limitation.
Caveat is you need these headers for security reasons (hence why the sandbox above has Express) or SharedArrayBuffer is not even defined (at least in Firefox):
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corpHierarchical data structures, in mindmaps, menu structures, block based notes apps, etc.
For example, I create a revision of a store (a deep clone) and apply the undo frames one-by-one.
Each undo frame has the setter name and its arguments. IOW, the frames contain what changed, such as:
[
[BASE.setCard, 'some-id', CF.title, 'The New Title'],
…more frames
]
Here's a post with more detailshttps://blog.uidrafter.com/architecture-of-a-desktop-alike-s...
Also these days there's BigInt.
As in, it’s amazing that anything actually works given the internal inconsistencies in its core.
(Disclaimer: I code ts/js for a living.)
If an object is immutable and you want to copy/mutate one item it then you can surgically clone it in O(how deeply nested the item to change is). Object spread being the simple case.
JS makes this messy to do though! Lots of spreads and array maps or you pull in a library like Immer and now have “coloured” objects. Immutability in JS is not idiomatic so for that reason I try to avoid unless I really need to.
I could do some of the changes in place, but to optimize Vue reactivity I work on copies.
I use prototypes so only lodash works for me.
If you’re so inclined you can google the benchmarks.
Instead of saying we should search for benchmarks why not post them here? Also, what is the small function you use?
I just did a test again with 40000 times cloning a simple object and JSON.parse/stringify was 6 times faster than structured cloning.
Since I heavily use deep cloning, I am still disappointed with the new native, but slower solution. (And was hoping there would be more than old news in the article)
Edit: Code for testing was as simple as
for ... 40000
JSON.parse(JSON.stringify(obj))
vs
structuredClone(obj)
Depending on the object you're cloning, that might be fine. But some types, like Date or Set, don't round trip between JS and JSON so you'd have mangled objects.
Can you enlighten us?
JSON.parse and stringify, or customly copying what is needed? (the fastest solution to my knowledge)
https://www.measurethat.net/Benchmarks/Show/17150/0/lodash-c...