Copying objects in JavaScript | Hacker News Reader

Copying objects in JavaScript

smalldata.tech

124 pointswheresvic192 comments

I've been programming JS for a very long time and have learned to just stop trying to do a generic deep copy. Since JS is a dynamically typed language, it will always lead to issues down the road. Instead I write domain specific merge methods for whatever objects I'm merging.

    function mergeOptions(...options) {
      const result = {};

      for (const opt of options) {
        result = {
          ...result,
          ...opt
          arrayValue: [
            ...(result.arrayValue || []),
            ...(opt.arrayValue || [])
          ],
          deepObject: {
            ...result.deepObject,
            ...opt.deepObject
          }
        };
      }

      return result;
    }

Know the shape of your objects and merging deeply becomes painless and won't have edge-cases.

Another way of looking at it: if you are frequently doing complex copies, you probably want immutable types.

Im away from my usual PCs, and Im here just to say when I realized this exact same thing it all clicked for me on why to use immutable objects

Immerjs is a very handy library for doing this kind of thing. It is a natural fit for react but can be used for any kind of copying like this.

+1 for immer. I love it.

If anyone here ends up using it, make sure you learn how it works first. There are performance implications. It may not matter but it's important to know what they are. Also it's generally best practice to learn how magical utilities like immer do what they do before using them!

I think it's not the best idea to keeping the shapes of objects in your head and manually cloning/merging them.

This will lead to bugs, as inadvertently as a human you'll miss a merge or a clone and retain references that you don't want.

Inability of a language or runtime to correctly and quickly clone a structure is an upsetting fact of JavaScript.

As of version 8.0.0 node has exposed a serialization api which is compatible with structured clone. https://nodejs.org/api/v8.html#v8_serialization_api

    const v8 = require('v8');
    const buf = v8.serialize({a: 'foo', b: new Date()});
    const cloned = v8.deserialize(buf);
    cloned.b.getMonth();

Have we learned nothing from Java's serialization fiasco?

I want to learn, can you elaborate?

https://dzone.com/articles/jdk-11-beginning-of-the-end-for-j...

Java's deserialization will instantiate any classes that the data tells it to, which in practice leads to myriad vulnerabilities as many classes have constructors that can be used to write files, execute shell commands, etc. Many programmers didn't realize this, and bad things happened.

This is a classic example: https://www.cvedetails.com/cve/CVE-2015-7501/

(Many more can be found under the CWE-502 "Deserialization of Untrusted Data" category)

I don't know how the JavaScript proposal does it, but you can certainly create generic clone structures that are safe for untrusted input.

That's awesome, I'll update the article to reflect this!

Using structured cloning for deep copies is clever, but may or may not give you the behavior you want for SharedArrayBuffers. The copied value would be a new SAB with the same underlying data buffer, so that changes to one value will be visible to the other. That's good for most uses of structured cloning, but it's not what I would expect from a deep copy.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

The point of structured clone was originally for passing data to web workers. Since the point of shared array buffers is to share data with workers, it makes sense that the structured clone algorithm keeps the SAB identity.

Programs that use SharedArrayBuffers are already defective, so it's no big deal.

Not true. Chrome has enabled SharedArrayBuffers again as of Chrome 68.

Desktop-only, not on Android, right?

Also, a program that only works in one browser, and only due to that browser being willing to open security holes other browsers are not willing to open, is arguably still defective. That said, that might describe a lot of things people are doing nowadays (e.g. anything that uses WebUSB).

Fundamentaly, deep-copy in Javascript is a typed operation and no "generic" deep-copy algorithm is possible--you have to know what meaning the value is supposed to have to copy it.

There's nothing inherently wrong with structured clone, but it's only for JSON-able objects with extensions for circular references and some built in value-like classes. It's also special-cased for safe transmission between Javascript domains so it has a bunch of undesirable behavior for local copies. (no dom, no functions, no symbols, no properties...)

Even primitive types can't be safely copied unless you know what they're going to be used for later, a nodejs file descriptor is just an integer but it's also a reference to an OS resource that can't be duplicated without a syscall.

> Fundamentaly, deep-copy in Javascript is a typed operation and no "generic" deep-copy algorithm is possible--you have to know what meaning the value is supposed to have to copy it.

Why? I see no problem if the deep-copy behaves exactly the same as the original, from the perspective of any operation in the Javascript API (except for the === operator).

Exactly the same as the original is a shallow copy.

Deep copy roughly means "If I do `dst = deepcopy(src)`, modifying anything in the world I can reach through a reference from dst should have no side effect visible through src" which is a reasonable thing to ask in some special cases (a tree of plain old javascript values, a DOM node) but not reasonable in others (a database connection object, a file descriptor, a user-id)

Like, what should a deep copy function do if it reaches a reference to the global object? or a function that closes over some references in the object being copied?

The answer is always "it depends on what the object will be used for later and what specific behavior you want"

The === operator is a coercion-free equality operator. It's not an object identity comparison operator.

But for `Objects` (not primitves) it is the identity comparison operator, right?

I wasn't aware of that. I thought ES2015 added Object.is() for this purpose: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

The main difference between === and Object.is() is that === treats +0 and -0 as equivalent, despite the fact that some math operations treat them differently. I think they added Object.is() because it was awkwardly difficult to make code tell the difference between +0 and -0.

Yes

technically for primatives too, since they are interned, immutable and their value is their identity?

It is fundamentally very hard (impossible?) to deep copy everything in JavaScript. References cannot be escaped because they may be hidden in non-introspectable (cruicially, non cloneabel) places. Viz:

  function f(){var x = 0; return function(){return x++;};}
  var x = {foo:f()};
  print(x.foo());
  var y = {foo:f()};
  print(y.foo());
  var z = someDeepCopy(y);
  print(z.foo());
  print(x.foo());
  print(y.foo());
  print(z.foo());

If a copy were sufficiently deep then one could expect:

However if it were not deep one would get:

Even if one allows a deep copying of closures then this still might not work as an object which contains two (potentially different) functions closing over the same binding (ie particular instance of a particular variable) may be copied into two functions each closing over their own separate binding.

I think the only good solution to this is to either give up trying to do deep copies or give up immutability and stop caring about deep copies.

The language really should have a true immutable type (without freezing, etc.) and deep copy method built in, with as many caveats and parameters as needed. Coroutines would be awesome as well. (Yes, I'm thinking, "How could JavaScript be more like Go or Erlang?")

And then it needs to stop adding new features for at least a couple years so the world can catch up.

i guess most languages do not add this because of circular reference problem?

I infer you don't mean to have both immutable data structures and deep copy as features to use together, since immutables don't need to be cloned.

You can implement coroutines using generators. Is there any feature you'd miss from other implementations if you used generators in that way?

var copy = JSON.parse(JSON.stringify(myObj));

Anything beyond this and you are begging for trouble because there's always context-specific gotchas.

This is presented in the linked document with the following context-specific gotchas:

> Unfortunately, this method only works when the source object contains serializable value types and does not have any circular references. An example of a non-serializable value type is the Date object - it is printed in a non ISO-standard format and cannot be parsed back to its original value :(.

That's false. If you use JSON.stringify on an object, it will call the method toJSON of each values recursively. The toJSON method of a Date returns the ISO string.

source: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

But JSON.parse will leave it as a string instead of converting it back to a date, so this cloning approach doesn't work for objects containing dates.

I did not say that JSON.parse would parse the dates.

I'm saying that the article is wrong when it says that JSON.stringify does not transform Dates into ISO string.

Right after that, the article says "cannot be parsed back to its original value" when it definitely can be parsed back. JSON.parse does not do it by default but that was never his point.

> Right after that, the article says "cannot be parsed back to its original value" when it definitely can be parsed back. JSON.parse does not do it by default but that was never his point.

The broader point, which is entirely correct, is that you don't get back an exact copy of the object you want to clone, because the date fields don't end up being the same type.

JSON.parse of an ISO string yields a string, not a date.

let d = new Date(); let s = JSON.parse(JSON.stringify(d));

s will not be a clone of d.

I never said that JSON.parse yields a date by default. But it can.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

The article says it does not return a ISO string. I'm saying it does. The author as agreed with me and will update the article.

Why? Why would people want to clone objects? When I have encountered this in the past it is from people who are new to the language.

My advise to any person who really believes they need a clone of an object: do some self-reflection on plan as to why you think you need a cloned object. Any other approach is more efficient and more simple in the code.

Objects are hash maps that store data.

Yah that's why I use the serializable override:

someObj.toJSON = function() { return { foo: this.foo, bar: this.bar } }

It is an extra step but when using redux or something like that you have to serialize stuff anyway to store it and is especially useful for mobile react-native stuff in keeping state when phone restarts or connection fails.

Another option if you can afford it is to just use immutablejs.

Or make your functions return new objects.

I suppose immutable.js supports partial reuse of objects, that is, if you only change the value of one attribute, the changed copy has this attribute set differently, but the rest is shallow-copied?

If so, indeed immutable objects would not run into the problem of copying, as long as you can afford them to be immutable. (That is, you're not working with any APIs that assume and use mutability.)

I cannot recommend ramda enough. It provides the immutability and flexibility of immutablejs, but because it's build in a functional paradigm, the logic is fully composeable so complex, deeply nested changes are very simple and readable.

That is an apples to oranges comparison. ramda does not provide persistent data structures.

both are used to approach the same problem: immutable manipulation of javascript data. The fact that immutableJS provides a persistent object is an implementation detail in the strategy used to solve that problem.

I would recommend immer (https://github.com/mweststrate/immer) instead of ImmutableJS. You can work with regular JS objects, plus it plays much nicer with TypeScript.

I wrote a tiny library called fn-update for this that does composable functional updates (https://github.com/benvan/fn-update).

Personally, I prefer not having to mutate data, even wrapped in a produce method

Having to specify the path using an array of strings would be an instant dealbreaker for me. It messes with all tooling and is bad for readability.

ImmutableJS is a pretty huge library. I'd conjecture most web apps don't have a legitimate need for something so comprehensive and would be better off with a simpler solution.

If you know the shapes of your objects ahead of time you can create one-off functions, which will probably be faster and require far less code.

Granted, this is Python, but I wrote this a while back: https://github.com/scooby/pyrsistent-mutable

Basically, it's an AST translator that lets you use imperative syntax against immutable types. That is, `x.a = b` becomes the clunky `x = x.set('a', b)`, and it really gets convenient when you have complex structures.

Would it be worth it to look into a babel plugin for Javascript and ImmutableJS?

> objects in Javascript are simply references to a location in memory

No variables are simply references to objects. Objects aren't references - they're referents.

I like the shallow copy, and never needed a deep copy. I am using JS for a few years tops, mostly React.

To me its exactly what native languages do with pointers. In some languages (like Delphi) its implicit (like JS) and some (like C) its explicit in syntax.

Been using this for a while: https://stackoverflow.com/a/44612374/663447 Best clone imo (for the correct usecases).

I followed your link, which has links to two perf sites, which show that Object.assign is nearly 20x more performant than your preferred solution.

Object.assign doesn't actually clone anything.

Don't use it with dates, it breaks them. You'll lose the data.

# var a = new Date();

# cloneDeep({ a }).a === { a }.a;

This returns false. Use JSON.stringify if you care about the content. cloneDeep might be useful if you don't care about data integrity.

JSON.stringify also breaks dates.

No it does not. It returns a ISO string. You don't lose any info. And you can parse it back with JSON.parse if you give it a reviver function. [1]

Do you want JSON.stringify to keep dates intact ? The whole point is to turn it to a string.

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

If you are writing a reviver, what difference is it to instead write a deep clone function?

JSON.parse(JSON.stringify(obj)) !== deepClone(obj)

EDIT: Also, how can you determine if the string should be a date, or a string? Sure, if it fits, you can always convert it to a Date, but if your first object is coming from something that sends ISO Dates as strings and you used JSON.parse with a reviver, you would get a different object.

"It works if you provide your own additional code that loops over and fixes known problems" is not the same as "it works". It's useful, but at most it saves you a few lines of boilerplate code (and perhaps a few negligible cycles) to write your own recurser to do the same thing immediately after the transform.

> Do you want JSON.stringify to keep dates intact ? The whole point is to turn it to a string.

There are ways to serialize data structures to strings. JSON doesn't generally do that, and it's been to its benefit as it helps keep it generic and easily usable from many languages.

That said, there are serialization tools which can correctly serialize and unserialize more complex structures, given the correct circumstances (only core constructs, or the objects in question have serialization helpers, or they are ensured to not have features that might cause problems such as references to outside data).

The point of this discussion is Javascript object cloning, not turning objects into a string. Offering up JSON and then responding to a criticism of it based on its methods as "the whole point is to turn it into a string" is somewhat ridiculous given many of the other options you're espousing JSON over don't use strings at all.

The whole point is to clone an object. You lost track of the whole point.

Fun fact: Not even the whole of number type is safe to clone with the JSON method, because Infinity or NaN turn into null.

So one can’t infer JSON-clonability from TypeScript/JavaScript types. Learned this the hard way.

Premature optimizations is the root of all evil. That said, creating new objects do slow your code down, so do it only when you have a good reason to.

Every single time I see a similar article on Javascript, I feel so lucky for being able to use Clojurescript instead. Seriously - Javascript platform is great. The language itself? Not so nice. Clojurescript makes so many things simply better.

Never do any of this. It's not 1990 anymore, we don't copy code from random blog posts to solve problems.

> x = 4

> y = new (x.constructor)(x)

[Number: 4]

> x.constructor

[Function: Number]

> y.constructor

[Function: Number]

> typeof x

'number'

> typeof y

'object'

> x

> y

[Number: 4]

Is y a clone of x?

no its boxed. take `y.valueOf()` and you've got a successful clone.

One of the weird, interesting parts of JS. Great article.

20 years on, and still no native object copy and merge in JS

Isn't Object.assign basically a merge?

I think the parent meant a deep copy/merge—Object.assign only does a shallow merge, which is the point of the article.

What language does have a deep copy/merge operation built into the language?

Most(all?) functional programming languages.

Most functional programming languages don't make a distinction between values and references, so this is kind of a moot point since the copying/merging is never exposed to the developer.

Rust's `Clone` trait is deep by default, except for

- Types with shared semantics (e.g. Rc<T>, Arc<T>), or holding onto these internally

- Types holding borrowed references (these will still reference the same data).

C++'s copy constructors are too, but there are more caveats, although they're similar in principal to Rust's caveats.

Rust has a Clone trait which provides the .clone() method, which isn’t a shallow clone or a deep clone, due to Rust’s ownership model—the terms “shallow clone” and “deep clone” actually don’t make sense in Rust.

For completeness, I must mention that when you get to types like Rc<T>, deep cloning becomes a meaningful operation, but even there it’s not exposed as a deep clone method, but rather via make_mut (https://doc.rust-lang.org/std/rc/struct.Rc.html#method.make_...) which skips the deep part of cloning if there are no other references to the inner value.

Rust’s ownership model is absolutely delightful to work with. I miss it all the time when working in Python or JavaScript, and encounter and write bugs that would have been structurally impossible in Rust from time to time—to say nothing of inefficiencies that would have been either inexpressible or unreasonable in Rust.

Python: https://docs.python.org/3.7/library/copy.html#copy.deepcopy

47+ years, and still no native file-descriptor copy in UNIX/C.

dup, dup2?

That article is wrong. A date can definitely be serialized in JS. It is in fact converted to a ISO string when it is transformed into JSON, but the article says it does not. The author needs to learn about JSON.stringify and the toJSON method of built-ins.

Indeed it can, but I believe the point is that it's not round-tripable, meaning you can't parse the resulting serialized JSON back into its original form, the one that contained the Date objects.

If you could do so, JSON.stringify/parse would be a convenient way to do a deep clone.

Yeah that's right. All I'm saying is that no matter what, the following sentence is false:

> An example of a non-serializable value type is the Date object - it is printed in a non ISO-standard format and cannot be parsed back to its original value :(.

It would be cool if there was a webpage for “Is your function ______?” with a list of things that functions can be:

- Roundtrip-able (reversible?)

- Pure

- Effectful

- Mutation-doing

- Idempotent/Nullipotent

- Algebraically closed over the set of its inputs

- Et c.

Every time I hear about a new one I wish I had such a list

> - Roundtrip-able (reversible?)

You mean injective

> Roundtrip-able (reversible?)

Involution, if you mean calling a function twice will give the original input.

Aha yes you are correct - it outputs an ISO string at the very least but does not parse it back to a date. I will update the article to reflect this!

Ok but please read carefuly because JSON.parse can in fact recover dates. The JSON.parse function accepts a 'reviver' function as an argument to do just that. 99% of the time it's not useful so there is no reason for JSON.parse to do it by default but JSON.parse definitely can parse dates back if you use all it's features.

I like that it does not do it by default but I can make it parse the dates if I want.

Saying that JSON.parse can do something when it's really just you implementing the transformation from string to date object is a little misleading.

You can make JSON parse arbitrary sublanguages if you're willing to put in the work.

That way you can get different result, as you could actually have date object and date as string stored somewhere. You will need custom strigify callback to differentiate which should be converted back to date on parse and which should be a string.

Still, this is not pure json this way, it's your own transformations

And what do you pass as the reviver argument? Your own function to check a string to see if it's valid IS8601 and parse it? That's just using parse()'s recursive walking of the object: it does no date reviving itself, right?

Reply on news.ycombinator.com