Hacker News Reader

Comment by ambulancechaser | Hacker News Reader

can you explain?

    ((juxt type identity) (clojure.edn/read-string "x"))
    [clojure.lang.Symbol x]

It seems that the reader returns symbols just fine.

View on HN

I don't use Clojure; I have no idea. If two occurrences of "x" are mapped to the same object, that is interning; maybe what that does is use its own package-like namespace, separately allocated for each call.

The documentation for EDN says that "nil, booleans, strings, characters, and symbols are equal to values of the same type with the same edn representation." The only way two symbol values can be equal is if they are actually same symbol, I would hope.

> The only way two symbol values can be equal is if they are actually same symbol, I would hope.

Why is this important? Specifically, why do symbols need to be interned?

In Clojure, "Two symbols are equal if they have the same namespace and symbol name." In general, "Clojure’s = is true when comparing immutable values that represent the same value, or when comparing mutable objects that are the identical object." [1]

[1] https://clojure.org/guides/equality

Answered above in https://news.ycombinator.com/item?id=23763071 symbols are compared using eq (same object)

eq: http://www.lispworks.com/documentation/HyperSpec/Body/f_eq.h...

If we read two symbol tokens from a stream, and the lowest-level equality function that is available to us does not distinguish them, then they are interned.

Because symbols are used to refer to things, whether or not they are mutable can be blurry. You can make symbols as immutable as you want, but as soon as you make one of those symbols a key which refers to a mutable object, such as a global environment, then effectively, the symbol appears as a gateway to something mutable, and you can't necessarily tell whether the mutability is in the symbol itself or something beyond it.

For instance, let's consider global variables. The definition of a global variable has an effect which we can inspect if we have a boundp function:

  (boundp 'x) -> nil
  (defvar x)
  (boundp 'x) -> t

That can be made to work by mutating the symbol (the global binding information can be right inside the symbol). Or it could be working by keeping the symbol immutable, but mutating some hash table of bindings.

Either way, the symbol looks interned, because we have mentioned it several times, and those mentions seem to be connected. The (defvar x) has an effect on (boundp 'x) and so they are referring to an x which is somehow the same.

It could work with x actually be a kind of character string, which got separately allocated three times. As long as we can't show any property of the system indicated by x to be different based on which copy of x we are using to enquire (e.g. boundp reports true for one x and false for another), then x looks interned.

In Clojure land, equal but not identical? symbols don't cause any issues; they can be used interchangeably as map keys, etc. It won't impact code correctness, just potentially cause slowdowns.

With that said, I always thought symbols would intern, but that's not the case. It is true with keywords, however.

(identical? (clojure.edn/read-string "x") 'x) => false

(= (clojure.edn/read-string "x") 'x) => true

(identical? (clojure.edn/read-string ":x") :x) => true

Objects that can be equal but not identical are not symbols.

They are, at best, cargo culted symbols: character strings with a tag bit which says "read/print me without quotes, so I visually look like something out of Lisp".

You don't use Clojure, but you're willing to jump in and criticize one part of it that, given the context of the rest of the system, could not be less important?

Whether Clojure's object model and equality semantics as a whole make sense is certainly up for debate. It's highly opinionated and no silver bullet.

But once it's in place, the decision of whether to intern symbols is a trivial implementation detail.

I incorrectly assumed they were interned for seven years of using Clojure professionally, it has never made a difference, and I can't come up with a scenario where it plausibly would.

Common Lisp Elitism is a real thing. This person appears to be from the "Clojure is not Lisp" clan of gatekeepers.

I think the characterisation of Clojure is not that unfair here. It's Clojure keywords have the role that Lisp's symbols have (and I think they have better ergonomics), and symbols are mostly only used for source code representation.

In other Lisps the detailed semantics of symbols are more important including the identity/interning thing.

Rich Hickey was a Common Lisp user before making Clojure so there's a fair chance he knew how symbols worked there, so the cargo culting characterisation should be applied only light heartedly :)

That Clojure is keyword heavy is true, but it's so important to note that objects are essentially never checked for reference equality in Clojure, even when e.g. looking up keys in a hash map (see demo).

With this is mind, you could stop interning keywords and damn near every Clojure program would continue to work just fine - but with a noticeable slowdown.

Or, more sensibly and to bring it back to the theme of the thread, for adding a second non-interning Keyword type which can safely be generated while deserializing user input in a long running process, that you can use interchangeably with standard keywords, but will be garbage collected away with the reset of the deserialized data when you're done.

You do pay a hefty penalty here because you're hiding everything behind interfaces and abstractions. It's totally fine to not like the system, or believe it's not worth the performance hit.

But it does mean that a potentially equal but not identical symbol isn't some off brand low quality replacement as GP suggests, it's just... a symbol.

Pastebin demo: https://pastebin.com/cbWiNyEL

Good points.

It's interesting that despite this keywords are serialized all the time in Clojure land (eg in the transit format that is commonly used for frontend/backend communication).

I think Google's warning definitely applies to Clojure.

Most json libraries will convert string keys to keywords, and they're not weak references.

An attacker can probably just send a few dozen gigabytes of random json to the average Clojure app and it's going to go down.

I wouldn't write a config file parsing library for C programs without interning, so == between two pointers could be used to test for keyword equality.

Interning is used outside of LIsp. See the XInternAtom function in the X Window system:

  Atom XInternAtom(
    Display *display,
    char *atom_name,
    Bool only_if_exists
  );

or RegisterClass in Win32:

  ATOM RegisterClassA(
    const WNDCLASSA *lpWndClass
  );

I wouldn't either :)

For Lisp the following is the usual behavior: Symbols are by default interned and identical symbols are tested with EQ to be T.

  > (eq (read) (read))
  a a 
  T

The default test function is EQL, which is using EQ to test symbols. In Common Lisp #:a would be an uninterned symbol with the name "A".

  > (find 'a '(#:a a))
  A

  > (find 'a '(#:a a) :test #'string-equal)
  #:A

setting the value of a symbol will basically work in all Lisps with symbols in similar fashion like this:

  > (dolist (item '(a b c a))
      (set item (if (and (boundp item)
                         (numberp (eval item)))
                    (1+ (eval item))
                    1)))
  NIL

  > (mapcar 'eval '(a b c a))
  (2 1 1 2)

This last example will for example run unchanged in Emacs Lisp and Common Lisp.

What is the purpose of creating uninterned symbols?

They could be used as symbols which can be GCed.

Though a typical use is in macros, where macros introduce new symbols and these should never clash with any existing symbol and to which there should be no access via the name.

Example: A macro which writes the form, the value and which returns the value. GENSYM generates a named/counted uninterned symbol.

  > (defmacro debugit (form &aux (value-symbol (gensym "value")))
      `(let ((,value-symbol ,form))
         (format t "~%The value of ~a is ~a~%" ',form ,value-symbol)
         ,value-symbol))
  DEBUGIT

If we look at the expanded code of an example, we can see uninterned symbols:

  > (pprint (macroexpand-1 '(debugit (sin pi))))

  (LET ((#:|value1093| (SIN PI)))
    (FORMAT T "~%The value of ~a is ~a~%" '(SIN PI) #:|value1093|)
    #:|value1093|)

We can also let the printer show us the identities of these symbols, labelling objects which are used multiple times in an s-expression:

  > (setf *print-circle* t)
  T

  > (pprint (macroexpand-1 '(debugit (sin pi))))

  (LET ((#2=#:|value1095| #1=(SIN PI)))
    (FORMAT T "~%The value of ~a is ~a~%" '#1# #2#)
     #2#)

Thus we can see above that it's just one uninterned symbol used in three places.

Example run:

  > (debugit (sin pi))

  The value of (SIN PI) is 1.2246063538223773D-16
  1.2246063538223773D-16

That seems like a bad idea, since you've now got two symbols with the same name that'll fail eq? Is this ever actually done?

Interesting that gensym returns uninterned symbols, thanks.

The uninterned symbols don't fail EQ if they are the same identical symbol.

Great, thanks for filling me in. Any idea why the Google guide is against using them for this purpose?

What does the guide say?

Keep in mind that this is a guide from a Lisp using company (bought by Google) who wrote specifically two large applications partly, but significantly, in Lisp: a search engine for flight travel and an airline reservation system. Other application teams may have different rules&requirements, given that they may use Lisp in very different ways.

I'm sorry, my question was based on a four day old memory from the main thread that doesn't reflect what the guide says. It just says don't intern symbols at runtime. Presumably you can pass some kind of a flag to the reader to read lisp data structures without interning the symbols that it reads?

I know the story of ITA well, it and PG's writings are what got me interested in lisp in the first place. Which makes me feel old. But not comp.lang.lisp old, it's all relative!

I thought I was one of these gatekeepers; and that was before I found out that Clojure doesn't actually have symbols, but just a string type with a quote-free read syntax.

Even AutoCAD's AutoLisp (the old one from the 1980's) has interned symbols.

How symbols work goes back all the way to the original MacCarthy work, and all of its actual (not cargo-culted) descendants.

It is not "Common Lisp" elitism.

For what it's worth, I agree with you. Clojure only weakly holds on to its lisp heritage. I'd liken it to the similarity between C# and C++ (without implying any superiority to either side). Superficially quite similar, some parts near identical, others a complete detour.

With that in mind, however, basing your critique of Clojure on the extent to which it carries the lisp tradition is bizarre. Your criticisms are born of an ignorance of the value proposition Clojure provides, which would not be terribly different even if it had eschewed lisp syntax in favor of something else.

If you actually learned Clojure, there is zero chance you'd be complaining about symbol interning. It's just so ridiculous. You'd probably still think the whole thing is a waste of time, and I'm sure you'd have a big long list of actual, meaningful complaints.

I've seen people criticize TXR for its ugly syntax once or twice here on HN (I pay close attention to lisp posts here), and I thought that was dumb at the time. I'm not interested in learning it but I'm glad you're trying something new. It's a shame to see you stoop to the same level of drive by dismissal.

But whatever. Let's flip each other's bozo bits and move on.

I mentioned AutoLISP to make it clear that the comment is completely unrelated to value proposition, but in hindsight, that may be too obscure.

You call Clojure a cargo cult descendant, which implies it's failing to achieve what its ancestors did, by copying features haphazardly without understanding the deeper motivations behind them.

Clojure is a shitty substitute for CL, but that's just not what it is trying to be. It is an interesting and worthy system in its own right.

No it doesn't imply that; you can easily succeed while copying features haphazardly and without understanding the deeper motivations behind them.

Speaking of TXR, and of "holding on weakly", as a result of this discussion, I made a little change.

A remark was made somewhere that interned symbols are held with a non-weak reference. But it occurred to me that this isn't something engraved in stone. A package should be able to hold on to its interned symbols via weak references. This means that if the only reference to a symbol is from within a weak package, that symbol can be removed from the package and relclaimed by the garbage collector.

Since a package uses hash tables, and hash tables support weak keys, it's trivial to put the two together. I added an argument to make-package to specify a weak package.

In the following test, the symbols interned into package foo get reclaimed because it is weak. Those interned into bar don't get reclaimed:

  (defun weak-package-test (name weak)
    (let ((wp (make-package name weak)))
      (let ((*package* wp))
        (let ((obj (read "(a b c d e f)")))
          (mapcar (op finalize @1 prinl) obj)))))

  (weak-package-test "foo" t)
  (sys:gc t)

  (weak-package-test "bar" nil)
  (sys:gc t)

  $ ./txr weak-package.tl 
  foo:a
  foo:b
  foo:c
  foo:d
  foo:e
  foo:f

I'm committing to this as a documented feature.

almost every language which has Lisp in its name is using some form of symbol tables for interning, from McCarthy's Lisp 1 implementation onwards. That's one of the defining features of the Lisp s-expression reader.

If the 'reader' reads an s-expression like (EMACS LISP IS A LISP DIALECT) then both occurrences of LISP are the same identical Lisp object, both are the same symbol.

If your language is doing something different, then it's not using symbols like Lisp-like languages usually do since the dawn of time.

Reply on news.ycombinator.com