TXR Lisp

TXR is not just a pragmatic, low-dependency, small-footprint tool for hackers, but a bit of a Lisp research platform.

For instance, TXR Lisp doesn't have keyword parameters "natively". But via the parameter list macro mechanism[1], a function can take keyword params, if the symbol :key appears as a left item in its parameter list. That triggers the :key parameter macro.

TXR Lisp also has a defset, very similar to CL's defsetf. That provides one of the ways by which we can make a form into an assignable place.

Now, here is the cool part. defset[2] and the :key[3] parameter macro do not know anything about each other. Yet if you write a defset for a form that takes key arguments, it works!

Quick demo:

  1> (defun f (:key x -- key1 key2) (list x key1 key2))
  f
  2> (defset f (:key x -- key1 key2) val ^(f-set ,x ,key1 ,key2 ,val))
  f
  3> (expand '(set (f 1 :key1 2 :key2 3) 42))
  (f-set 1 2 3 42)

By contract, the Common Lisp implementations of defsetf I've looked at deal with keyword parameters explicitly. They analyze the parameter list and know which symbols need gensyms and such.

More complex call:

  4> (expand '(set (f 1 :key1 (a b) :key2 (c d)) 42))
  (let ((#:g0065 (a b))
        (#:g0066 (c d)))
    (f-set 1 #:g0065
     #:g0066 42))

defset ferreted out, through the :key expander, without knowing anything about it, that argument (a b) corresponds to the key1 parameter, and that it needs a gensym, which is then inserted in place of the ,key1 unquote in the ^(f-set ...) backquote template.

[1] https://www.nongnu.org/txr/txr-manpage.html#N-00B4065C

[2] http://www.kylheku.com/cgit/txr/tree/share/txr/stdlib/keypar...

[3] http://www.kylheku.com/cgit/txr/tree/share/txr/stdlib/defset...

kazinator provided examples in two recent k threads:

https://news.ycombinator.com/item?id=22631649

https://news.ycombinator.com/item?id=22666539

I like it but I don't know what to use it for.

My Lisp usage can be summarized as such:

For anything heavy/performant, I use Common Lisp.

For everything else, I use Emacs Lisp.

TXR's FFI can describe a C linked list and convert it in both directions in a call, with correct malloc/free memory management.

Given this code, which is compiled as part of a larger library called "crazyffi.so":

  struct lnode {
    char *datum;
    struct lnode *next;
  };

  void list_update(struct lnode *list)
  {
    struct lnode *iter;
    int i;

    for (i = 0, iter = list; ; iter = iter->next, i++) {
      char buf[256];
      printf("lnode[%d]->datum = %s\n", i, iter->datum);

      /* Edit every node by adding numeric prefix to the string.
       * We free the old datum, and install a newly malloced
       * string in its place.
       */
      snprintf(buf, sizeof buf, "%d:%s", i, iter->datum);
      free(iter->datum);
      iter->datum = strdup(buf);

      /* When visiting the last node, add one more node.
       */
      if (!iter->next) {
        snprintf(buf, sizeof buf, "%d:%s", i + 1, "cow!");
        iter->next = malloc(sizeof *iter->next);
        iter->next->datum = strdup(buf);
        iter->next->next = 0;
        break;
      }
    }
  }

This TXR Lisp code calls the function:

  (typedef lnode (struct lnode
                   (datum str)
                   (next (ptr (struct lnode)))))

  (with-dyn-lib "./crazyffi.so"
    (deffi list-update "list_update" void ((ptr lnode))))

  (let ((ll #S(lnode datum "how"
                     next #S(lnode datum "now"
                                    next #S(lnode datum "brown"
                                                 next nil)))))
    (prinl ll)
    (list-update ll)
    (prinl ll))

Run it:

  $ txr linked-test.tl 
  #S(lnode datum "how" next #S(lnode datum "now" next #S(lnode datum "brown" next nil)))
  lnode[0]->datum = how
  lnode[1]->datum = now
  lnode[2]->datum = brown
  #S(lnode datum "0:how" next #S(lnode datum "1:now" next #S(lnode datum "2:brown" next #S(lnode datum "3:cow!" next nil))))

We see that the list was altered with numeric prefixes on the strings, and a new node was added at the end.

Valgrind is completely clean:

  $ valgrind txr linked-test.tl 
  ==6707== Memcheck, a memory error detector
  ==6707== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
  [ ... ]
  #S(lnode datum "how" next #S(lnode datum "now" next #S(lnode datum "brown" next nil)))
  lnode[0]->datum = how
  lnode[1]->datum = now
  lnode[2]->datum = brown
  #S(lnode datum "0:how" next #S(lnode datum "1:now" next #S(lnode datum "2:brown" next #S(lnode datum "3:cow!" next nil))))
  ==6707== 
  ==6707== HEAP SUMMARY:
  ==6707==     in use at exit: 2,681,402 bytes in 9,346 blocks
  ==6707==   total heap usage: 14,827 allocs, 5,481 frees, 3,034,573 bytes allocated
  ==6707== 
  ==6707== LEAK SUMMARY:
  ==6707==    definitely lost: 0 bytes in 0 blocks
  ==6707==    indirectly lost: 0 bytes in 0 blocks
  ==6707==      possibly lost: 1,286,596 bytes in 1,533 blocks
  ==6707==    still reachable: 1,394,806 bytes in 7,813 blocks
  ==6707==         suppressed: 0 bytes in 0 blocks
  ==6707== Rerun with --leak-check=full to see details of leaked memory
  ==6707== 
  ==6707== For counts of detected and suppressed errors, rerun with: -v
  ==6707== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Now with --free-all:

  0:sun-go:~/txr$ valgrind txr --free-all linked-test.tl 
  ==6721== Memcheck, a memory error detector
  [ ... ]
  #S(lnode datum "how" next #S(lnode datum "now" next #S(lnode datum "brown" next nil)))
  lnode[0]->datum = how
  lnode[1]->datum = now
  lnode[2]->datum = brown
  #S(lnode datum "0:how" next #S(lnode datum "1:now" next #S(lnode datum "2:brown" next #S(lnode datum "3:cow!" next nil))))
  ==6721== 
  ==6721== HEAP SUMMARY:
  ==6721==     in use at exit: 0 bytes in 0 blocks
  ==6721==   total heap usage: 14,830 allocs, 14,830 frees, 3,034,665 bytes allocated
  ==6721== 
  ==6721== All heap blocks were freed -- no leaks are possible
  ==6721== 
  ==6721== For counts of detected and suppressed errors, rerun with: -v
  ==6721== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

What's missing is circularity detection; we cannot pass a graph back and forth in this manner. If we pass a DAG structure, its shared structure will get expanded.

Since circularity detection is expensive, that would have to be something enabled by some sort of attribute mechanism.

Interesting