The Search for the “perfect” Advent Calendar (involves Python and Processing) | Hacker News Reader

The Search for the “perfect” Advent Calendar (involves Python and Processing)

blog.jgc.org

40 pointsjgrahamc31 comments

This is more or less a discrete version of a 2D low discrepancy sequence, which is often used for choosing different colours.

The best known solution is plastic numbers. I expect you could use something like that here rather than brute force.

http://extremelearning.com.au/unreasonable-effectiveness-of-...

Although using a low discrepancy sequence (eg Halton, Sobol, R2) might be a good starting point, these sequences are designed to maximize the distance between points assuming that the left-and-right edges wrap, as well as the top-and-bottom. As it seems that the OP does not require this condition, I wold presume you get better solutions which involve lots of going to and from the far-left to the far-right, etc..

(Ordered dither matrices might offer another good starting point.)

Yeah that's what my "more or less" was referring to...

This is a fun problem to play with. Like others, I've been trying the general approach of generating random configurations, then iterating by swapping pairs of elements (each time choosing the pair that yields the best local improvement).

Lowest value found, has vertical symmetry (notice how (1,24), (2,23), (3,22), (4,21) and so on are paired)

  [[14  7 19 12  5 17]
   [21  9  2 24 22 10]
   [ 4 16 23  1  3 15]
   [11 18  6 13 20  8]]

  375.923809971073

Highest value found, has rotational symmetry (see how the path from 1-24 is almost like a squished Hilbert curve)

  [[ 6  7 10 11 23 24]
   [ 5  8  9 12 22 21]
   [ 4  3 13 16 17 20]
   [ 1  2 14 15 18 19]]
  
  529.6200384399018

Code is at https://gist.github.com/jffry/fab43b5b65499c3f513fea70159780.... Probably full of new-to-Rust mistakes, so please let me know where I've strayed from the light if you're particular about Rust!

As mentioned elsewhere I'm very curious about how this generalizes to grid sizes other than 6x4

Noticed that my best was almost horizontally symmetrical apart from two slots - except swapping those slots changes the score from 375.99867 to 376.19293. Most perplexing.

This is good, but it bugs me that the "perfectly ordered" calendar doesn't have the worst score. Maybe consider the calendars tiled on an infinite grid? That way 5 will be one right and one down from 4, instead of four left and one down. Maybe even shift the next line up, so 4 is next to 5...

I'll toss my hat in the ring for maximizing the measure instead of minimizing it

  "Cortex"

  [[ 6  7 10 11 23 24]
   [ 5  8  9 12 22 21]
   [ 4  3 13 16 17 20]
   [ 1  2 14 15 18 19]]
  
  529.6200384399018

The path traced from 1-24 is rotationally symmetric and folds up in on itself sort of like a Hilbert curve.

And now I'm curious how this problem might look for other MxN aside from 6x4

I think you nailed it, it seems to me that a Hilbert curve will always have the maximal score.

Just a quick note to say thanks for posting this: I've had great fun exploring it: it's a simple puzzle to understand, yet harder than it first appears, and a low barrier of entry thanks to your posted code. Genuinely interesting seeing the symmetry appearing in the low-scoring solutions, and also an element of friendly competition and shared discovery as we post our scores. Cheers!

The perfect advent calendar already exists

https://adventofcode.com/

I was hoping for a bit shuffling approach that elegantly fills the rectangle in some quasi-random order.

EDIT: Played around a bit myself, here's a quite pleasing order I came up with:

  1   9  17  24  16   8 
  5  13  21  20  12   4 
  3  11  19  22  14   6 
  7  15  23  18  10   2

Python snippet here: https://pastebin.com/XNvjEmxc

The funny thing about this one is while it mostly satisfies the constraint of not having consecutive numbers too close, it has a very obvious pattern to it. A nice example of overfitting, perhaps ;)

What additional constraint could you add to shake this up? Perhaps a rule that, as well as number adjacency being penalized, adjacent distances between consecutive numbers also get penalized?

It doesn't score very well on my metric. Not badly but not well.

    [[ 1  9 17 24 16  8]
     [ 5 13 21 20 12  4]
     [ 3 11 19 22 14  6]
     [ 7 15 23 18 10  2]]

    400.634917287

Compare with this one found using the 'genetic algorithm':

    [[ 9  4 18 10 15 20]
     [16 21 12  1 22  7]
     [ 2  6 24  5 17 13]
     [14  8 19 11 23  3]]

    386.229085028

Best I've found (with a random shuffle of the numbers every 5M swaps) is (score verified with `advent.py`):

         [[11 16  8 13  2 20]
          [18 21  5 24  7 15]
          [ 3  9  1 19 22  4]
          [14 23  6 17 10 12]]

         382.46224448143795

That's a lot of swaps! I played around with a simulated annealing approach in my lunch break which consistently gets down to 383.xx with about 10000 swaps, or 379.xx in 100000 swaps. I've yet to tweak the annealing parameters to see if I can get down to (or beat) the 378 example in the article.

It's basically the same as the swap() function in advent.py except you sometimes allow swaps that result in worse scores according to an ever-decreasing probability ("temperature"). The added controlled randomness allows you to break out of local minima at the start and hone in on the local optimum towards the end.

EDIT: after a little tweaking of the temperature schedule I got 377.59 after 10K swaps.

The matrix

   8 20 15 10  5 17 
  13  4 22 12 23  1 
  18  2 24 21  3  7 
  11  6 16  9 14 19

gets a score 376.9629. This came from starting with a random matrix, trying all possible swaps and taking the best one, and iterating until a local minimum is reached.

This is a purely random approach. Currently on 382.08832 after 1.4B swaps. Might have a look at tweaking things later when I'm not at work...

Interesting approach. Can you share your code?

Sure. My best so far is:

    np.matrix('12 19 6 14 4 9; 8 17 1 24 22 16; 21 3 23 2 20 11; 15 10 5 13 18 7')

which has a score of 376.899. This was produced in 20K swaps with the line:

    best, score = anneal(n=10000)

which was then further refined with:

    best, score = anneal(start_temp=0.2, advent=best, n=10000)

where anneal() is defined here: https://pastebin.com/xBVGJfQd

The score() function is a bit slow at the moment. If I get some time later, I might see if I can speed it up a bit, which will allow for some faster experimentation.

EDIT: replaced code with pastebin link to save space in comments

Thanks. That's great.

Slight improvment to the above:

  [[12, 19,  6, 14,  4,  9],
   [ 8, 17,  1, 24, 22, 16],
   [21,  3, 23, 20,  2, 11],
   [15, 10,  5, 13,  7, 18]]

Score = 376.6144674353488

Found by increasing number of iterations to 100000

Sim.annealing followed by an exhaustive pair-swap search to find the local minimum would have found this more efficiently. Possibly there are some further refinements left -- I haven't run the exhaustive pair search on the above!

My best so far, but not using the improvements above, just small tweaks to the code from the blog post:

  [[ 9 15  4 20  6 12]
   [18 23 11  1 22 17]
   [ 7  2 16 24  3  8]
   [13 21  5 10 19 14]]
  376.364049355

My Go code, slightly tweaked from the pure random approach, got this one:

    [[17 11  6 19 14  8]
     [ 4 22 24  1  3 21]
     [ 9 15  2 23 12 16]
     [13 20  7 18  5 10]]
    375.998672775885

Ooh nice! That's the first one I've seen below 376.0

I've gotten close, but not cracked it yet. I was wondering if anyone would break the 376.0 barrier!

375.923809971073 :-)

https://gist.github.com/jffry/fab43b5b65499c3f513fea70159780...

Added to https://rjp.is/calendars/topthree.png (which is now four but nvm) to show the exceptional symmetry. I'm thinking we might be right on the lower limit here - doesn't seem much room for more symmetricalising.

I'm wondering if vertical symmetry might be involved (and a way of optimising future efforts) - plotting the journeys of the top three on this post definitely seems to imply that.

https://rjp.is/calendars/topthree.png

Compare and contrast the original set from @jgc's article:

https://rjp.is/calendars/originals.png

Thanks for keeping track. Quite fascinating how algorithms employing so much randomization can arrive at such structure and symmetry.

Finally got a < 376.0 from my own code using simulated annealing method...

375.998672775885

  [[13 20  7 18  5 10]
   [ 9 15  2 23 12 16]
   [ 4 22 24  1  3 21]
   [17 11  6 19 14  8]]

...only to realise it's the same as yours, but with the rows reversed! Rather surprised, but I now wonder if there is only a small number of very-low-scoring solutions. So perhaps this is less of a coincidence than it first appears.

I'm now using a much more aggressive temperature drop-off to find decent candidates early, followed by a tempering phase to search for nearby solutions, and a final cool-off to refine the final answer. I'm still using only random pair swaps in Python, so probably wasting a lot of cycles, but I'm still quite surprised how quickly it converges to some pretty decent scores. Beyond that I'm just going to try lots of random starting layouts.

I'm interested to see if my method can find any of the other posted solutions or (fingers crossed!) any new ones, but I may need to crunch through a lot more candidates... I will have to translate from Python into something faster to up my game!

Not entirely scientific but starting from the trivial 1-24 board, I did 100M random swaps (in Go, keeping all of them, takes about 4 minutes) and counted how many scores were in each bucket of 10 (ie 370-379.99, 380-389.99, etc.) Having done this a few times and never got anything in 37, I'm thinking the number of solutions below 380 must be relatively small, yeah.

    37	0
    38	2377
    39	1103812
    40	16535778
    41	39376324
    42	29525491
    43	10609914
    44	2394340
    45	395239
    46	51463
    47	4880
    48	363
    49	19
    50	0

The symmetry thing makes sense, but now I wonder if there's are really "interesting" or not. Does the symmetry spoil the fun of searching or not?

Why not leave out "max -" and maximize instead of minimize?

Reply on news.ycombinator.com