When would you ever want bubblesort?
buttondown.email
buttondown.email
(Always check your assumptions. Sorting has an k log k lower bound if all you can do is compare elements. With integers 1..n, one can do much more.)
I showed up at 5pm at his office, as he was bracing himself for the drive from Swarthmore back to Philly, wondering why he ever took this gig. I proposed a linear algorithm: Break 1..n into k bins, and only sort to detect collisions within each bin. Bins would have an average occupancy of 1, so it didn't matter what sort you used.
(I found this idea later in Knuth, and was briefly disillusioned that adults didn't read everything.)
I looked like George Harrison post-Beatles, I probably smelled of pot smoke, and he dismissed my idea. Then my dorm hall phone rang at 10pm, a professor of mine relaying an apology. For that and other ideas, the second edition of his book opened with thanks to me.
I'm surprised now that I realized then that bubble sort actually had the least overhead, in such an application. People don't use Strassen to multiple matrices unless the matrices are really big. For sorting, "really big" is 2 or 3 elements.
That's not quite right, though - you only care about the average occupancy if your sorting algorithm is O(b) for b elements. If you're using a O(b^2) sort, you actually care about the root-mean-square occupancy (or in general just the average runtime, or just the total runtime since you'll multiply the average by the number of bins anyway).
The k-bin approach is definitely much faster than O(k log k), but I'm fairly sure it's slightly slower than truely linear, asymptotically, since the runtime will be something like 1+0+4+1 in typical cases, where linear would have to be 1+0+2+1.
Edit: on some cursory Monte Carlo testing, the sum of squares of occupancies actually seems to still converge on a constant factor (of two) times the sum of plain/linear occupancies, which I was not expecting but does make the above point rather moot. (Also, the sum of cubed occupancies seems to converge on a factor of five, which seems downright bizzare.)
(Tangential edit: The factors seem to be 1 2 5 15 52±2 200±20, which might be https://oeis.org/A000110, though none of the interpretations there seem obviously related.)
Like I said about Strassen and matrix multiplication. Great algorithm, but it takes a while to kick in for the lead.
Pretty much; my mistake was underestimating just how rapidly it vanishes. In actuality any sorting algorithm that isn't pathologically horrible garbage like bogosort will work fine, because it runs on meaningful numbers of elements so rarely that it doesn't matter (and I'm not totally sure bogosort is garbage enough).
> where the constants in front of fancy sorts took them out of the running.
For beating O(k log k), the constants don't actually matter, since they just increase the constant factor inside your O(k) runtime. It's just that apparently, the asymptotics don't matter much either for that.
Can you clarify the problem?
k elements uniformly at random, independently, with replacement, sounds like it’s trivially O(k), or perhaps O(k log n) if n is so big that you need O(k log n) bits to merely represent the answer. (Those logs are pesky and can appear and disappear in different models of computation!)
Maybe you mean k distinct elements? Off the top of my head, if n is not huge (n = O(k), or more concretely, n < ck for your choice or c > 1, and I would probably start with 2 if I were implementing this), I would make an array of all the elements and draw k of them using any of a number of straightforward algorithms. (Fischer-Yates, for example, but I admit I had to look up the name.)
For larger n, make a hash table to store the elements selected so far. Now repeat k times: choose an element at random and retry if it’s a duplicate. Then add it to the hash table and continue. Each iteration takes at most 1/c tries on average, so the overall expected time is O(k). (For n < ck, the last few draws may retry quite a lot.)
Thinking of this as sorting seems odd, but I admit I’m thinking about this in 2023, not the 1970s :)
They were concerned with the case where n could be absurdly large, and wanted no time or space dependence on n. They improved my idea to cut the multiple of k needed for space.
These were rare cases where I didn't have a sort algorithm in a library -- perhaps because I needed to pass in a comparator that wasn't easily supported in the language. So I banged out whatever was easiest to get me moving on to the actual problem. If the sort became a noticeable part of the running time, I'd deal with that when I had performance numbers.
Bubble sort was easy to remember and quick to bang out. For all I know I really was writing an insertion sort; all O(n^2) sorts look pretty much the same.
I haven't written any sorts at all in well over a decade, since all languages have substantial support. If I'm writing a sort, something has gone very wrong.
In my opinion, a bubble sort is a solid choice for that sort of use case.
I would pick a standard library sort function at that point.
The author might have misunderstood the hardware in question and the reasons why bubble sort works well here. You probably do have applicable hardware; it’s any GPU. The bubble sort wasn’t running on ray tracing hardware, it was just CUDA code. The reason bubble sort works well is because it’s running on a SIMD machine, and it’s better when all threads in a work unit (a “warp” in CUDA parlance) all do the same thing at all times.
If you try to do insertion sort on a SIMD machine, you’re very likely to approach worst-case O(N^2) performance very quickly, even when each individual thread involved would have had average-case perf on it’s own. Bubble sort is good for the parallel SIMD machine because of it’s deterministic and regular evaluation order that is the same across all threads.
Of course, it’s still true that for anything larger than very small arrays of data, bubble sort complexity is unacceptable on SIMD or parallel processors. There are better parallel sorting algorithms for that.
You do the compare-and-swap when you're iterating over the objects to do some other render pass, so it adds almost nothing to the CPU time.
If two threads go to CAS-sort at the same time, the cache coherency overhead would be apocalyptically bad. I'm assuming that's not what you meant?
> If two threads go to CAS-sort at the same time, the cache coherency overhead would be apocalyptically bad. I'm assuming that's not what you meant?
I think it's apparent they didn't mean the CAS atomic instruction, just the compare-and-swap operation in bubblesort. Nothing in their comment suggested this was a multi-threaded situation.
OP is describing using CAS to bubblesort an array while it is concurrently read by another thread rendering something based on its contents? That seems kind of wild to me but I don't know why else you add the words "some other". Sometimes sorting is an optimization and isn't necessary for correctness.
The idea of two threads concurrently bubblesorting the same array with CAS is hilarious and intriguing. On x86 the double-wide CAS requires double wide alignment so I guess it couldn't actually work without extra trouble...
for (int i = 0; i < length; i++) {
// do some rendering or something
}
for (int i = 0; i < length; i++) {
if (obj[i] > obj[i+1]) swap(&obj[i], &obj[i+1])
// do some rendering or something
}
No. I'm not OP, but 99% certain this is not the case.
aappleby is saying the following: He wrote a game engine, it renders transparent items using the painter's algorithm, which requires the items to be sorted in Z. He has a routine that iterates over all items to render them, and bubble-sorts the items in-place while iterating over the items (all in the same thread). The "compare and swap" in this case is the innermost functionality of bubblesort.
I assume the author is https://github.com/aappleby but I don't see any obvious candidate for this code. Maybe this? https://github.com/aappleby/metroboy but I can't see why a gate-level emulator woudl need to render transparent items using painter's algorithm.
A single pass of bubble sort per frame is often enough to keep things in order. And if something glitches because it takes 2 passes it's just a single frame glitch!
Most objects aren't moving that much. The lists are thus almost in order. So running a bubble sort on each axis to check the ordering is not much worse than O(N).
Startup has a transient. The very first sort will be O(N^2). If the number of objects is large enough it may be worthwhile to do something different at startup.
> Most objects aren't moving that much.
There are certainly simulations where this approach is fine (if you can be absolutely confident that no object ever "skips" around, like when you're running a simulation that is configured with an explicit upper bound on velocity), but I think that this still requires this warning label.
Insertion sort is great if you don't have any cores or memory to spare. It uses one core and works in-place.
Selection sort is much like insertion sort, runs in-place, single-threaded, but it will be faster if your CPU is much faster than your memory access as it uses more comparisons and fewer swaps.
In practical terms using integers, this makes almost no difference *except* it can be performed using faster shift and add operations:
gap = ((gap << 1) + gap) >> 2;
For even more speed, I'd also recommend in-lining this to eliminate the function call. Properly implemented, respectable speed can be achieved from a really trivial implementation.
When you read Knuth's The Art of Computer Programming he challenges you to come up with your own sorting algorithm. I feel really fortunate that I was able to do this without knowing any algorithms already. I came up with bubblesort. I think it was the intuition of thinking about it becoming "more sorted" on each pass that led me there.
1. If you have a very large (say 100 million elements ) array in descending order, you can use Bubblesort to sort it in the ascending order - the worst case complexity is kn^2, and this is literaly the worst case, so its the equivalent of a sleep(t) routine for a large t ( depending on how sow your system is).
2. If you have a very small array (say n < 10), you can create toy examples where kn^2 < m*nlogn, for carefully chosen values of k, m & n. Say n = 3, k = 1, m = 7.
Array v with N elements:
First pass: go from v[0] to v[N-1] comparing v[i] with v[i+i] Second pass: go from v[N-1] to v[0] comparing v[i] with v[i+i]
This is so obvious that it should have a name...
Do it in job interviews only if you've already decided you're unlikely to want to work there and be sure to be smiling and humble as you demonstrate this to their peers, noting that the pure forms of the algo, selection sort is arguably better (Knuth!) but this is unlikely to be an interesting practical concern and totally dwarfed compared to readability of the code.
Not compared to insertion sort, which is the actual contender here.
Weirdly aggressive comment.
Why? It was the easiest sort I had memorized, I didn't have my college Data Structure 201 book handy. It did exactly what I wanted. And for my datasets, it was not impacting performance. Measurably? Probably. Noticeably from the user level? No. If it was slow, I would have changed it. I did it in BASIC.
But it was never my bottle neck. We're talking 10 or less items here most of the time, my bottleneck was sucking rows from the ISAM database on our 10 user, 16Mhz 68020.
I was doing best fit scheduling across a selection of resources.
It was a very cool system.