Polynomial interpolation
cohost.org
cohost.org
This video covers it well: https://youtu.be/h7apO7q16V0
Also, chapter 30 of CLRS: https://elec3004.uqcloud.net/ebooks/CLRS%20(2nd%20Ed)%20-%20...
sidenote: polynomial multiplication != interpolation
I've benchmarked FFT/non-FFT based approaches for almost everything in that scale for the last ten years and gotten quite different results.
Edit, see for example: https://www.zprize.io/prizes/accelerating-ntt-operations-on-...
Calculate the left Product(0..i-1) of each term in left-to-right order, and the Product(i+1..N-1) of each term in right-to-left order. Your mileage may vary on a GPU.
Build a table left[j] of product-of-ratios for i < j, and another right[k] for i > k, each in linear-time, and combine the two to find the product-of-ratio for i != m in linear time as well.
What do you mean by left product and right product?
https://en.wikipedia.org/wiki/Introduction_to_Algorithms
And CLR (in a sibling comment) refers to the 1990 first edition.
https://github.com/mentat-collective/emmy/blob/main/src/emmy...
I discovered while writing this that I could express each of these algorithms as folds that consumed a stream of points, accumulating a progressively higher order polynomial.
Here's the same sort of thing but for rational function interpolation: https://github.com/mentat-collective/emmy/blob/main/src/emmy...
These days that's a luxury. Open source projects tend to take the "and then a miracle occurs" approach to comments.
I'm old school, my comments look like yours; lots of detail, background and explanations, so I can easily get back into the code five or ten years later.
I really leaned in to these essay-style namespaces for this project; I spent _so_ much time deciphering old numerical algorithms ported from uncommented FORTRAN, and decided that I had to go the other direction.
Someday I'll get around to publishing a static site that renders the math in the comments, like these:
- https://samritchie.io/functional-numerical-methods (numerical integration written in stateless functional style) - https://samritchie.io/dual-numbers-and-automatic-differentia... (automatic differentiation that works on higher order functions etc)
Send me some of your (public) code to read!
That's a bit difficult, as most of my work is proprietary, under NDA or aerospace.
Here's an excerpt from a C control panel menu processor, just to show how far I go. One line of active code vs. 21 lines of comments. This was written 16 years ago. I can get back into this codebase almost instantly because of the documentation. More importantly, code reuse is also very easy for the same reason.
case BUTTON_ENTER:
// ENTER button clicked
//
// This has the effect of calling the "DURING" handler for the currently selected
// menu.
//
// ***NOTE***
// It is the responsibility of the ON_ENTRY handler to decide whether or not
// to capture the message loop by asserting "state_handler_wants_priority"
//
// Once done with the message loop "state_handler_wants_priority" should be cleared,
// typically in the ON_EXIT routine
//
// It is up to the handler to do something (or not) with messages.
//
// Parent menus do not generally do much because they are navigation
// stopping points and their messages are fully handled by the menu_processor() parser.
//
// Child menus that are not parents and, therefore, perform a function,
// might want to do some work before the menu_processor() parser by capturing messages
// before they are delivered to the menu_processor() parser.
//
dispatch_handler(current_state, DURING);
break;
I also like to do this sort of thing, in this case, Verilog code from a custom FPGA-based SDRAM controller: // COMMAND TABLE ----------------------------------------------------------------------------------
// Use the following template for assignment of SDRAM commands from this table:
//
// {SDRAM_CSn, SDRAM_RASn, SDRAM_CASn, SDRAM_WRn} <= COMMAND_NOP; // Issue command
//
// CSn RASn CASn WRn
// | | | |
parameter COMMAND_NOP = {1'b0, 1'b1, 1'b1, 1'b1};
parameter COMMAND_ACT = {1'b0, 1'b0, 1'b1, 1'b1};
parameter COMMAND_RD = {1'b0, 1'b1, 1'b0, 1'b1};
parameter COMMAND_WR = {1'b0, 1'b1, 1'b0, 1'b0};
parameter COMMAND_BST = {1'b0, 1'b1, 1'b1, 1'b0};
parameter COMMAND_PRE = {1'b0, 1'b0, 1'b1, 1'b0};
parameter COMMAND_REF = {1'b0, 1'b0, 1'b0, 1'b1};
parameter COMMAND_LMR = {1'b0, 1'b0, 1'b0, 1'b0};
parameter COMMAND_IOEN = {1'b0, 1'b0, 1'b0, 1'b0};
parameter COMMAND_IODIS = {1'b0, 1'b0, 1'b0, 1'b0};
parameter MODE_REGISTER_DEFAULT = 14'b000000_0_011_0_111; // CAS length = 3
// | | | |
// | | | Burst length: 000 = 1; 001 = 2; 010 = 4; 011 = 8; 111 = Full page
// | | |
// | | Burst type: 0 = Sequential; 1 = Interleaved
// | |
// | CAS Latency: 010 = 2; 011 = 3;
// |
// Write mode: 000000 = Burst read and burst write.
// 000010 = Burst read and single write.
Yes, it's a lot of work, and it is, without a doubt, worth the effort.Also the book Approximation Theory and Approximation Practice
https://epubs.siam.org/doi/book/10.1137/1.9781611975949
is really great, by the author of Chebfun.
There's a neater general formula for generating Lagrange interpolator polynomials of any order that avoids having to perform matrix inversion.
Interpolated values can be calculated in O(N) time, so Lagrange interpolation is always faster than FFT interpolation. Calculate the left Product(0..i-1) of each term in left-to-right order. If you carry the value from term to term, you only need one multiplication per term. and the Product(i+1..N-1) of each term in right-to-left order.
Lagrange interpolators with degrees that are close to PiN tend to be smoother, with less aliasing, when processing audio signals. (N=3, 6, 9, 13, &c). Truncating the Langrange polynomial is akin to windowing in FFT. Having an order that's roughly PiN ensures that the analogous window of the Langrange polynomial has smaller tail values.
Source: https://people.maths.ox.ac.uk/trefethen/barycentric.pdf
This paper is (admittedly) a significant improvement to the method I've been using to date. Probably the most significant improvement is a method for calculating Lagrange polynomials that provide stability guarantees. After having read it, I'm going to point you to that paper, instead of describing how I did it.
The application that I'm familiar with, is interpolation of audio signals, and fractional delay lines. In these cases, x[N] can be considered to have the fixed values [0,1,2,3,...], and then subsequent interpolations can be calculated using an offset into the buffer of audio samples. Initialization takes O(N^2). Evaluation takes O(N).
https://groups.google.com/g/comp.lang.c/c/s7yeYGuMs8s/m/juwf...
Google Groups no longer allows "View Original". Let's reformat the code. Actually, nope; I found the file from 2014 easily.
Seventh degree polynomial for Fibonacci, with Easter egg:
#include <math.h>
#include <stdio.h>
int fib(int n)
{
double out = 0.002579370;
out *= n; out -= 0.065277776;
out *= n; out += 0.659722220;
out *= n; out -= 3.381944442;
out *= n; out += 9.230555548;
out *= n; out -= 12.552777774;
out *= n; out += 7.107142857;
out *= n;
return floor(out + 0.5);
}
int main(void)
{
int i;
for (i = 0; i < 9; i++) {
printf("fib(%d) = %d\n", i, fib(i));
switch (i) {
case 7:
puts("^ So, is this calculation fib or not?");
break;
case 8:
puts("^ Answer to life, universe & everything responds negatively!");
}
}
return 0;
}
Execution: $ ./fib2
fib(0) = 0
fib(1) = 1
fib(2) = 1
fib(3) = 2
fib(4) = 3
fib(5) = 5
fib(6) = 8
fib(7) = 13
^ So, is this calculation fib or not?
fib(8) = 42
^ Answer to life, universe & everything responds negatively ax1^3 + bx1^2 + cx1 + d = y1
ax2^3 + bx2^2 + cx2 + d = y2
ax3^3 + bx3^2 + cx3 + d = y3
ax4^3 + bx4^2 + cx4 + d = y4
As a matrix: [x1^3 + x1^2 + x1 + 1] [a] = [y1]
[x2^3 + x2^2 + x2 + 1] [b] = [y2]
[x3^3 + x3^2 + x3 + 1] [c] = [y3]
[x4^3 + x4^2 + x4 + 1] [d] = [y4]
If the matrix is M, we just want to find M^(-1)y.I think hackers should know enough linear algebra to do that, without "having to go asking the maths gods".
[x1^3, x1^2, x1, 1] [a] = [y1]
[x2^3, x2^2, x2, 1] [b] = [y2]
[x3^3, x3^2, x3, 1] [c] = [y3]
[x4^3, x4^2, x4, 1] [d] = [y4]Also, OP was solving a degree four problem and wanted a closed form.
For better stability just use `x = np.linalg.solve(M, y)`, or whichever linear solver you expect to be fast.
I came to the same conclusion as the article: The answer is to solve a system of linear equations with 4 unknowns.
Even if your data isn’t noisy and the point added lies exactly on the previous polynomial (making the problem degenerate) chances are float rounding and numerical stability of your code can have that effect (https://arachnoid.com/polysolve/#x_limits)
In the gaming context this is not a relevant question because, as you rightly say, the problem is "just to fit" what we have.
But in scenarios where the polynomial is an intensional representation for a function specified extensionally, i.e. via as a set of finite data points, and where we gradually become aware of additional data points, we can use the polynomial to forecast further data points - the interpolated ones.
Recent example paper: https://arxiv.org/pdf/1808.03216.pdf