You keep saying this and it's still wrong. Rust is quite capable of expression templates, as its iterator adapters prove. What it isn't capable of (yet) is specialization, which is an orthogonal feature.
Assuming I'm interpreting what you're saying here correctly, this seems wrong? For example, this compiles [0]:
const fn foo(n: usize) -> usize {
n + 1
}
fn bar<const N: usize>() -> usize {
N + 1
}
pub fn baz() -> usize {
bar::<{foo(0)}>()
}
AFAIU iterator adapters are not quite what expression templates are because they rely on the compiler optimizations rather than the built-in feature of the language, which enable you to do this without relying on the compiler pipeline.
vector l;
product& r;
auto operator[](size_t i) {
return l[i] + r[i];
}
vector l;
vector r;
auto operator[](size_t i) {
return l[i] * r[i];
}
Right, I understand that. What is not exactly clear to me is how you get from the tree of deferred expressions to the "flat" optimized expression without involving the optimizer.
Take something like the above example for instance - w = x + y * z for vectors w/x/y/z. How do you get from that to effectively
for (size_t i = 0; i < w.size(); ++i) {
w[i] = x[i] + y[i] * z[i];
}
OK, so how would you write an expression template for the given computation, then?
> Expression templates do not rely on O1, O2 or O3 levels being set - they work the same way in O0 too and that may be the hint you were looking for.
This claim confuses me given how expression templates seem to work in practice?
For example, consider Todd Veldhuizen's 1994 paper introducing expression templates [0]. If you take the examples linked at the top of the page and plug them into Godbolt (with slight modifications to isolate the actual work of interest) you can see that with -O0 you get calls to overloaded operators instead of the nice flattened/unrolled/optimized operations you get with -O1.
You see something similar with Eigen [2] - you get function calls to "raw" expression template internals with -O0, and you need to enable the optimizer to get unrolled/flattened/etc. operations.
Similar thing yet again with Blaze [3].
At least to me, it looks like expression templates produce quite different outputs when the optimizer is enabled vs. disabled, and the -O0 outputs very much don't resemble the manually-unrolled/flattened-like output one might expect (and arguably gets with optimizations enabled). Did all of these get expression templates wrong as well?
[0]: https://web.archive.org/web/20050210090012/http://osl.iu.edu...
[1]: https://cpp.godbolt.org/z/Pdcqdrobo
I think my question is pretty simple: "How does an optimizer-independent expression template implementation work?" Evidently the resources I've found so far describe "optimizer-dependent expression templates", and apparently none of the "expression template" implementations I've had reason to look at disabused me of that notion.
> My comments here is not work, and I am not here to win arguments, but most of the time learn from other people's experiences, and sometimes dispute conclusions based on those experiences too.
Sure, and I like to learn as well from the more knowledgeable/experienced folk here, but as much as I want to do so here I'm finding it difficult since there's precious little for me to go off of beyond basically just being told I'm wrong.
> If you don't believe me, or you believe expression templates work differently, then so be it.
I want to understand how you understand expression templates, but between the above and not being able to find useful examples of your description of expression templates I'm at a bit of a loss.
Of course in many cases the optimization level does matter: if you are optimizing small vector operators to simd inlining will still be important.
How does that work on an implementation level? First thing that comes to mind is specialization, but I wouldn't be surprised if it were something else.
> What does depend on the compiler is whether the incidental trivial function calls to operators gets optimized away or not.
> Of course in many cases the optimization level does matter: if you are optimizing small vector operators to simd inlining will still be important.
Perhaps this is the source of my confusion; my uses of expression templates so far have generally been "simpler" ones which rely on the optimizer to unravel things. I haven't been exposed much to the kind of matrix/BLAS-related scenarios you describe.
struct F { double x; };
enum Op { Add, Mul };
auto eval(F x) { return x.x; }
template<class L, class R, Op op> struct Expr;
template<class L, class R> struct Expr<L,R,Add>{ L l; R r;
friend auto eval(Expr self) { return eval(self.l) + eval(self.r); } };
template<class L, class R> struct Expr<L,R,Mul>{ L l; R r;
friend auto eval(Expr self) { return eval(self.l) * eval(self.r); } };
template<class L, class R, class R2> struct Expr<Expr<L, R, Mul>, R2, Add>{ Expr<L,R, Mul> l; R2 r;
friend auto eval(Expr self) { return fma(eval(self.l.l), eval(self.l.r), eval(self.r));}};
template<class L, class R>
auto operator +(L l, R r) { return Expr<L, R, Add>{l, r}; }
template<class L, class R>
auto operator *(L l, R r) { return Expr<L, R, Mul>{l, r}; }
double optimized(F x, F y, F z) { return eval(x * y + z); }
double non_optimized(F x, F y, F z) { return eval(x + y * z); }
The magic happens by specifically matching the pattern Expr<Expr<L, R, Mul>, R2, Add>; try to add a rule to optimize x+y*z as well.
I think at this point I can see how my initial assertion was wrong - specialization isn't fully orthogonal to expression templates, as the former is needed for some of the latter's use cases.
Does make me wonder how far one could get with rustc's internal specialization attributes...