Productivity is the ratio of output to input.
Code is the output of a programmer. Time (or wages) is the primary economic input to software development.
Productivity is not a measure of whether something is worth doing. It says nothing about whether the code is valuable or worth writing. A factory has increased productivity if it has produced more widgets with the same inputs, even if nobody wants those widgets or those widgets pollute the environment.
Similarly, programmers do not choose what software to write. They are employed to write software someone else decides is valuable to the business. Their productivity is independent of whether the software has business value.
Consider a one line program. A programmer that produces that program in 5 minutes is more productive than one who produces it in 1 hour.
Stories with sizes are proxies for magnitude of code change or time. A larger project is larger because it takes more time or more code change, or both. Story size/points is an attempt to estimate this. So it is more accurate to measure the actual code change and time taken.
It’s true that the same program can be written with different amounts of code. But this is largely irrelevant at the organization level, where programmers are not writing multiple versions of same program in different languages and toolsets. A team is typically writing software within the context of a particular toolchain and runtime environment.