Hacker News Reader

Comment by dragonwriter | Hacker News Reader

Python can almost do this with Variadic Generics from Python 3.11; its missing an exception to the single-unpacking rule (which exists to prevent ambiguity) to allow unambiguous cases. Then you would have:

  from typing import TypeVarTuple

  Ts = TypeVarTuple("Ts”)
  Us = TypeVarTuple("Us”)

  def tconcat(
    t1: tuple[*Ts], 
    t2: tuple[*Us]
  ) -> tuple[*Ts, *Us]: ...

View on HN

No this is actually wrong. It works but it doesn't capture the meaning of the nature of a tuple.

A Tuple is Typed as something as a Fixed size. That's right, it's like this at the Type level. The entire concept of a tuple is a Product type or essentially like a struct but with no names for each parameter.

   Tuple[int, str, float] #correct

Doing what you're doing here is equivalent to creating a Struct with variadic properties.

If you want some container that holds an arbitrary amount of things that is a List

The correct type for what you want is actually this:

   List[Any]

It absolutely isn't wrong, and it's strange that you have some aversion against it. Think of Tuple[int, ...] as a union of Tuples of all lengths containing only ints. It's a specific language feature, and it's not "wrong" in any way.

And I have no idea why you claim that a list is "correct".

You're thinking fixed size arrays. It doesn't make sense to have tuples of arbitrary size. To make it arbitrary size you have to constrain it on a single generic parameter which makes it functionally equivalent to a fixed size array. You essentially just do this with an array or a list.

Read my other reply to you to see the problem of taking functions with "generic" number of parameters. What are you going to do with each parameter? What are the types of each parameter? You can't do anything other then breaking out of the type system. Basically you're filling those tuples with a bunch of "Any" types. So what's the point of the type system now? There is No point. Just like List[Any]

To get your idea to work you need dependent types.

> To make it arbitrary size you have to constrain it on a single generic parameter

Your information is out of date since Python 3.11:

https://peps.python.org/pep-0646

Bro, Look at this:

https://peps.python.org/pep-0646/#multiple-type-variable-tup...

Does this not exactly mean: "To make it arbitrary size you have to constrain it on a single generic parameter"

> Bro, Look at this:

I’m not your bro and I've been discussing what that PEP does and does not allow in every post in this subthread.

> Does this not exactly mean: "To make it arbitrary size you have to constrain it on a single generic parameter"

No, it doesn't:

  Ts = TypeVarTuple(*Ts)

  def f(cond: Callable[[],bool], t: tuple[*Ts]) -> None | tuple[*Ts]: ...

The parameter t is an arbitrary-sized (and shaped) tuple, and the return value of f has the same shape as the parameter it is called with.

No multiple unpacking means you can't type things like concatenations of arbitrary-shaped tuples without losing type specificity (and resorting to just tuple), but it doesn't mean you can't type functions using arbitrary sized and shaped tuples outside of that restriction.

>I’m not your bro and I've been discussing what that PEP does and does not allow in every post in this subthread.

It's called an expression. It's a common one in the US and in countries that have English as a first language.

>No, it doesn't:

Lol yes it does. I said what I said and meant what I meant. I'm in control of that you aren't. What's going on here is that you either misinterpreted it or you're just wrong.

>No multiple unpacking means you can't type things like concatenations of arbitrary-shaped tuples without losing type specificity (and resorting to just tuple), but it doesn't mean you can't type functions using arbitrary sized and shaped tuples outside of that restriction.

You literally and I mean literally re-explained what I meant. Then Proceeded to imply I meant something else. Read it: to make a parameter of arbitrary type you have to constrain it on a single parameter.

Word for word. Don't twist it.

> You're thinking fixed size arrays. It doesn't make sense to have tuples of arbitrary size.

My god. No, Python doesn't support "fixed size arrays", but it does support tuples of arbitrary size, no matter how often you claim that to be wrong.

> Read my other reply to you to see the problem of taking functions with "generic" number of parameters. What are you going to do with each parameter? What are the types of each parameter? You can't do anything other then breaking out of the type system. Basically you're filling those tuples with a bunch of "Any" types. So what's the point of the type system now? There is No point. Just like List[Any]

I could, for example, do the thing I've been writing dozens of replies asking for: concatenate two arbitrary tuples. It's a perfectly well-defined operation, and without an artificial limitation on unpacking of TypeVarTuples, it would already be possible.

It is already possible to do what you claim impossible with one arbitrary tuple. How can that be? You keep going on and on about how that doesn't work. Why does it?

>My god. No, Python doesn't support "fixed size arrays", but it does support tuples of arbitrary size, no matter how often you claim that to be wrong.

My god I never claimed this. I'm just talking about programming in general. I'm trying to show you that what YOU are asking for is programming operations to happen at the type level. You're not seeing it correctly.

>I could, for example, do the thing I've been writing dozens of replies asking for: concatenate two arbitrary tuples. It's a perfectly well-defined operation, and without an artificial limitation on unpacking of TypeVarTuples, it would already be possible.

You're not getting it. Ok let me spin it to you another way.

Imagine you Do have two tuples that you concat:

    function concat(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 + arg2)]:

Right? You notice the plus operator at the end there where I concatenate all the generic arguments and unroll it right? You're saying TS supports this and that it's common languages to support that "common" operation.

Well here's another common operation. What if I want the INTERSECTION of the two tuples?

   function intersect(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 | arg2)]:

You see what's going on here? You're asking for programming operators like concat and intersect and union INSIDE of types. You are asking for features that DON'T EXIST in practical type based languages. These features exist in academic languages. And those academic languages are completely off topic.

That's what I'm getting at. Read it and understand it.

You can do this and define a function that does this in python and likely typescript. But you would be EXITING the type system when you do this. That means using keywords like Any and potentially running into runtime errors.

I want to bold that part above. I'm pretty sure whatever you're doing either uses a bunch of Any's which DOES not preserve the shape fully. This is likely what's going on. It may pass the type check but it's not doing what you mentioned in your initial post which is Preserving the shape of the type.

I haven't used typescript. But IF what you say about typescript is TRUE and that it type checks the concatenation of shapes then it's likely an arbitrary one off feature. Have a look at those functions again:

       function concat(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 + arg2)]:
       function intersect(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 | arg2)]:

Basically you're saying that you want some language that supports type level programming where you can concat types, intersect types, divide types, likely bring programming terms up into the type level and have propositional terms like

      func add(x: int < M, y: int < N) -> int < M + N - 1

All of this is unlikely to be supported by any practical language. Likely TS has a one off where if you do this:

      func concat(x: Tuple[*args], y: Tuple[*args]) -> Tuple[M]:
           return x.concat(y)

Where x.concat(y) is just special in that it is the only method that instantiates a typed tuple that is the concatenation of both tuples. Again I repeat it is unlikely for typescript to support what you are asking for in a very generic way where you can program your types.

The operator exists. I can use it today, without exiting the type system, both in Python (limited to only unpacking one tuple type) and Typescript (not limited). Why do you keep claiming this to be impossible, when it already exists right now?

It's also in no way a special case in TS, the type system supports many things you need dependent types for. Maybe read up on it before you claim it can't do these things?

Great. Define the concat and intersection function. Do it in python and ts. That is your claim prove it.

>Maybe read up on it before you claim it can't do these things?

Maybe don't be rude. If I'm wrong I'm wrong. But if you say something like this you need to back up your words. You "read" it. So prove it.

I defined the concat function in TS and Python below: https://news.ycombinator.com/item?id=38391702

Now, obviously Python doesn't support it due to a restriction in the TypeVarTuple PEP, which is the literal entire point of everything I have written. But what more are you asking for? What more do you want as an example?

I am not going to touch the intersection function, as I haven't made any claims in that direction.

> It works but it doesn't capture the meaning of the nature of a tuple.

That's backwards. It doesn't work (as noted in GP, because Python supports unpacking only a single variadic parameter in a type annotation), but it does capture the nature of a tuple.

It is (or, more precisely, would be, if the syntactic limitation was relaxed to allow it) a generic function that operates on two tuples of arbitrary tuple types and returns a tuple of a third tuple type whose shape is a simple concatenation of the shapes of the input tuples.

> Doing what you're doing here is equivalent to creating a Struct with variadic properties.

Well, a "struct" in Python is a particular low-level byte-mapped datatype, but the generic concept of a Struct differs from a tuple in two ways -- fields identified by name and that Structs are generally mutable.

> If you want some container that holds an arbitrary amount of things that is a List

But... I don't want that, and that's not what this works on.

There's a difference between a function that works on containers of changeable length and a generic function that works on immutable containers (so, fixed length and contents), but is adaptable to any length (or more specifically, shape, including not only length by order of types of elements) of arguments, producing a result of a container type whose shape is strictly determined by the shapes of the inputs, with the shapes statically verifiable.

Python is very close to allowing that, which is very different than:

  def concat(l1: list, l2: list) -> list: ...

which is a fine function, but not at all what the other one is about.

> a generic function that operates on two tuples of arbitrary tuple types and returns a tuple of a third tuple type whose shape is a simple concatenation of the shapes of the input tuples.

This isn't definable without dependent types or a concat operator at the type level. Dependent types aren't a thing we are talking about here and languages that support this feature are waaay out of scope.

You're basically asking for stuff like this:

   func concat(a: Array[int; M], b Array[int; N]) -> Array[int; N + M]

   func concat(a: Tuple[*args1], b Tuple[*args2]) -> Array[*(args + args)]

Which is bringing those programming terms up a level into types. This doesn't exist in the realm of programming we're talking about. Idris, Lean and Coq are more you're game here and those languages are clearly off topic.

>But... I don't want that, and that's not what this works on.

You want something well out of scope of this conversation and well out of scope of most practical programming languages. You still don't get what a tuple is. Nobody defines a Product type with an arbitrary amount of properties anymore then someone defines a struct with an arbitrary amount of properties anymore then they would do it for a tuple.

This is NOT the definition of a tuple. People are confusing the Tuple for some Immutable list.

>Python is very close to allowing that, which is very different than:

Python shouldn't allow this, neither should TS. If typescript allows it, it's probably some arbitrary shortcut feature. Dependent types at the type level supported in a very generic way is very very unlikely.

Like you people are trying to explain to me what a generic is while doing all kinds of programming operations on type level entities like concatenating them and combining shapes. Is the concat the only way to do it? What about an intersection? These operators only come into play with advanced theoretical langauges like Lean or Idris and it's very off topic.

> This isn't definable without dependent types or a concat operator at the type level.

It takes relaxing one overeaeger limitation in Python's type annotation syntax, a limitation which is necessary in general to avoid ambiguity, but could be relaxed in specific, enumerable cases that do not produce ambiguity.

> You're basically asking for stuff like this:

I'm not sure exactly what your examples mean because I'm not sure exactly what syntax that is supposed to be, but in actual Python type annotations with a relaxation of the no multiple unpacking rule for variadic generics, what would be allowed is things like the example I posted upthread for tuples:

  def tconcat(t1: tuple[*Ts], t2: tuple[*Us]) -> tuple[*Ts, *Us]: ...

or, for Arrays:

  def product(a1: Array[DType1, *Shape1], a2: Array[DType2, *Shape2]) -> Array[tuple[DType1, DType2], *Shape1, *Shape2]: ...

concat for arrays would take dependent types, but that's not what is being discussed.

> This doesn't exist in the realm of programming we're talking about.

All the concepts necessary for what the upthread poster noted TypeScript already has and was superior to Python because Python lacks, and which I pointed out Python is relaxing one limitation short of also having, already exist in Python since Python 3.11, there is just a syntactic limitation on using the necessary combination to avoid ambiguities (which occur in different circumstances). And, of course, all the necessary concepts also exist in TypeScript. And those two languages are "the realm of programming we're talking about".

> You want something well out of scope of this conversation

No, literally, I'm discussing the thing that is the advantage of TypeScript's type system over Python's that has been the subject of this conversation, which is not, ipso facto, well out of the scope of the conversation.

> and well out of scope of most practical programming languages.

It exists in TypeScript which is widely used, whether or not you thin it is "practical".

> These operators only come into play with advanced theoretical languages like Lean or Idris and it's very off topic.

The discussion was about a comparison between TypeScript's type system (which supports this) and Python's (which does not quite, though it has all the pieces with a syntactic restriction that prevents certain useful combinations.) These are not "advanced theoretical languages", and the discussion you jumped into is not at all off-topic (certainly not of itself, despite your jumping in to derail it, and not of the broader discussion of Python and scale because JS/TS is a similar industrially popular language ecosystem to Python with a dynamically typed base and optional statically type-checked layer.

I see. So ts arbitrarily supports concatenation of generic parameters via unrolling.

All right. I can admit TS is superior from this angle because of that arbitrary support. And I can admit that the arbitrary support follows from the way both languages use unrolling syntax.

That's the conclusion I also arrived at. The type system is slowly getting there, but many operations like this one are still not possible.

It's a shame, because it makes features like decorators significantly harder to use with static typing.

I haven't seen a type system that allows variadic types for Tuples. This would be equivalent to creating a struct with variadic amount of properties.

The definition for tuples here is similar to a struct.

They are one in the same except structs have names for each property while tuples don't. That is literally the main concept of a tuple, just a struct with no names for properties.

The type system for python is already "there", it is in fact superior to many other type systems from other popular languages.

That's confusing, considering Python has them with TypeVarTuples?

Yeah they're getting flexible with the definitions. The developers are committees of people many of which don't know type theory and introduce arbitrary concepts based off of misguided intuitions.

It's the same with typescript I'm sure.

You will note that this thing doesn't exist in haskell for tuples because haskell devs tend to be well versed with concept of what a tuple is.

So you keep going on and on about how this is wrong, and doesn't work, and doesn't exist, and it's impossible, because it isn't implemented in Haskell? Seriously?

This is one of the strangest and worst interactions I've had on this site, because I essentially made a comment that goes "the sky is blue, but it looks better at sunset when red" and you keep going "nu-uh! Impossible! Doesn't work! The sky is blue, nothing else makes sense!"

>So you keep going on and on about how this is wrong, and doesn't work, and doesn't exist, and it's impossible, because it isn't implemented in Haskell? Seriously?

Yeah seriously. You actually don't know what you're talking about. You're not convinced. I'm failing on that end, but this is real, you're not understanding me. Haskell one of the languages with the most advanced type systems in practical programming. Algebraic data types. You clearly haven't used it so of course you remain unconvinced.

>This is one of the strangest and worst interactions I've had on this site, because I essentially made a comment that goes "the sky is blue, but it looks better at sunset when red" and you keep going "nu-uh! Impossible! Doesn't work! The sky is blue, nothing else makes sense!"

It seems this way to you because frankly, you're not very knowledgeable of what I'm talking about. Read my other replies. I left one more to explain it to you in a bigger way. Hopefully you'll get it and stop violating the rules here with personal insults and calling this the "worst" and "strangest" interactions you ever had on this site. It's just fucking rude.

This is my last explanation for you:

https://news.ycombinator.com/item?id=38391431

If you still don't get it, well I can't help you.

> Haskell one of the languages with the most advanced type systems in practical programming. Algebraic data types. You clearly haven't used it so of course you remain unconvinced.

Wrong.

> It seems this way to you because frankly, you're not very knowledgeable of what I'm talking about.

This wouldn't come across quite as bad if I didn't literally post an example long ago. If you're so knowledgeable about all this, why can't you acknowledge the examples I've already given? Why are you ignoring them?

I'm not, I acknowledge everything you say. It may seem that I'm not acknowledging stuff. But this is not the case. I read and understand all your posts.

You on the other hand. I doubt you read or understood all my responses to you. You only partially understand.

I also don't think you have used haskell before. This is what I mean. You aren't knowledgeable.

> I'm not, I acknowledge everything you say. It may seem that I'm not acknowledging stuff. But this is not the case. I read and understand all your posts.

Then why have you not yet acknowledged the Typescript concat function I posted here yesterday? You have since written 16 replies, not a single one of which acknowledges this. Why do you keep not acknowledging that I gave you the function you consider impossible?

> I haven't seen a type system that allows variadic types for Tuples. This would be equivalent to creating a struct with variadic amount of properties.

No, it wouldn't.

It would be equivalent to creating a generic type where the concrete types that would be compatible with it would be structs of different shapes.

But if you use the same variadic type within a particular context (e.g., a function signature and definition), then any given call to the function, that variadic type must represent the same concrete type each place.

Reply on news.ycombinator.com