Adding C-style for loops to Python (2022) | Hacker News Reader

Adding C-style for loops to Python (2022)

sadh.life

211 pointsRojerGS48 comments

Anyone looking to do this kind of thing, the macropy library makes it a fair bit easier: https://macropy3.readthedocs.io/en/latest/

You really do need to be modifying the AST and not just applying a regex to the source code text. This can be a challenge if you want to modify python syntax as you first need to get an AST you can modify, personally I recommend starting with the LARK library and modifying the python syntax grammer it includes.

https://lark-parser.readthedocs.io/en/latest/examples/advanc...

I use similar techniques to transpile openscad code into a python AST in my "pySdfScad" project, while still theoretically getting the benefits of fancy tracebacks and debugging and the like. Probably should have gone with a simple parser instead, but what can you do.

I think they should have stopped at the "cursed way" and not the "truly cursed way", if they really wanted the syntax changes than having your own python parser like the LARK implementation I mention above is a must.

It's true, the truly cursed method should never be used in a project that you actually aim to use. But, I just wanted to see how close I could get to the original idea, regardless of how "bad" the solution ended up being.

The second method indeed goes very far! I've made some very cool libraries and talks with that alone.

I don't understand the reasoning why the transformation directly on the source code is better than on the AST level?

Because you can use coding and then it's somewhat automatic?

But you can do sth similar on AST level, namely installing a meta path finder, i.e. an import module hook which does the AST transformation automatically (https://docs.python.org/3/library/sys.html#sys.meta_path).

Then, there is also the new frame evaluation API (https://peps.python.org/pep-0523/), which allows you to dynamically rewrite bytecode. This has been used by PyTorch 2.0 for torch.compile.

In part 1, they say, "I wanted it to look like a regular for-loop from C as much as possible."

The AST method seems to require you to write "with _for" instead of "for".

The method in part 3 allows you to write "for", which is closer.

So presumably it's better because it gets closer to the objective, although at quite a cost of course.

That only work in an import statement, not when you run the code directly.

I did something similar to option 3 to make the builtin numeric types "callable", since the dunder __call__ methods can't be overwritten for builtins. For example, in regular arithmetic notation, something like 6(7+8) could be read as 6*(7+8), but trying to do this in Python gives you `SyntaxWarning: 'int' object is not callable;`, since you're essentially trying to do a function call on the literal 6. The workaround was to use a custom codec to wrap all integer literals to give them the expected call behavior.

Repo if anyone is interested: https://github.com/ckw017/blursed

This was inspired by a way less silly usecase, future f-strings, which added f-string support to older versions of Python in a similar way using codecs: https://github.com/asottile-archive/future-fstrings

There really is nothing greater than an completely idiotic idea implemented with pure genius - this was immense reading and I learnt a lot!

Sounds a lot like the SIGBOVIK tag line: joke realizations of joke ideas, joke realizations of serious ideas, and serious realizations of joke ideas

https://sigbovik.org/

it is like a sitcom.

I've attempted something more crazy -- to make Python support multi-line lambda expression [1].

It's done with AST manipulation as well, but on a larger scale and completer functionalities.

[1] https://github.com/hsfzxjy/lambdex

So it seems you can register a source transformer with python which will trigger when a `# coding: ...` header is used (to signal character encoding).

This seems like an interesting avenue for static analysis tools, code generators, (etc.) to explore. Is it a viable approach?

I'd be interested in some analysis on this as a mechanism: performance, maintability, etc. wise.

The "dont do this" argument writes itself; nevertheless, its worth exploring.

Funnily this reminds me of how racket defines a language (https://beautifulracket.com/explainer/lang-line.html). Or how basically a shebang line does the same for any kind of script (https://en.wikipedia.org/wiki/Shebang_(Unix)).

True, but I think that source transformation on import are already supported in Python ("officially") based on importlib [0-1].

Here's an example use cases where the author is trying to create a Python enum from Thrift specs: [2]. The issue here is that the spec code is not necessarily valid Python, but can be fixed easily with some search & replace operations.

[0] https://docs.python.org/3/reference/import.html [1] https://docs.python.org/3/library/modules.html [2] http://wakandan.github.io/2018/06/07/python3-import-hook.htm...

One of the benefits of implementing your own module finder and loader is that you could use a separate file extension to avoid confusion with actual Python code.

behold, the .jsx but for python

https://github.com/dropbox/pyxl

https://github.com/pyxl4/pyxl4

Didn't know about codecs ...

The idea of using (abusing in fact) codecs to pre-process python code before it gets to the actual python interpreter is just fantastic!

I'm starting to think of all the terrible things I'm going to be able to do with this to work around things that have annoyed me for years in python.

[EDIT]: I'm thinking one can probably plug the C preprocessor in python now ... oh the sheer joy :D

> The idea of using (abusing in fact) codecs to pre-process python code before it gets to the actual python interpreter is just fantastic!

Coming from Javascript which has Babel, this is kind of an everyday occurance. Through the magic of source code transforms You can easily add whatever experimental new JS features you want to your project!

This is the kind of shit I love about python. You can put this in a pre processor script so that it works for every python program you run

A long time ago I stumbled upon (and contributed to) https://github.com/delfick/nose-of-yeti. The most delightful bit of cursed Python I’ve seen!

There is something to be said for doing something just because it's ridiculous. It's terrible and I love it.

"It is recognized that you have a funny sense of fun."

Thanks for this wild ride, it went way further than I anticipated ..

you're welcome!

It’s a bit sad that with-blocks can’t yield more than once. It would be very convenient to write retry code this way.

Wow, you just gave me an idea to make retry code with a loop:

    for _ in repeat_until_success(timeout=10, backoff=0.1):
        if success(...):
            break

That’s not taking exceptions into consideration.

There is actually a way to do it with loops.

    for attempt in retry(timeout=10, ..)
        with attempt:
            do_stuff()

Assuming you have a clever implementation of the retry function. IIRC tenacity has one that works like this.

It's better to make exception handling explicit. It has the same semantic complexity comparing to passing possible good exception types into retry or attempt.

Inline loop logic to decide if iteration is successful can be tricky.

This is evil, and I love it.

I wonder if the codec could use python's lexer (assuming it's exposed) to parse the for loops and nothing else. Then replace the loops with a placeholder, and then replace the placeholder in the AST after a parse. Might be cleaner than source->source transform by the codec, maybe not.

Where did the "cursed" adjective start being used like this? It's not used in my country, and I've only seen it being used in the last couple of years as part of subreddit titles.

Just from the description I think this doesn't handle continue.

it does!

Codecs free to transform all code before it’s executed? Sounds like the perfect place for hackers to hide RCEs very difficult to spot. Just hide an innocent comment at the top of a file.

So it's true that you can do anything with Python. Even if you shouldn't.

Very good!

(2022)

Two things:

1. I wish more HN posts ended in "for fun" -- so much of what makes being a progammer fun and enjoyable is plumbing the depths of what is possible, not because it's "best practice" or whatever. I think these types of forrays are where true mastery comes from.

2. The less reserved keywords a language has, the better. It makes things like this easier. Years ago I figured out how to implement Goto in Smalltalk "for fun". It was easier because Smalltalk had less reserved keywords.

I love 4 fun posts here, it's a shame they aren't more common, as usually when they're here someone inevitably with their nose as high in the air as possible chimes in with how this isn't profitable or a business model and will die soon.

Some people have a lot of time on their hands.

  (defmacro cfor [initialize condition advance #* body]
   `(do ~initialize
        (while ~condition (do ~@body ~advance))))

  (cfor (setv i 0) (< i 10) (+= i 1)
    (print i))

This runs on the python vm with hy right now. Took about 2 mins to have it up and running after noticing this post on hn. Couldn't pass it up.

I don't understand, this gives C style for loops with C syntax?

This adds c style for loops to hy, which runs on the normal cpython vm, and therefore can make use of the thousands of open source python libraries available.

My reply to the post was tongue in cheek. The point is that you can add on arbitrary language features to python using hy (any lisp, really). The syntax in question is normally referred to as an s-expression.

looks like it to me

  cfor(setv(i, 0), i < 10, i += 1) {
    print(i)   
  }

Do the brackets actually work or do you still have to get the indentation right?

no, just saying you can trivially convert any lisp code to c style syntax by moving the function call parenthesis to the right and converting wrapping parens into curly braces.

> Or alternatively: How I made the most cursed Python package of all time.

Beg your pardon, http://entrian.com/goto/ exists.

The range builtin already gives 98% of the functionality, just need to put "i in range" before it.

This was done for fun, not to be useful.

Indeed, but it’s not very fun when the result is already part of the language.

I say this with love: He needs a girlfriend (or life partner)

Reply on news.ycombinator.com