PunyForth: Forth inspired programming language for the ESP8266 | Hacker News Reader

PunyForth: Forth inspired programming language for the ESP8266

github.com

88 pointsdentrado45 comments

I'm reminded of my good old Forth times (6502 on Commodore PET), and I'm glad that Forth gets the increasing attention which it actually deserves.

I don't know if there is any other programming environment at all where you can have a extensible programming language, editor, compiler, interpreter, debugger, libs, and interactive shell in a few KB.

As much as I like Forth, it isn't the only player in the restricted memory arena. Several models of the Commodore PET came with 32kb of RAM, and 18kb of ROM. Here are some that reflect similar requirements.

uLisp [0], 2kb of RAM, and 32kb to store the interpreter in. You can write programs at the REPL, and burn them into the EEPROM.

TinyBASIC [1], since dead, is a port of the version of BASIC that used to run on the Altair 8800, which was an even more constrained system than most PETs! Altair BASIC could run on 4kb of RAM.

Python-On-A-Chip [2], is a subset of Python. At a guess, from the supported platforms, it can run on around 2kb of RAM.

iArduino [3], is a C interpreter that can run in 2kb of RAM.

MicroPython is looking to come to the ESP8266 [4], which is much, much bigger at 512kb. Which basically makes it the fattest of these, and impossible to run on something like the PET.

Bitlash is a completely new language [5], and can run on 2kb of RAM, and takes up surprisingly little storage space.

All of these languages offer at least a REPL, as well as compatibility with a wide range of libraries, and most have at least traceback and exception support.

This is not to say Forth is a bad choice! It works great in tiny memory, with better support for a full language specification than many other languages.

But not everyone wants a Forth... And there are plenty of choices out there.

[0] http://www.ulisp.com/

[1] http://hackaday.com/2011/08/28/basic-programming-on-an-ardui...

[2] https://github.com/jsnyder/python-on-a-chip

[3] http://n.mtng.org/ele/arduino/iarduino.html

[4] https://github.com/micropython/micropython/tree/master/esp82...

[5] https://github.com/billroy/bitlash/wiki

Additionally one should add that the mainframes where Lisp was initially developed, were quite constrained, even when compared with 8 bit micros.

Yes, there are plenty of options. However, most of those options lack something while Forth is all-inclusive. All your options have no compiler while Forth compiles to threaded code which is pretty fast, about 1/10 the speed of assembler according to my experience. Your options have a linear space consumption while Forth code compiles "logarithmic" due to its extensible vocabulary. That means, if you have an 8K system (like on my former PET-2001) you need 4K for a convenient basic Forth for instance, while you can put a lot more stuff in the other 4K, due to the threaded code.

Btw. for those who are interested in Forth, I recommend to read the "Forth Dimension" archive. It was rightly called the "best special-interest technical magazine ever".

http://www.forth.org/fd/FDcover.html

Well, TinyBASIC was available as both an interpreter, but more often, as a compiler.

Though it was a primitive compiler compared to today, due to the size of the language, it easily fit within 2-3kb, which on your 8kb system, would leave 5kb or more for program memory.

In fact, some TinyBASIC systems had a virtual machine and JIT compiler in 120 instructions, or around 2kb, as early as 1964.

Though BASIC is slower than Forth, or was in those days, the main benefit is instant availability. BASIC syntax was the jumping on point for more than a few programmers, myself included, because of it's simplicity.

This is fantastic! Forth's simplicity is a great fit for exploratory programming on resource-constrained systems, and I'm a fan of anything homoiconic.

Maybe one day I'll get my dream of having a forth or lisp with the type system of haskell.

Did you see the Factor language?

Factor is cool, but never really took off or got much attention. The creator, Slava has moved on to other things (Swift compiler I think). I posted something on another concatenative language (kitten) on here recently and the author jumped on and the conversation was good (Factor was brought up as well).

Aren't these conflicting ideas? To me a big advantage of dynamic type systems is that specifying behaviour and specifying what parts of the behaviour are statically computed are separate tasks.

Yes. For example, the powerful "transducers" concept from Clojure essentially cannot be represented in the Haskell type system.

Here's an attempt: http://hypirion.com/musings/haskell-transducers

There are some strongly typed Forths --- although I've never seen one that was out of prototype. Here's one I had a brief look at:

http://www.arestlessmind.org/2009/02/03/intro.html

Of course, since Forths are built around a REPL, to do type checking you need to keep the type objects around at run time, so it may not be suitable for very small systems.

Yes, but they are not conflicting in the sense that they are competing to serve the same purpose.

For exploratory programming, dynamic type systems are nice. For building reliable long-running systems or ones where failure is extremely costly, I like static.

Maybe Shen would work for you? [1] A Lisp with optional static type checking, optional lazy evaluation, pattern matching, a built-in Prolog, and more.

[1] http://www.shenlanguage.org/

Have you looked at Kitten? It's statically typed[0], and a descendant of Cat[1], which is a descendant of Joy[2]. So those might also be interesting languages to investigate

> The Kitten Programming Language

> Kitten is a statically typed, stack-based functional programming language designed to be simple and fast. It is a concatenative language, combining aspects of imperative and pure functional programming. There is an introduction available and a tutorial in progress.

http://kittenlang.org/

[0] http://evincarofautumn.blogspot.se/search/label/Concatenativ...

[1] http://www.codecommit.com/blog/cat/the-joy-of-concatenative-...

http://www.codecommit.com/blog/cat/the-joy-of-concatenative-...

[2] https://en.wikipedia.org/wiki/Joy_(programming_language)

This is great work. I was expecting a toy Forth, but this one is pretty serious. I was happy to see quotations, immediate words, and Factor-style combinators.

I am wondering why the team from mbed has not made an attempt to make ESP8266 compliant with mbed. Yes, I understand that the architecture is totally different but just that I like mbed so much so that I want to see it used in more hardware.

The "team from mbed" is employed by ARM.

mbed team would not do that. The compiler used in mbed platform is NOT gcc but ARM Compiler.

Great work indeed. Does it support the new ESP32 processors?

The ESP32 also uses an Xtensa, the LX6, which should have the same basic instruction encoding, etc (and I think they both use FreeRTOS, as well, but I only have an ESP32). Probably shouldn't be too hard. You can also stuff more functionality into it, too.[1]

[1] Related: I can't actually even find the documentation on what's precisely changed between the LX6 (ESP32) and the LX106 (ESP8266), other than the basics on the datasheets - I don't know of any documentation on e.g. any different architectural details. Maybe you have to ask Espressif. Or Cadence. Anyone have any idea?

Their datasheets only mention the PIN definitions and electrical characteristics of the microcontrollers. Try to get the information from the reference manual[0]. Also there are these books[1][2] by Neil Kolban.

[0] https://www.espressif.com/sites/default/files/documentation/...

[1] https://leanpub.com/ESP8266_ESP32

[2] https://leanpub.com/kolban-ESP32

edit:

More information about the differences: http://www.cnx-software.com/2016/03/25/esp8266-and-esp32-dif...

The ESP32 datasheet is really woefully inadequate for this part; it mostly describes the chip, but I guess what I'm really looking for is things like "A description of the ISA itself", i.e. the "LX6 datasheet", so to speak. So I want something like the the actual instruction encoding, instruction listing, etc for the LX6 series and what it supports precisely.

Most importantly, the ESP32 has register windows.

Right, this is the kind of information I was looking for (maybe some actual ISA documentation; the instruction encoding method itself, etc).

But for this specific point -- I thought all of the Xtensa series used register windows? Or did they both have them but the ESP8266 never actually use them (somehow?)

The xtensa series is highly configurable, afaik the ESP8266 doesn't have them in hardware.

This is second hand knowledge though.

Why can't we get Rust for the ESP8266? :( I really dislike C, I'd love a higher-level language. As great as MicroPython is, the interpreter needs too much RAM, and I think Rust would be a great alternative.

Isnt there an axpetimental Rust backend for ARM Cortex? ISTR seeing it discussed here.

The ESP8266 uses an Xtensa core, not ARM.

Rust has Tier 2 support[0] for ARMv6 to ARMv8 and Tier 3 support[1], so there are experimental limited[2] backends for the various Cortex in Tier 3.

However the ESP8266 is built around Tensilica's Xtensa, not ARM's Cortex. There currently is no support whatsoever for Xtensa in LLVM (let alone in Rust).

Current microcontroller work is on MSP430[3] with work being planned for Amtel AVR[4] since the AVR branch was merged into LLVM in November 2016. Xtensa is quite literally nowhere[5].

[0] guaranteed to build & binaries provided but tests are not automatically run

[1] supported in the codebase but no guarantees that it even builds

[2] only Core, no standard library

[3] https://github.com/rust-embedded/rfcs/issues/20

[4] https://github.com/rust-embedded/rfcs/issues/3

[5] https://github.com/rust-embedded/rfcs/issues/2

>However the ESP8266 is built around Tensilica's Xtensa, not ARM's Cortex.

DOH!

I have to say, though, that having the AVR code generator merged is very cool. At one time I was a big user of avr-gcc and remember when the AVR LLVM effort was just getting started. I haven't been following it, though.

One of the annoyances with gcc is that the optimizer kept wanting to use code motion to relocate the interrupt disable instruction to the beginning of the function it was found in. The only way to keep the optimizer from doing that would have been to make the instruction depend on all of RAM, which would effectively disable optimization for any function that disabled interrupts. So coding critical sections required a bit of jiggery-pokery to keep the optimizer from breaking your code. I hope the LLVM back-end found a way around that.

What's wrong with Lua? It has first class support thanks to NodeMCU, you can customise your firmware builds to be lean on resources using a dead simple online tool, and it even lets you run code on the ESP8266 in an interactive REPL environment.

Lua is strictly worse than MicroPython, I find. Memory usage is worse, and managing programs on it is bad too, although I haven't used it as much as MicroPython.

Have you seen the eLua LTR (Lua Tiny RAM) patch?

http://www.eluaproject.net/doc/v0.9/en_arch_ltr.html

As for Rust, why do we think that should use significantly less memory than MicroPython or Lua? I've seen some threads with people doing measurements on simple programs and finding Rust up to 10x heavier on RAM than C.

http://stackoverflow.com/questions/32762102/why-do-rust-prog...

If you looked at the link you included you'd see why the Rust version is larger, because they include the static library.

There's nothing about Rust that intrinsically makes it larger than C.

You would need a basic standard library built with the binary when building for xtensa (esp) aswell. So yes, the size of the core parts of the stdlib between c and rust definitely do play a role.

My problem with Lua and MicroPython is that your actual code takes up memory, ie even comments. Micropython can now compile to bytecode and load that, though, so it's better. I've had problems with Lua trying to load one or two hundred lines, though.

Lua is perfectly capable of compiling to bytecode and running that as well...

Oh is it? It's been a while since I last tried it, I'll have another look, thank you!

Can't say about NodeMCU or whatever, but Lua has had bytecode and all since Lua 1.1, back in 1994

Wouldn't Rust be overkill?

I think Forth is much more suitable here. You can implement a lot of features in 512K. You can extend the code (and language!) at runtime, you can debug remotely, and do other nice things.

Yes, but Forth doesn't have a borrow checker, so how will you know you're using memory safely?

These Rust trolls (at least I hope, for your sake, that you are trolling here and not serious) are getting annoying.

One would wonder how we managed to cope before Rust. Also, you should ask these guys [1] [2] how they know they're using memory safely. Has Rust ever been used to fly aircraft and fricking space shuttles? I didn't think so.

[1] https://web.archive.org/web/20110204160744/http://forth.gsfc...

[2] https://en.wikipedia.org/wiki/Open_Firmware

Because no one has yet bothered to port it?

As a side note, there are quite a few options to do safe micro controller system programming for those that prefer AOT compiled languages without C like unsafety.

http://www.mikroe.com/mikropascal/

http://www.mikroe.com/mikrobasic

http://www.astrobe.com

http://playground.arduino.cc/Code/AVR-Ada

Reply on news.ycombinator.com