Pharo: An immersive programming experience
pharo.org
pharo.org
I like to think of each running instance as more akin to a Unix virtual machine than anything else. You end up using "in-image" version control systems like Monticello just the same way on a Unix server you use "in-operating-system" version control systems like git.
> I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages
So a Smalltalk instance is actually a system of systems.
[0] https://softwareengineering.stackexchange.com/a/58732/179286
Pharo now has pretty neat git support with Iceberg and Tonel file format.
greetings, eMBee.
i love the serendipity of this topic coming up just days after that post you found when i have not looked into it for a few years after trying for a few days to get that original squeak terminal to run in pharo.
greetings, eMBee.
https://www.fun-mooc.fr/courses/course-v1:inria+41010+sessio...
The original course is in French (dubbed in English) with English, Spanish and Japanese subtitles.
Other comments here are right: think of Smalltalk as being the complete computer. When I use Pharo I usually put it in full screen mode and don't think about the macOS or Linux host.
1. What's the advantage?
2. How do you manage the state of the environment?
The advantage is programming-as-teaching, as opposed to programming-as-carpentry. What I mean by "programming-as-teaching" is that old-fashioned Smalltalk (and Lisp) enviroments like Pharo are designed to support a style of development where you build a program by starting up the runtime and teaching it interactively, incrementally, how to be the program you want.
(What I mean by "programming-as-carpentry" is the more widespread and better-known model in which you are essentially working to build a planned artifact from a blueprint, and your development environment is analogous to a workbench or toolchest, rather than a student.)
If you don't like programming-as-teaching, then it's not an advantage. Most people don't work that way, but I think that's mostly because they don't even know it's an option. I'm not saying that everyone would prefer programming-as-teaching, but it gets so little exposure that I think it's a safe bet that more people would like it if they knew what it was.
I prefer programming-as-teaching over the alternative. I am dramatically more productive when I can work that way. I've been one of those fabled 10X programmers, as confirmed by VCS statistics, when working with such tools. It wasn't because I'm so brilliant or because I have amazing magical powers; it's because we were working with that kind of environment, and I knew how to use it and the other programmers didn't.
So, in brief, the advantage is that if you have such an environment and can use it effectively, you can be dramatically more productive. It changes the feel of programming from "let's build this variant and see how it turns out" to "reach over and correct that student's posture." The cycle of iterations becomes very fast, so fast that sometimes the duration of a cycle is imperceptible.
How do you manage the environment? First, let me acknowledge that modifying your working environment willy nilly is indeed a problem. The solution is that you don't do that.
Instead, you start from a known good system image. Environments like Pharo and other old-fashioned Smalltalk and Lisp environments offer a feature that can save the entire state of system memory to an "image" file. If you start the environment from an image, you get an in-memory state that is an exact duplicate of the state the environment was in when saved--right down to the positions and contents and activation states of all the windows. You start the image, and you're in the environment, exactly as it was when you saved it.
The way you manage incremental state changes is that you make a reference image that contains a known good state. A common choice for a reference image is a clean, unmodified release image. Then you make any site-specific modifications and customizations and save the result as a reference image as well. Then you save a working image based on that reference image and make your changes to the working image.
This practice enables you to easily recover if you get into a bad state. It's very fast to kill a bad session and restore a reference image. For example, a Pharo 6.1 image launches in under half a second on my older iMac. Clozure Common Lisp starts from an image in a similar amount of time.
When I've done ongoing serious work using this kind of system, we kept orderly collections of reference and working images--reference images that served as checkpoints in case of trouble; working images in which we had all the stuff we were working on loaded up and ready to go.
Besides making it quick and easy to resume yesterday's work, and to recover in cases where we messed up, images also served the useful purpose of enabling us to capture precise dynamic states of work in progress. For example, if a rare bug arises, I can save a working image with the problem on the screen, and give that image to my colleagues. They can start up the image and immediately see the problem state on their screens, and use the introspective capabilities of the development system to diagnose the problem.
But I guess that's more an advantage than a how-do-you-manage-it observation.
One question I have is on version control. I understand the value of reference images and release images and debugging images (with a rare bug front and center) and so on. Would this play well with diffing as I am familiar with it from a carpentry perspective, e.g git diff?
Would you serialize and dump a set of source code from an image? Would you build the reference image from a source code project and manually handle the consistency between source files and images?
I know it's a vague multi-question, but I hope you understand what I'm getting at, and I would really appreciate learning more about this style of development.
I've tried something like this with ipython, since you can save the session's input as a .py file. As long as you don't depend on the outside world you can recreate your image by using the bare repl as your "base image" and just load the (gittable) previous sessions input (which itself possibly loads older and older sessions).
Use of git/monticello within the image directly mirrors use of git at the command-line in Unix.
You use git/monticello to clone a remote repo, installing the code it contains into the live image. Because of other aspects of the Smalltalk design, this has an additional effect of being a bit like package installation - the code isn't just present, it's installed and ready to run. Unlike Unix, the lack of an explicit system-compilation step in Smalltalk means it's also ready to edit in-place, which means, if you've started some service instances, ready to edit live.
It's very much like Docker or working with a Unix VM: you start from a known image, like a Docker FROM command or a Debian ISO, and install files/code/packages from there, using version control tools or not as you like.
Yes, traditional Smalltalk systems can serialize sources to files, and can read them back into a running image. In Smalltalk jargon that's called "filing out" and "filing in".
Lisp systems generally use plain old text files as the canonical form of storage for source code. A given project consists of a set of source files, just like a C or Java program. The utility of image files is that they give you (1) the starting Lisp environment; (2) a reference image customized for your local site that starts up fully customized and ready to go; and (3) working images that record a specific state that you were working on, to preserve your context.
Git and tools like it are not great for managing image files. Images are large binary files, and successive versions of images tend to have collections of small changes to them. This is exactly the kind of file that git does not handle well at all. Consequently, I don't generally use git or similar tools to manage images, and I don't imagine a lot of other Smalltalk and Lisp programmers do, either (correct me if I'm wrong!).
It would be great if we had some sort of git for large binary files. I don't mean something like git-annex--a way for git to duck the problems of managing versioned large binary files by outsourcing them to some other storage. I mean it would be great to have a revision-control system that could manage versioned binaries as well as git manages versioned text.
Lisp and Smalltalk programmers are not the only people who would benefit. Any project that needs to manage a lot of media files would benefit. One obvious example is in game development. Even small games tend to need to manage versioned files containing thousands of image and sound resources. It's a giant pain. Git for binaries would be a godsend for projects like that.
And for managing Lisp and Smalltalk image files, too, but we can't rely on that demand. The number of people who need to manage a lot of Lisp and Smalltalk images is even smaller than the number of people regularly using Lisp and Smalltalk in their daily work.
"Would you serialize and dump a set of source code from an image? Would you build the reference image from a source code project and manually handle the consistency between source files and images?"
Yes, this is a normal way to manage changes in Lisp projects.
I should mention that, even when I'm using images as I've described, I use text files, not image files, as the canonical storage for my code. I use images in the ways I've described, but I don't use them as the main way to store the reference version of my code. Consequently, the normal way I manage Lisp projects (and the way I've seen other Lisp hackers do it) is to use git or similar tools to manage versions of the source code. I build reference images, working images, and release images from those source files.
Smalltalk systems have always been more oriented toward managing everything from within the image, and the Smalltalkers I've worked with generally treated text files as a serialization format for exchanging code with others. The canonical form of the source code was in the image. (Well, really, it was in the sources file, but that's basically just backing store for the image.)
I work with Lisp much more than with Smalltalk, so I'll defer to other commenters on the current best practices for managing Smalltalk projects with revision-control tools.
I will mention that some older Lisp systems were a little more like Smalltalk systems, in that they had more tools for version management than currently-available systems. Interlisp, for example, had a pretty impressive change-management facility called Masterscope, and Lisp Machines had a versioned file system--the machine's filesystem was a revision-control system.
Basically I'm a "code-as-carpentry" guy who's progressively being required to write applications in a way that DevOps can readily consolidate in our cloud provider for optimal resource use. But I don't have a lot of experience developing for this environment, so some problems like scheduling tasks are proving difficult (apparently this is doable with message queues but I haven't had much luck with that so far).
Your description sounds like it solves all of these problems. If you ran this in a container/pod/whatever, when it received a signal from DevOps to shut down for consolidation, it could immediately respond by saving the system state to an image and picking up from that state when the container/pod/whatever was started back up. This would also solve the problem of long-running processes since they could freeze in mid-run and pick back up like nothing happened, if I understand how this works.
So if you're right that programming-as-teaching, and image-based development, is a good solution to the issues you face, then an old-fashioned Smalltalk or Lisp system seems like a good choice. If you want those features, but you don't want Lisp or Smalltalk, then I think your only option is to build a new development environment for your language of choice that has the requisite support.
That's a tall order.
Sometimes I've idly toyed with the idea of writing a Ruby implementation on top of a Common Lisp or Smalltalk runtime, but that would be a huge amount of work with no guarantee that there's any demand for it.
For fun, here's some Smalltalk code from the MagLev codebase that modifies the image's Object class to better support Ruby:
https://github.com/MagLev/maglev/blob/master/src/smalltalk/r...
Usually, this makes one think: "ha, this is not right" and then goes back on the execution stack and either fixes the test and continues or refactors something etc.
But in a dynamic language, tests are required to avoid regressions.
Programming as teaching goes both ways, programmer is teaching Smalltalk, Smalltalk helps in revealing thinking flaws to programmer. Loop continues.
Indeed, in my experience, any expression I write in the repl is more likely than not to be a test, or part of a test.
The way I normally work is with a running Lisp image and one or more files open in editors. For greenfield development I usually create a project and write a loadfile to load any library dependencies I expect to use. Once that's loaded, I write some expressions to build some test data, and then some more expressions to operate on the test data. I evaluate the test-data constructors and then fiddle with operations on the data to assure myself that what happens is what I expect to happen. Basically, I start with an idea of how to accomplish something, sketch out the data structures and APIs I think I'll need, and start exercising them to see what I got wrong, what I need to add or change.
The contents of those source files gradually become the source code of my program and of the test suite that go with it. When the project is new and small, the sources are a few files with a small number of lines of code, and the organization is minimal and loose. As I flesh out the program, the source files grow more numerous and more formally organized.
As a project gets complex, I may choose to save working images at points where I'm confident that the state of the system is good. Such images enable me to start the Lisp environment with all of my code already loaded and ready to use. I can then work from that point in the same way that I've described.
The normal way I work on anything, unless the tools I have to use prevent it, is to have a work-in-progress running, and use variable references and function calls in the repl to query the and modify the dynamic state. For any expression more complicated than a short one-liner, I actually write it in a source file and copy it to the repl (actually, I usually use Emacs with SLIME, so "copying it to the repl" really means typing a keystroke or two in the source file).
One of the differences between old-fashioned Lisp and Smalltalk systems and newer systems with repls is that the old Lisp and Smalltalk repls are capable of doing every part of program development--defining and redefining functions and data structures, running and debugging code, inspecting dynamic state, catching, inspecting, and modifying erroneous code, and building the finished app for deployment--all by typing expressions at the repl.
To put it another way, the entire language and development tools and all of your application's internals are always in memory and completely accessible from the repl.
It's not that you would normally do all your development in the repl, although you could if you wanted. Rather, the important point is that, because the repl can do anything the language and development tools can do, there are no artificial limitations on what you can do from the repl. Outside old Lisp and Smalltalk systems, every repl I've used always imposes some restrictions--some things you can't do from the repl. Invariably the way I discover this is by needing to do something in the repl and finding that it won't let me.
Also, there's no guarantee that you'll find it congenial. Some people profess to prefer programming-by-carpentry, and that's fine, of course.
In fact, maybe it's better for your sake if you don't like it. The great majority of programming work is programming-by-carpentry. You might find that you really like programming-by-teaching, and then spend the rest of your life pining for the opportunity to work that way.
There are a few videos floating around the net that demonstrate some aspects of this style of work. Some are ancient promotional videos, like Symbolics people showing how to interact with a Lisp Machine. There are also a few more recent ones--for example, there are a couple of videos that Rainer Joswig made showing various interactions with Lisp systems.
Those videos don't really do the subject justice, though. They're short, and they show off one or a couple of nifty features, but programming-as-teaching is a whole-system approach, and it's hard to grasp it without comprehensive exposure to the whole of the system, and how the parts fit together.
I think that contributes to the obscurity of the subject: not only do few programmers get any exposure to these systems, but small exposure isn't really enough to explain what's distinctive about them. You need a pretty thorough exposure to the whole system and how its parts work together to understand what the deal is.
"7 minutes of Pharo Smalltalk for Rubyists"
http://www.virtuouscode.com/2015/05/11/in-which-i-make-you-h...
Nowadays, an image should not be considered as a long lived artifact, one would have a CI job to create the image. See https://github.com/hpi-swa/smalltalkCI
There is also Metacello with BaselineOf constructs etc.
Advantage is that when you get an exception, you just go back in the stack, fix, and then proceed forward instead of looking at a stacktrace and wondering what happened (yes, pdb is nice but not the same feel at all).
Smalltalk, like Lisp, allows for image customization.
a similar concept exists in F# type providers. it doesn't always work out well in practice but there is a ton of promise there.
I've only read the hype, and it sure reads well... :-)
For given example data, type providers output just something and typically it's 95% what you want. But how to fix that 5%? You cannot just say that "Type.Property, could you please be an integer". Instead of you have to tweak example which in some case is just fine but on some cases it's not possible at all.
Example: you use these SQL libraries which are based on type providers. You have some clear schema and it's just works. Then some day you would like to use runtime defined columns but at that point you have to opt out from that type provider library (at least for queries related to that table).
I still love working with F# but I don't use much of type providers. I think they are like proof of concept that compile time meta-programming can be provide real value but it need still more freedom.
Edit: I use SQL library called Rezoom and while it has it's limits it has given great benefit during development. When you change DB schema every effected query and model transformation gives compile time error. Value there is real.
Or does it link back into the IDE & debugger in some way I don't know?
I don't have a specific goal in mind but have always been interested in Smalltalk, and am not sure which one to try out. Thanks!
But I'm sure there are people that know more on here. Like Alan Kay, for example :)
Pharo started as a kind of Squeak fork, where they wanted to pursue other ideas related to JIT, GC, compiler technology, traits and so on.
So basically both are Smalltalk-80 implementations, with Pharo trying to be a more modern version of it.
I tried out Squeak about ten years ago, but never could get past the awful bitmap font. Naturally, the first thing I noticed on the Pharo page was that the fonts look nice! So I installed it on my WQHD ThinkPad to try it out.
The first thing I noticed is the way it responds to the TrackPoint. It is not like any other application I have ever used. Instead of letting Windows control the pointer position, it's doing something very different, I just don't know what.
It's not so bad with the touchpad, but I don't use the touchpad except for scrolling.
It doesn't support high DPI either, so the text is all blurry. Way better than the old Squeak bitmap fonts, though.
I don't want to sound like a complainer, it just wouldn't be pleasant to program in this environment when every other app and dev tool I use supports high DPI and correct TrackPoint response, both on Windows and Linux.
[1] https://twitter.com/AliakseiSyrel/status/1050781535983550464
The point is to be a system to be improved over time.
Squeak is more conventional I'd say.
I don't think that it's a language thing. It's probably popularity thing most of all.
In popular languages the quality of the average line of code is pretty low. Plus, you know, real-world constraints of real developers working under real-world constraints....
With normal languages like Java etc. You can just use a different, accessible IDE (i.e. Eclipse instead of Net Beans). That creates some problems, especially when working with a team, but in the end, it can be done if you try hard enough. Smalltalk and smalltalky things that try to be GUI only, no command line at all, are completely hopeless, though.
This is an area where Apple really doesn't get enough credit. VoiceOver has absolutely revolutionized blind persons' ability to interact with consumer tech.
For more technically demanding applications, the Unix philosophy is inherently open to being made accessible. https://pjb.com.au/blin/.
As an aside, I personally really enjoy accessible websites, because they focus on clear communication in the presence of a serious obstacle, rather than assaulting the visual cortex with attention grabbing nonsense.
Edit: Also one could argue that ed(1)¹ is the pinnacle of accessibility.
http://www.cincomsmalltalk.com/main/successes/education/laza...
OK, I will wait a few more years. I love Smalltalk, but I don't think I am qualified to fix this myself, unfortunately.
Upcoming is Bloc, which uses Mozilla2D as rendering engine and is vector based.
Next Pharo UI will use it.
The TrackPoint response is all wrong too, but one step at a time... :-)
Should be better anyway, but it is usable.
I already got it better but the VM has changed since and the rendering is not as good (but is faster for some reason).
Vs IntelliJ, it sucks a tad.
Pharo's code browser is visually compact. The code too. Current VR HMDs have only a small VGA-ish region of non-blurry pixels. Hmm...
Might be a good match. Emacs and IDEs with non-compact languages can be head-sloshing. And the HMD OEM market is being dysfunctional, so higher resolution is on the "I don't see a unicorn here - who cares about easily improving the world?" slowmo snail plan.
With better lenses and a better panel, an HMD might be competitive. But it's not perceived to be market, and there's pervasive confusion about the (minimal) GPU and (slow) fps needed. And there's only limited diy infrastructure.
But I enjoyed live-coding a vr environment (hot loading three.js and react.) At least with desktop mirroring, to flip-up to screen when the world broke.
Perhaps https://pharojs.github.io/ ? Croquet Project, Open Cobalt, and OpenQwaq are all dead, and 3dicc seems to have gone closed-source commercial. Ah, ST - feels like the 1980's all over again. :/ But https://github.com/ronsaldo/woden looks alive. And someone apparently hacked a bridge to Unreal. I wonder what else is out there?
Kinda sounds like an object oriented Emacs? Or what the creator of Nightlight is trying to do with Clojure.
Deutsch and Schiffman designed the first Smalltalk JIT compiler in 1984.
The IDE and the repl-equivalent, along with the entire windowing system and the rest, are all just Smalltalk programs within your installation. So you use the tools to edit the tools.
https://github.com/guillep/Scale
"Scale aims to take Pharo into the shell. That is, to write shell scripts in Pharo, use its power, and have a better syntax instead of the ugly bash one"
There are a set of command line handlers that can be invoked and do eval or load code etc.
Current work ongoing on ClapSt, yet another command line handler. https://github.com/cdlm/clap-st
In practice, things like using a browser for documentation is acceptable but not super common because viewing in the IDE is so much more powerful in certain ways (e.g. executable examples, finding "senders" of a message).
The idea is about having one's computer be "turtles all the way down" where you have the same power to change and explore every layer of the system without learning a whole new technology - from what we usually leave to the OS (which Dan Ingalls said "shouldn't exist") to what currently exists in apps (which Alan Kay and the creators of Smalltalk envisioned as services, which didn't stovepipe possibilities/power down to a small fraction of what's available).
There's also Monticello which is similar, but uses different repository types than git.
You can also just "File out" a class or package, which exports it to the file system.
(Internally, they are actually represented as 31-bit numbers for performance reasons)
Floats are also boxed automagically in a 64-bit entry.
However! When the rubber meets the road, there has to be some kind of bit-level representation somewhere. In Smalltalk, the VM hides these details from the programmer, who gets to reason uniformly about objects without caring about their representation. Specifically, Smalltalk VMs often use pointer tagging [0] just like lots of other dynamic languages do.
[0] Lots of interesting stuff on pointer tagging here: https://www.snellman.net/blog/archive/2017-09-04-lisp-number...
For example in Eiffel is even more complex, when you create a class it is possible to tag it with the default behaviour (reference or value), which you can later override when declaring variables or type parameters in generic code.