Conception - a live programming environment experiment after 1 year | Hacker News Reader

Conception - a live programming environment experiment after 1 year

zef.me

51 pointsshurcooL22 comments

This is interesting not only for the way it lets you code, but for how it lets you pipe commands together graphically. Unix pipes are awesome and all, but chaining any more than 3 commands gets a little confusing with all of the parameters.

I agree, the ability to connect widgets is one of my favourite aspects.

It lets you create a customized for your specific needs instant feedback so you can keep changing the source and see how it affects your "calculated view."

I want to be able to answer questions like:

- What is the diff between the output of my program as is and at master^1 commit.

Sure, I can manually copy my repo, check it out at master^1 and save its output to file1.out. Then save the current output as file2.out and do a diff. But since that takes more than 5-10 seconds, the cost outweighs the benefits and I'll say "screw it, I'll just keep going as is."

On the other hand, if I build a tool that makes it possible for me to answer that question in under 10 seconds (and keep answering it as I make relevant changes, if I want to keep getting the feedback), then that will change how I work.

Unix pipes are absolutely great, but there are some areas for improvement.

- It's hard to visualize the output at the intermediate steps of the chain. So if you have "cmd1 | cmd2 | cmd3", you'd have to spend more effort than it has to be in order to see what cmd2 and cmd3 got as input.

- They are limited to text for visualization of output. Most of the time this is good, but what if you want to see a plot of 2D points?

- They're typically executed when you press Enter. Sometimes you might want to say "always execute this whenever source file changes."

The awesome part about software is that you can create and do anything. If something is harder than it could be, you can make the computer make it easier for you [1]. It only costs time and money.

[1] http://alarmingdevelopment.org/?p=711

Dataflow models are definitely convenient. One of the nice thing about QuartzComposer is the ability to directly "probe" connections between patches to see what data was flying across them, whether this was text or something richer (e.g. graphics).

On the other hand, dataflow models are definitely limited in their abstraction capabilities, which is in opposition of directness. They work well in shells but have a harder time being used in large scale projects. I've never been able to make them scale past a certain point.

> - They're typically executed when you press Enter. Sometimes you might want to say "always execute this whenever source file changes."

You may use incron [1] for this. incrond watches for file changes and can execute based on file actions.

[1] - http://incron.aiken.cz

This is pretty cool!

This is yet another piece of the puzzle. At this point, I think the only missing ingredient is the ability to create temporary files without having to specify names (those can be created automatically based on time or whatnot). As long as the text box you're working with is addressable via some file descriptor, you can then start to create connections with something like this incron. And you could replicate the functionality that can be done in the current version of Conception.

> - It's hard to visualize the output at the intermediate steps of the chain. So if you have "cmd1 | cmd2 | cmd3", you'd have to spend more effort than it has to be in order to see what cmd2 and cmd3 got as input.

cmd1 | tee input2 | cmd2 | tee input3 | cmd3

> - They are limited to text for visualization of output. Most of the time this is good, but what if you want to see a plot of 2D points?

GNU Plot

> - They're typically executed when you press Enter. Sometimes you might want to say "always execute this whenever source file changes."

I don't know how to solve that off the top of my head, but I'm sure you could do it with a small tool.

Although I'm challenging some of your statements here, I do really admire your work on this. Also, I'd like to know what you mean by "make unnecessary duplication optional".

Just to clarify, I'm not saying what I have right now is better than anything. I'm just prototyping various things that I think may be better, then try using them, and compare. When I get ideas how to improve it, I make the change and see. For what it's worth, most of the things I've tried ended up being bad ideas that I threw away, but in the process of trying them, I understood _why_ they were bad and what stopped them from being as awesome as I initially imagined.

> cmd1 | tee input2 | cmd2 | tee input3 | cmd3

Thanks for that tip, that's handy. However, it's still not quite good enough for me for the following reasons. You have to come up with a file name, and make sure it doesn't exist, or you'll overwrite a valuable file. Why should you have to come up with a file name when it's just some temporary output. Perhaps using a script that generates a temporary file name could help. But then you also have to cat that file to see its output. I don't see why the intermediate steps can't be visualized in an easier way, such as with a mouse hover or click. Also, what if you want to manually modify one of the intermediate values and see the effects? It can be done, but probably not in the most succinct way possible.

> GNU Plot

I was actually thinking that there might be something along those lines. I suppose there might be similar command-line tools that take png images via stdin and display them in a window. That's pretty close to what I'm trying to achieve. :)

> I don't know how to solve that off the top of my head, but I'm sure you could do it with a small tool.

This is still the most important ability to me. I'd like to be able to establish various "views" into the existing code base, that help me understand the things I want to know. Their main property is that they're completely generated from existing content, and have nothing user-generated that needs to be stored. When the source content changes, the views should update automatically (if possible). There's no reason they shouldn't unless you're trying to preserve battery life.

> Although I'm challenging some of your statements here, I do really admire your work on this.

Thanks, I appreciate your constructive criticism. :) I will use your feedback in trying to make this project more awesome.

> Also, I'd like to know what you mean by "make unnecessary duplication optional".

I should probably find a better way to phrase that, given how important to me that statement is.

Essentially, I want more things to follow the DRY principle [1]. I'm very annoyed when I fix a bug in a function, and then realize there are many more duplications of that code, so now I have to fix the exact same bug in many other places. I don't like like having to do the same work many times. But today's world is filled with such situations [2], and I want to my best to make it easier to reuse code rather than duplicate it.

Making something that I think is bad "optional" is just me trying to put a nicer spin on this. I would say "I want to eliminate all unnecessary duplication [of code/work/manual efforts]" but then what if someone wants to duplicate things on purpose for some reason? I don't have a problem with that. I just want to make it easier for you to not have to repeat work.

[1] http://en.wikipedia.org/wiki/Don%27t_repeat_yourself

[2] Actually, I find that the situation has improved quite a bit in the last few years, and it seems to be improving slowly. But there are still many notable offenders that drive me mad and motivate me to work to make it a problem of the past.

Watch is a small script in Inferno sh to date and print differences in an arbitrary command's output, each time the output of that command changes. (Not an original idea.)

https://github.com/catenate/inferno-cat/blob/master/sh-infer...

You could obviously update it to execute an arbitrary command: e.g. (plot watch.out.new.$pid) instead of diff, whenever a data-generating program generates a different set of data.

https://github.com/catenate/inferno-cat/blob/master/sh-infer...

> Unix pipes are awesome and all, but chaining any more than 3 commands gets a little confusing with all of the parameters.

That's what abstraction is for. Make a shell script that hides some of the complexity and substitute the script into the pipeline. No?

This is interesting. Superficially at least, it reminds me of MAX/MSP or PureData, which are used to create music and other media. For me, it has the same shortcoming: it is very 'mousey', and with MAX/MSP and PureData at least, I grow tired of using them quickly because of this. Basically, I would like to use the keyboard for everything. Still, I respect the work, and I imagine this type of environment could be given a (perhaps VIM-like) mode that could enable keyboard selection and chaining of widgets.

I agree, it's becoming too navigation heavy for larger scenarios and code bases. This is a problem I need to address. Right now you can't "pin" things to another layer and have them stay there regardless of how you move the main layer. I think being able to do something like this might be helpful. After that, I'll have to see what else can be done... I'm thinking global fuzzy search to help get around.

It is becoming [1] a bit more keyboard-able, and being as open as it is, you could pretty much add any kind of shortcuts to anything you want. That's the whole idea that the user is in full control. Right now that part is probably too inaccessible for most people, but it's my goal to make it friendlier to customize.

[1] http://img341.imageshack.us/img341/6977/imageay.png

Nice. It reminds me a little of Prograph/Prograph CPX http://en.wikipedia.org/wiki/Prograph

Couldn't get it to compile on Ubuntu or OSX 10.8; I am definitely interested though... Is there anything like this that is more stable?

You should absolutely be able to compile it under OS X 10.8 without any issues. All you need is Xcode and golang installed, the project repo has everything else. You would also need git for the git-related stuff.

What was the error you got?

I've made it compile under Ubuntu earlier [1], but there may be some issues with dependences there. This one is trickier as it's not my main dev environment (and I'm taking some shortcuts for now).

[1] https://dl.dropboxusercontent.com/u/8554242/dmitri/projects/...

Actually I mis-spoke. It compiles but I'm getting an error when it tries to load the background texture. I definitely have OpenGL but I am getting some saort of OpenGL error. I will test that Ubuntu installation in a bit. Thanks.

Can you tell me what the error was (see the stdout + stderr output of Xcode)? I'd really like to help you get it to run, and figure out why it didn't out of the box (it should).

Which video card do you have? It should work even on an Intel 3000.

Sure, here it is:

  Using GLFW 2.7.5.
  CPU Count: 1
  GL Renderer: NVIDIA Corporation NVIDIA GeForce GT 330M OpenGL Engine 2.1 NVIDIA-8.10.44 304.10.65f03
  GLFW_ACCELERATED: 1
  GLFW_RED_BITS: 8
  GLFW_GREEN_BITS: 8
  GLFW_BLUE_BITS: 8
  GLFW_ALPHA_BITS: 8
  GLFW_DEPTH_BITS: 0
  GLFW_STENCIL_BITS: 0
  GLFW_REFRESH_RATE: 0
  GLFW_FSAA_SAMPLES: 8
  Window Dimensions: 1536x960
  Background thread (tid = 1) created.
  Background thread is starting.
  Error: Failed to load texture.

  rm: ./GenProgram: No such file or directory
  rm: ./GenDiff1.txt: No such file or directory
  rm: ./GenDiff2.txt: No such file or directory
  sh: ./bin/gocode/gocode: No such file or directory
  sh: ./bin/gocode/gocode: No such file or directory
  Sending kill to last child pid 92715.
  Background thread has ended.
  Background thread has been shut down.
  ~App()
  ~InputManager()`

Thanks.

I can see the issue is that the current working directory for the binary is not set correctly. It should be set to the repo root (the folder that contains README.md). The ./bin/gocode/gocode file should be present.

How are you running it? You should open the Xcode project and Run it (Cmd+R). The Xcode project is configured to set the current working directory as the project root folder.

That's what I had been doing. What I ended up doing to get it to work was unzipping the file, running xcodebuild and then moving the binary to the root of the project. Thanks for the help.

EDIT: Still getting this error though from within the running application: libc++abi.dylib: terminate called without an active exception ls: ./GenProgram: No such file or directory

Thanks for trying. It's really strange that it doesn't work on your OS X while it does on mine (and on someone else's system who tried it somewhat recently). I'd like to figure out why.

I've pushed a small update [1] that prints the current working dir of the app. Perhaps you can try it and verify that it's being set correctly. See this screenshot [2] for Xcode setting that sets the running app cwd.

[1] https://github.com/shurcooL/Conception/commit/37bbbb57e56caa...

[2] http://img526.imageshack.us/img526/7548/imagexcsb.png

Awesome that fixed it.

> Please steal any good ideas you see here. I’d rather have them benefit the world than die in obscurity.

Class.

Reply on news.ycombinator.com