Developing Dataflow-Like Programming Systems | Hacker News Reader

Developing Dataflow-Like Programming Systems

kickstarter.com

57 pointsgerry7815 comments

I've worked with Flow-based programming in the past, and I use a similar model and principles when writing network servers.

Looking at the proposed table of contents, the topics listed are the topics I would like to read about.

I'm going to support this Kickstarter campaign because I think this model of programming is going to become increasingly more and more prevalent in the next 5 years.

One of the hardest problems I've run into is scheduling, and I look forward to reading his opinions on it.

I experimented with event-driven scheduling for the flow-based system I developed for home automation, but in the end I took the straightforward route of scheduling everything in lockstep, based on distance from source nodes. I've considered some parallelizable hybrid approaches, but haven't got around to replacing what already works well enough.

Perhaps this book will be worth checking out to see some additional ideas on scheduling.

I'm the author of the book...

It may be fairly simple for you to change your design to generate a static call graph for the nodes using topographical sort. Then on each "tick" you simply execute each node, in order, from top to bottom.

I always like to use a stack to schedule execution because I can then change the algorithm that schedules them to use a very different method.

- Matt Carkci

Topographical sort is actually what I use, storing the results of the sort in a run list as you describe. The tricky part was dealing with cycles in the graph.

So, there's this guy named Arvind Mithal, and he's thought a lot about this problem.

http://en.wikipedia.org/wiki/Arvind_(computer_scientist)

Are there any links etc to systems or tools the author has worked on? I'm interested in the subject matter but usually I look for a book by someone with demonstrable domain expertise.

Did a bit of clicking around - looks like he's written something called [Relay][1].

[1]: http://www.deepfriedcode.com/relay/index.html

Relay looks a bit like MAX or Quartz Composer or vvvv. After writing a couple data flow DSLs for visual music, I think I'll support his work. Would prefer C++ over C#, as I now code for iOS.

Yeah, I'd prefer C++ as well. He does say the the language may change though.

There are quite a few FBP systems available for various languages

The UI was modeled after Quartz.

- Matt Carkci

Not sure why a book needs crowdfunding: what costs need to be covered that prevent an author from just writing it?

supposing it takes you N hours to write it and you value your time K dollars/hour than the cost is N*K that you may have spent doing something else be it consulting, writing technical articles for a magazine or spending time with your family.

Given the goal (3K$) I'd be inclined to consider this a "if I pre-sell this much, it's worth my time, otherwise I'll do something else".

I asked the author this very question and his response seemed to be that he has already been planning on writing the book for a long time and the goal is just a small compensation for doing something he loves.

As you said, $3k doesn't go very far if you're thinking in terms of $/hr.

The only issue for me with backing this project is, the author does not have experience being a write/teacher. As a developer he has experience documenting the systems he creates, but I'm not sure how easily that translates to writing a book aimed at teaching a complex concept like dataflow systems.

I'm the author of the book and it is true that this would be my first published book.

My life has been very busy up to just recently. Now that I have extra time I want to spend it productively. $3000 is the minimum I was willing to make to spend my time to create a complete book, not just a series of blog posts. In short, I'm home with the kids while my wife works.

I started "Relay" more than a year ago just as a personal project. But in talking to others about it, I found that many people have lots of questions about dataflow. At first I thought I would just create a short blog post about dataflow to attempt to answer the questions. So, I started working on it... and it kept growing... and growing.

Even topics that you are an expert in, when you look hard you can find new things to learn. I started collecting links to articles related to dataflow just for my own edification. Eventually I posted them all to a new subreddit /r/DataFlowProgramming so that everyone could benefit.

So... I had a lot of text already written, a huge collection of notes, and time on my hand and enough material for a book that I felt would answer many questions people have. I thought I would give it a shot.

Kind Regards, Matt Carkci

Reply on news.ycombinator.com