How I Built a Self-adaptive System | Hacker News Reader

How I Built a Self-adaptive System

avalanche123.com

35 pointselasticdog6 comments

While I like the connection you made between your self-adaptive system and self-adaptive systems by Mother nature, I couldn't see the analogy beyond the fact that they both use the word 'self-adaptive'.

>> These systems are also decentralized and consist of a number of lower level primitive components, collaboration of which on a higher level produces very complex and intelligent behaviors.

After reading the first paragraph, I was getting excited and put my Emerging Behavior thinking hat on -- Particle Swarms, Ant Colony Optimization algorithms, or even Wolfram's New Kind Of Science and The Game of Life -- only to find out that the system was a if-then rule-based program?

>> While in most biological systems all of those rules are written in cell’s DNA, for a software system I needed to find a good framework for defining those.

Fancy stuff, the likes of Genetic Algorithms, Evolutionary Systems, Memetics? But no... Not those kind of self-adapting systems..

Am I missing something?

PS: Fuzzy control and Neural Nets sound like a whole different ball-game to me ;)

But hey, good stuff nonetheless -- thanks for sharing and glad to hear it was well-received

Thanks for feedback. Indeed, complex emergent behaviors are not demonstrated in the provided example. But they are possible when changes in environment affect other control loops. Which might result in conflicts, at which point control loop for conflict resolution can be added, this is why MAPE-K tower is a useful abstraction.

From the description, this sounds quite over-engineered. You need a system to queue jobs and to automatically restart jobs that fail? I must be missing something here, can you help me figure out what it is?

This was my initial idea. However, consider this scenario. I need to boot a web stack (db master, pool of db slaves, webservers and a load balancer). I boot a master, job succeeds, then I boot slaves, job succeeds. I boot webservers, but by this time master is shut down or becomes unavailable. Because of a job-based approach, this will never be detected and fixed by the controller. The chosen approach would detect the state of the environment and determine that a master is missing and proceed to correct the situation or abort the process altogether. Hope this makes sense!

Fundamentally, each of the tasks is a separate job. Why not just order the jobs in an array and start them in order? So if a job at some index needed to be restarted/started for some reason, you would first go through and start everything before it.

More generally, looks like you just form a DAG to map out the dependencies and use it to figure out what to do. The daemon could then periodically traverse the DAG starting from the root to each leaf starting jobs as required. Could you explain why this kind of approach was unfeasible in your scenario?

Well, success of a job is transient and not permanent in this case. Like I mentioned before, successful boot of a dependency doesn't mean that dependency exists by the time we get to boot a host. You need to continuously check the status of your group to determine actions that can be taken for current state. A stateful system that marks a job as complete upon successful execution wouldn't work. DAG computation does happen, but inside MoreHostsCanBeBooted condition that is a pre-requisite for BootMoreHosts action. If what you're proposing is to re-run the same job until target state transition has been achieved and code job in idempotent way, then this is essentially what's being done with the current approach, except there are no jobs, and idempotency is a side-effect of not tracking progress. I hope my explanations make sense :)

Reply on news.ycombinator.com