Boiling Sous-Vide Eggs Using Clojure's Transducers | Hacker News Reader

Boiling Sous-Vide Eggs Using Clojure's Transducers

blog.eikeland.se

91 pointspi-rat26 comments

This is a really cool toy example of one of my favorite algorithms. For an algorithm that predates the transistor, I'm constantly blown away at its modern use cases. For example, dynamic pricing, used everywhere from airline tickets to parking meters, is nothing more than a variant of the pid algorithm.

It is also used quite a bit in capacity allocation in a wide variety of industries. Instead of using NP Complete scheduling and discrete optimization that might not terminate, you take a page out of Hayek and use the PID algorithm to set a price on the capacity of each independent asset. Then, scheduling and allocation often becomes a matter of convex linear optimization.

In terms of practical use, I have zero doubt in my mind that the PID algorithm accounts for more economic impact than Page Rank or many other HN-popular algorithms. Just adding up the value of applications that I personally know about, it saves its users > $10B/year. I would wager that the only algorithm that surpasses its economic impact would be Simplex.

It is also incredibly extensible. I've seen variants where the integral term is swapped out with fourier transforms, or the derivative term is swapped out with ARIMAs or Hierarchical Forecasting algorithms or Neural Networks. You can hack the shit out of it, tailoring it to even the most obscure use cases if you wanted to. Like, for example, cooking an egg.

PID is everywhere although for many real control systems the implementation isn't very sophisticated.

I was surprised to learn that it only dates back to the 1890's (according to Wikipedia). Seems to me like the Greeks or Romans would have figured it out already.

I was surprised as well when I first learned about them...Control Theory as a formalized scientific/engineering discipline didn't even exist until after the proliferation of the Centrifugal Governor, which has some varied accounting of its history, but generally is not quoted any earlier than the late 1700s.

One of the beauties of the algorithm is how well it scales to complexity. It can be implemented in something as simple as a mechanical or hydrolic controller or DSP, but can also be used in heavily modified forms that take advantage of advances in Machine Learning, OR/Optimization, etc.

The Greeks and Romans to my limited knowledge weren't very good at continuous maths. That whole domain of math seems fairly recent, at least in the western world. Calculus itself wasn't invented until Isaac Newton or a little before.

That is fascinating. Can you share some references on this?

For temperature control, you probably want to use a PI controller with Anti-windup. This will compensate for the limits on actuator (heating element) output, and prevent large overshoots from "windup" of the Integral element.

http://jagger.berkeley.edu/~pack/me132/Section15.pdf

Very good tip, thanks! I'll consider that for my proper setup, I've had some problems finding proper calibration values for some water containers and heaters because of the lag in the system.

To elaborate on why you can typically avoid derivative control for thermodynamic processes:

There's no natural resonance in the system caused by energy-storing elements. For instance, if you drop a cold egg into a pot of water at exactly 80 C, it won't overshoot and go up to 85 C before settling to 80 C. Mechanical systems on the other hand often have "spring-like" behavior that causes them to resonate at certain frequencies.

If you want to get hardcore with the control design then you could look into fuzzy logic controllers. It's common in high-end appliances: http://en.wikipedia.org/wiki/Fuzzy_control_system

I built a sous-vide rig with an Arduino[1] last year. The first recipe I made was deep fried sous-vide egg yolks[2].

[1] https://learn.adafruit.com/sous-vide-powered-by-arduino-the-... [2] http://seattlefoodgeek.com/2010/09/deep-fried-sous-vide-egg-...

  (defn pid-transducer [set-point k-p k-i k-d]
    (fn [xf]
      (let [pid (volatile! (make-pid set-point k-d k-i k-d))]
        (fn
          ([] (xf))
          ([result] (xf result))
          ([result input]
             (vswap! pid (fn [p] (calculate-pid p input)))
             (xf result (:output @pid)))))))

If guessing that the first 'k-d' on the third line should be a 'k-p'. I'm glad this bug didn't mess up your eggs!

Doh!! Thanks, post updated! No wonder it seemed to converge faster when I added a tiny derivative. (The gains were pretty much just guesswork, did two runs with just water before adding the eggs. It's a one evening toy setup after all, so don't want to spend too much time calibrating the pid loop)

Please take this in the ha-ha-only-serious spirit: I wonder if a static type system would have caught the error in this case...?

Probably not unless you make k-p, k-i, k-d types - as they are simply floats. Checking for unused function arguments however would have caught this error.

Running a linter (such as eastwood) in clojure does the trick:

  $ lein eastwood '{:linters [:unused-fn-args]}'
  == Eastwood 0.1.4 Clojure 1.7.0-alpha2 JVM 1.8.0_05
  == Linting pid.core ==
  {:linter :unused-fn-args,
   :msg
   "Function args [k-p (line 32, column 33)] of (or within) pid-transducer are never used",
   :file "pid/core.clj",
   :line 32,
   :column 1}

Sadly it only works if you actually use it, and I'm a lazy bastard :p

If you used Cursive, it would have marked that parameter as unused right in your editor. Even lazy bastards get the advantages!

From the picture it looks to me like egg is undercooked i.e. the white is still runny and kind of mucousy, ideally you want a solid tender white and a totally separate uncooked yolk. A slightly higher temperature (67 deg C) would ensure the white properly solidifies whilst still being cool enough to avoid cooking the yolk. 45 minutes is unnecessarily long a long as he egg remains under 70 degrees not much will be happening chemically once it's warmed through.

I agree, 67 could be better, you can also use two different temperatures, one for the yolk and then one (higher) for the whites for a short time - by ramping up the heat (or using two baths) at the end of the cooking. Better picture of another egg from this run: http://cl.ly/image/030N390L0D06 - white not totally set.

Also, the sensor might not be calibrated correctly, I have a reference thermometer (therma-pen), but won't bother calibrating a toy setup when I have a proper one ;)

The two temp solution works, or you can just get rid of the unappealing loose/runny parts of the white by gently rolling the cooked egg over a paper towel. The following chart can be helpful when you're getting started.

http://www.cookingissues.com/2011/05/03/tea-time-cooking-iss...

According to Dave Arnold and his chart 62 will be the perfect temperature for set white and runny yolk (he has a nice chart)

http://www.splendidtable.org/story/theres-more-than-one-way-...

he has a video demonstrating temperatures https://www.youtube.com/watch?v=GpvbNG1Dzhk

Guess it boiles down to personal preference

Dave Arnold and Harold McGee are awesome. Anybody who is interested in food, cooking, and science should listen to the Cooking Issues podcast and read 'On Food and Cooking'.

62 and 63 are often viewed as ideal temperatures (for some definition of ideal), but yes one needs to accept some runniness of the whites.

Over at the Food Lab, Kenji did some research - via mixing methods you can manage to set the outer white at those temperatures and not just the inner white: http://www.seriouseats.com/2013/10/sous-vide-101-all-about-e...

Personally I normally cook mine to 63, and yes the white can sometimes not be perfectly set. As with everything there is some unit to unit variation.

wrt

    (def pid-output (chan (pid-transducer 65 0.1 0.02 0.01)))

Unless something has changed with core async I believe you need to provide a buffer if you are going to use a transducer.

From the current doc: "If a transducer is supplied a buffer must be specified."

Good catch! Originally while developing and running this I had a buffer of 1000 on both temperatures and pid-output (to avoid deadlocks). I removed the buffers for clarity in the blog-post, but obviously should have left the transducer channel alone). Thanks!

Out of interest, why are you using a record for the PID, and not a map?

I had this grand plan of playing with core.typed, type annotate everything (ann-record Pid [set-point :- Number .....]). But the types made the examples too confusing for a blog post, so I dropped the idea.

The record stayed though - but there's no reason why it couldn't simply be a map.

I believe you could also use the heterogeneous map (HMap) type if you wanted static typing:

    (defalias Pid
      (HMap
       :complete? true
       :mandatory {:set-point Num
                   :k-p Num
                   :k-i Num
                   :k-d Num
                   :error-sum Num
                   :error-last Num
                   :output-max Num
                   :output Num}))

And then use it as you might expect:

    (ann calculate-pid [Pid Num -> Pid])

There's no real downside to using a record, but they're really only necessary for polymorphism.

Thanks! I'm just getting started with core.typed, and record felt more natural as it's annotated just as you would annotate a function. I expected there to be a way to create a type a map just like that, but just haven't had the time to read up on it. So thank you for the example, will definitely play more with core.typed.

Reply on news.ycombinator.com