Ask HN: What happened to fuzzy logic? | Hacker News Reader

Ask HN: What happened to fuzzy logic?

107 points_44875 comments

Fuzzy Logic was a very talked about field few decades ago. What is the state of that field of research? Is it still being persued or are there applications of it already implemented commercially?

View on HN

I don't endorse the following as an accurate history, but it is a narrative that is taught within machine learning and probably has some elements of truth to it.

If you want to take a system that's working on Boolean logic, and introduce uncertainty, replacing true/false with probabilities does a great job of this. However in the 1960s/1970s, people believed it was hopeless to have AI systems use probability to deal with uncertainty. This is because probability requires you to use Bayes' Theorem, and computing the denominator in Bayes' Theorem requires summing over an exponentially large number of different outcomes.

Thus people came up with all kinds of alternative systems to avoid dealing with probability. Among these would have been fuzzy logic.

However people came up with ways to cope with the computational intractability (belief propagation on Bayesian networks, better Markov-chain Monte Carlo algorithms, etc.), so probability became practically viable. And on the merits, if you can deal with the computational issues, probability seems to be much nicer than these other formal systems. So since the 1980s/1990s the probabilistic approach to AI has become dominant (even if deep learning has displaced the actual models).

I'll first admit you likely know way more than I about this. So this is not an attack on your statements, but an ask for clarification.

>replacing true/false with probabilities does a great job of this ... people came up with all kinds of alternative systems to avoid dealing with probability

a probability is a number between 0 and 1, instead of either 0 or 1. a number between 0 and 1, instead of 0 and 1, is called fuzzy logic.

so I'm really lost about what you're saying here.

I believe fuzzy logic had its own axioms which aren't the axioms of propositional logic extended to probabilities. So the representations look similar but are manipulated differently.

For example, in Bayesian logic:

x AND y is xy, x OR y is x + y - xy

Whereas in (Zadeh) fuzzy logic:

x AND y is min(x, y), x OR y is max(x, y)

IIRC, “fuzzy logic” is actually a class that includes all generalizations of crisp binary logic to continuous values over [0,1] with operators meeting a set of definitions which basically boil down to “reduces to crisp logic when the input values are constrained to 0 and 1”, so that Bayesian logic is a fuzzy logic.

The Zadeh operators in particular I remember being constructed, or at keast rationalized, as ways to combine the degree of truth of propositions as distinct from the probability of truth of uncertain proposition. But I think interest in the kind of epistemic differences in alternative extensions to propositional logic faded with the lack of a practical need in terms of computational efficiency to avoid Bayesian probability (and I think there was also a separate philosophical battle and the side in favor of “Bayesianism is the only meaningful extension of propositional logic” was winning that battle when the computational problems were resolved, which helped sweep aside the alternatives.)

> For example, in Bayesian logic: x AND y is xy, x OR y is x + y - xy

No it isn't. P(x AND y) = P(x)P(y) only in the special case where x and y are independent. Unlike fuzzy logic, probabilistic logic is not truth functional.

> No it isn’t. P(x AND y) = P(x)P(y) only if x and y are independent.

You are obviously correct, and I shouldn’t post when I should be sleeping.

> Unlike fuzzy logic, probabilistic logic is not truth functional.

Its been a long time since I had much engagement with fuzzy logic, but I distinctly remember it being constructed as a class such that Bayesian probability was a fuzzy logic, though the others of interest were much simpler. But that may be as wrong as the other part of that post...

The 0..1 of fuzzy logic isn't the 0..1 of probability. When, in fuzzy logic, you say "X is true to degree 0.7" you are not saying "70% of the time, X is true" or "70% of samples display X"[0]. You are saying that the state you observe conforms to X by 70%. You can, at the same time, say "Y is true to degree 0.4". X and Y are allowed be somewhat contradictory: there's no need for their degrees to sum to 1.

[0] Not necessarily. You might choose to measure X that way, but it's not required.

You'll need to be more specific, because probability can be described the same way, numerically.

Fuzzy logic deals with degrees/scores, probability theory deals with how likely something is.

When you say that "The hotel room is 75% clean", that's a degree. It means a room that's not as clean as a "90% clean" hotel room, but definitely cleaner than a "50% clean" hotel room on some kind of cleanliness scale that you have. You need a scale because the boundary between clean and unclean is not sharp, but fuzzy. Fuzzy logic gives you tools to construct well-behaved scales for logical combinations of variables that already have scales associated with them. E.g. if you have a scale for a motorcycle being loud and a scale for a motorcycle being expensive (both fuzzy concepts), the tools of fuzzy logic can e.g. give you a scale for loud OR expensive.

In contrast, when you say that "The hotel room is clean with probability 75%", you're reasoning about how likely it is that the room is clean under uncertainty. Maybe the cleaners only work 3 out of 4 days, and you're unsure what day it is. But these are not degrees of cleanliness: if you say that a room has a 75% chance of being clean you're not claiming that it's cleaner than some other room that has a 50% chance of being clean.

The concepts involved in probability need not be fuzzy, or even measured on a scale. E.g. when one says "there's a 75% chance that the car repair will cost more than $50", there is no fuzziness involved in whether the repair costs more than $50 or not: in the end, you'll get a bill, and it will state a number that is unambiguously either above $50 or not above $50, a pure binary variable, no scales involved.

A simple example I once heard was,

Probability: “There’s a 70% chance the grass will be wet tomorrow morning”

Fuzzy logic: “The grass will be 70% wet tomorrow morning”

> so I'm really lost about what you're saying here.

Numeric representation may be the same[0], but the rules of how to do math on them, and what that math means, are different.

[0] - AFAIK dealing with Bayesian math, it's usually more useful to take a logarithm of all probabilities, so you can add them instead of multiplying. This translation expands the range from <0, 1> to <-infinity, +infinity>, and incidentally makes it clear why some people say that "zero and one are not probabilities".

`log(x)` does not give `+inf` on the range `(0,1)`, I think you're thinking of "log odds" aka a "logit" which is `log(x/(1-x))`

In addition to the other replies here, one fundamental difference between probabilistic and fuzzy logic is that fuzzy logic is truth functional and probabilistic logic isn't. Truth functional means, for instance, that if we know the (numerical) truth values of the propositions A and of B then we also know the (numerical) truth values of the propositions (A and B), (A or B), and so on. In probabilistic logic this does not hold. That is, P(A and B) is not fully determined by P(A) and P(B). If A and B are independent, we have P(A and B) = P(A)P(B), but in general we only know that P(A)+P(B)-1 <= P(A and B) <= min(P(A),P(B)). I also believe there's no generally accepted notion of conditioning in fuzzy logic, whereas conditioning is crucial in any probabilistic approach, see e.g. Bayes' theorem.

Truth functionality always struck me as a bodge in fuzzy logic, because you get to choose the implementation of (A and B) fairly arbitrarily. You're getting to choose some mix that lets you pretend the degree of dependence between the variables doesn't matter. It's a useful engineering hack, but a hack nonetheless.

Fuzzy logic uses very different rules.

It is common to saysthat the 'truthiness' of (a and b), is the minimum truthiness of a and b. Or becomes a maximum, not becomes 1 - truthiness. Then normal predicate logic works by combining these rules the usual way. E.g. (a implies b) becomes b or not a, which is max(b, 1-a).

Basyan probability is much more sophisticated, but also more difficult to calculate.

Beyond the already well explained aspects of truth-functionality, you're thinking of t-norm fuzzy logic (basically the only one actually still studied today). There were other kinds.

When was the reparametrization trick developed?

I've built digital systems based on Fuzzy Logic in the US sometime in the mid 1990s. Fuzzy was good because it allowed us to implement very complex decision making (outcomes, classification...) on very, very basic hardware. It was, in all forms, a simplification of a neural net which was much harder to implement hardware-wise.

Fuzzy didn't go very far because it could already be implemented back then with cheap processors (we had Moore's law!) and some slap-on programming. Or domain-specific logic hardwired by engineers who knew what the outcome should be given a bunch of inputs: think "balancing robot" or washing machine torque control.

Modern ML and powerful Arduinos make fuzzy logic irrelevant today, except that Markov chains should also be irrelevant but they aren't, so who knows, maybe fuzzy could stage a come back. Despite its huge limitations, FL certainly is better than current neural nets when it comes to transparency so that humans can better visualize what it's doing. Disclaimer: back in the day I built a "FL tracker" in Visual Basic so that you could visualize the full system trickle-down a decision in realtime and it was very enlightening. Today I'd love to build the same visualizaer but for LLM generation!

Why do you say Markov chains should be irrelevant? Outclassed by NN?

(positives:)

Fuzzy Logic has some utility in engineering applications, because it can be intuitive for humans interpret and to create rules using it. Interpretability is also the strong point of neuro-fuzzy logic (hybrid system). Applications are usually control systems, business logic, or embedded systems with tight resource constraints.

(negatives:)

In academic research it's not that interesting, because it's just one multi-valued logic among others, subset of infinite-valued logic. It's not very useful in machine learning because there are no efficient way to learn fuzzy rules automatically.

This is pretty much my recollection of learning it at university in the early 2000's. Learning rules automatically wasn't so much of a thing; people were far more interested in being able to efficiently get rules from expert humans into the systems without too many intermediate steps.

could you have LLMs generate LFLMs (large fuzzy logic models)? :D

Pratchett's "Wooly Thinking" was the 2nd version of the idea, much more popular, took over the world immediately after he coined the phrase.

Thruly, tho look at things like QAM encoding schemes and the tricks stepper motor controllers are doing. We have found ways to analog-ize a lot of things that were formerly viwed as totally digital; which i think is what the "fuzzy logic" hype boiled down to anyway.

Would you be able to supply a reference to some of the tricks of stepper motor controllers?

See "microstepping controllers" where they're presenting a digital control (steps) to varying levels of holding current in the actual motor. AFAIK its the trick that enabled 3d printing without position sensing.

Microstepping is adding virtual half-steps (or even further) to stepper motors. It "merely" enhances resolution and vibration characteristics of the motor. Microstepping itself is not strictly necessary for 3D printing without position feedback

Academic silo with little to no real transfer to business. ML eventually enabled building better continuous and discrete models for inference, control, and prediction.

I would have though the other way around: marketing buzzword for a non-problem solved by engineers since Maxwell's times by variations of applying the concept of hysteresis.

At least my recollection of fuzzy logic are from around the late eighties / early nineties and always involved the example of a thermostat that can only turn fully on or fully off. :)

It was also popular in automatic transmissions in the 1990s. These are always asking "Should I change gear? Up or down?"

and rice cookers. fuzzy logic was not speculative it founds its way into a fair number of real world control applications

The only place I've seen Fuzzy Logic advertised is on washing machines

True, this is the first thing I’m reminded when I hear this term. Then I wonder what Fuzzy logic has anything to do with washing machines. Modern washing machines has everything routine programmed and are deterministic

Do they? Mine noticeably takes much longer to run on very dirty kitchen towels compared to clothes, even on the same setting and even though the clothing load is heavier. I had been assuming there's something to detect how soiled the water is.

Random fact of the day: (some?) Dishwashers detect soiled water by using a resetable fuse on the water pump and counting how many times the fuse trips; dirty water makes the pump work harder and trip the fuse.

I don't think that would work for clothes washers though, because there isn't the same kind of pumping water for recirculation. Somewhat related, my washing machine (upright, high efficiency) measures the load by spinning it with a calibrated input impulse and measuring the rotation of the basket, less rotation means more clothes means more water. It's helpful to load the clothes around the edge to get the right amount of water. But that doesn't illuminate how cycle duration is determined.

Not all of them do, I've taken apart a Siemens dishwasher (pump was clogged) and it has a turbidity sensor to check if the water is clean or not.

Rice cookers are still marketed as fuzzy logic: https://yum-asia.com/us/product-category/rice-cookers/

I went to a Zadeh lecture, and he said that it was responsible for some big improvements in auto-focus cameras.

It's took the form of neural networks and deep learning.

humans like to invent many names for the same things

"Deep learning" as used today isn't fuzzy logic, it's actually multi-layered neural networks. (hence the "deep": because it has multiple layers of neural networks instead of a single one).

The reason why researchers on neural networks started to talk about "deep learning" in the 2000's is because neural networks had a bad press and articles were being rejected. Machine learning on the other hand was hot so it was a way to get their papers accepted.

logic is binary. yes or no. classic AI used pure logic via means of rules engine. you could tell exactly which path to take to solve a problem.

fuzzy logic is not binary, but multimodal/range. what maps unto that today? probability on top of layered NN, aka deep learning. It's no coincidence that no one knows how the deep learning models "think." They are by design very fuzzy.

There was a popular conception (like taught in textbooks even) that one hidden layer was enough since a neural network with one hidden layer could approximate any function. Deep learning became a thing when it was found to work great in practice.

Not really? Relu and softmax look a bit like the components of a fuzzy associative map if you squint a bit, but the lineage is distinct.

Exactly: writing out all colors of "maybe" explicitly in programming languages was infeasible, so english is the one programming language that can and will capture fuzzy logic, encoded in weights and connections.

Fuzzy logic always seemed misguided to me. You're just calculating values with real arithmetic instead of booleans. It's not a whole different kind of mathematics.

When I'm drawing on screen and placing a widget not on the left or the right, but at x=f*width, is that already fuzzy logic? The classical examples of Japanese washing machines determining detergent dose are not much more complicated.

Coming not from the background of IT but from electrical engineering and industrial control systems, like programmable logic controllers (PLC) and embedded control, I learned that 1990ish.

Fuzzy logic was in favor to the relative new field of neural nets (ML as we know it today). Because from the limitations in industrial control neural nets were too costly. Yes, we had already things like the first i386embedded. But that was way too expansive. Most of the PLCs were based on 16-bit microcontrollers and even those were more on the expansive side.

So if you think commercially in terms of industrial control systems, then Fuzzy Logic was implemented and used. I assume old systems can be found anywhere even today. Remember that in industrial control 20 years is no time.

Today, everybody in industrial control is also about ML. But here again, that is still on the expansive side. Such Nvidia Jetsons do not come for cheap.

I spent some time in small industrial controls and we never touched fuzzy logic, but we did a ton of stuff with PID algorithms. Maybe PID was just easier to implement?

It depends on whether your problem maps well to PID, or whether you need to adapt a lot of expert knowledge on a more complex control curve.

PID is alive and well in espresso machines. PID for temperature control is a marketed feature (of cheaper machines: expensive ones you just trust they know what they are doing and have designed it to work at the right temperature)

From what I recall from Fuzzy Logic in University, the main difference is that with PID you give it a setpoint and it reaches and maintains it - while with fuzzy logic you can encode essentially different strategies for different input ranges, something that quite possibly would in turn provide setpoints to PID controllers.

Although fuzzy logic was developed by Lotfi Zadeh in the USA, it never gained the level of interest and respectability here that it did in Japan and Eastern Europe. I remember American engineers saying that fuzzy techniques were things they already understood and were doing under different names. They saw no benefit in adopting a new conceptual system.

Among the more philosophically inclined, there was controversy other whether fuzzy logic was really something new or just a repackaging of old ideas with a misleading vocabulary. There was a book on how this debate played out in journal articles, but I can't think of the title now.

I believe Bart Kosko, in his book on Fuzzy logic, said to expect the division of use of FL techniques to break down on cultural lines - that western thought would resist using it when "more granular binary logic" and cpu speed would prevail.

IEEE Spectrum's profile on Lotfi Zadeh and the Birth of Fuzzy Logic. https://spectrum.ieee.org/lotfi-zadeh

Contrary to popular belief, perhaps we are not yet ready for fuzzy logic-based mathematics†. Like asking that does Boolean logic do in the 13th century, before Leibniz had any glimpse of characteristica universalis [1] (software, as opposed to calculus ratiocinator, hardware). If cryptocurrencies are solutions in search of a problem, fuzzy logic, and especially fuzzy dual numbers logic [2] [3] [4], doesn't even know it is a solution.

† one speculative use would be to assign degrees(?) of agency, some kind of a future unit of measurement: a person can be 100% evolved, or maybe 49% evolved and 51% designed (prosthetics, neural implants, and so on).

[1] https://en.wikipedia.org/wiki/Characteristica_universalis

[2] 2017, Fuzzy Dual Numbers: Theory and Applications, https://dl.acm.org/doi/book/10.5555/3172931

[3] "Fuzzy Dual Dynamic Programming", https://hal-enac.archives-ouvertes.fr/hal-01372543/document

[4] "Dual Numbers & Automatic Differentiation", https://blog.demofox.org/2014/12/30/dual-numbers-automatic-d...

There are two things that are referred to as "fuzzy logic".

The first, is the true fuzzy logic, which consists of Fuzzy Set theory and Fuzzy Measure theory. The two are distinct but interrelated, and heavily mathematical in nature. In particular, Fuzzy Measure theory completely subsumes Probability theory (i.e. probability theory is a strict subset of fuzzy measure theory, and therefore anything that can be expressed in probabilistic terms can be expressed in equivalent fuzzy measure theory terms, with leeway for added generality).

The second, is a class of techniques, based on Fuzzy Set / Fuzzy Measure theory, which implement some very simple behaviours which allow you to solve a particular class of problems. This is the standard Sugeno/Mamdani logic, fuzzy inference as the application of a few straightforward rules for conducting conjunction/disjunction among fuzzy rules.

The first is still thriving, but largely in the theoretical realm.

The second is shunned, on the basis that it feels like a bunch of interpolation heuristics, and a Bayesian approach is usually "better", even if not as intuitive of straightforward.

Unfortunately this has had a self-prophetic impact, in the sense that any literature which would have been better described as fuzzy, is labelled as probabilistic, even if strictly speaking the concepts involved are not strictly-speaking probabilities, but should more correctly have been addressed as fuzzy measures.

Personally I have published papers where I had to "disguise" fuzziness as probability just to get it considered for publication, or where I had to "apologize" for the term, or making it explicit that fuzziness in this context is unrelated to this set of tools that is typically thought of as "the fuzzy approach".

Having said that, one area that is thriving is the use of Fuzzy methods in the context of more general AI / NeuralNetwork methods, particularly in the context of explainability. Type 2 Fuzzy methods in particular (which introduce one or more layers/dimensions of fuzziness/uncertainty over the membership function itself) are quite an active area of research.

Finally, there are some frameworks that borrow heavily from fuzzy theory, but strictly speaking are independent. The one that interested me the most in recent years was "Subjective Logic", proposed by a guy called Audun Jøsang; this sounded like a very interesting logic framework and I was very keen to use it in my work, but in the end I just didn't have the time to justify that investment ... It's worth looking up though. The basic idea is that you have a framework where you have rules that have a built-in way of carrying uncertainty with them, with a one-to-one mapping with beta probabilities over your fuzzy-like rules, and a set of logical operations (which map to the usual logical operators) that are adapted to carry the uncertainty with them. This means that you can reduce your problem into a set of simple logical statements, and the framework would then take care of the uncertainty calculations for you seamlessly.

What would you recommend as the canonical work to read for the first thing you mention - fuzzy set theory & fuzzy measure theory? I've run into fuzzy set theory before, but not the more sophisticated elements you mention.

I don't know about canonical, but back when I was dealing with fuzziness I found this book to be very good: https://eresources.nlb.gov.sg/printheritage/detail/f41c8610-...

But, it's probably a bit out of date by now.

Also, if you'd like to learn more about Type-2 Fuzzy Systems, look up anything by Hani Hagras (disclaimer: colleague of mine from the same university, but highly regarded in this field)

thank you for the references!

It would be interesting to see how fuzzy and subjective logic evolves when powered by different numerical systems.

For example, surreal numbers are much more expressive than reals, and there are ways to express gaps as first order concepts (although not strictly a number) that maps very well to this space. I just skimmed through Audun Jøsang paper and the ideas of Belief/Disbelief to represent uncertainty resonates with Conway's construction of surreal numbers quite a bit.

Reminds me that I played around a tiny bit with Fril back in the day. Prolog with probabilities/uncertainties

https://en.wikipedia.org/wiki/Fril

It always struck me as an interesting way to get out of having to do a bunch of horrible control theory calculus and still get credit for it.

Should ask Bart Kosko

https://www.edge.org/3rd_culture/serpentine07/Kosko.html

lol, i did not know fuzzylogic was seperate field, thats what i call ml/ai . I also named one of my repo fuzzylogic [0].

[0]: https://github.com/temphia/fuzzylogic

yes, washing machines...you put your dirty clothes in and just press start...the fuzzy logic works out the best program for it

Also, higher end rice cookers.

the washing machine with it was made in the 1990s

I'm not convinced they use fuzzy logic.

Is it fuzzy logic because there is presumably hysteresis in the water heating phase?

Or is it fuzzy in the way to converts turbidity of water to wash times?

They generally use fuzzy logic to better drive various components with minimal BOM. You can easily encode a bunch of rules from expert on best decisions given certain conditions (for example for motor control for best efficiency, or heating, etc) and use fuzzy logic to switch between modes smoothly.

I remember reading about dedicated fuzzy logic circuits which essentially gave you simple fuzzy state machine programmed with cheap ROM, and it was all you needed instead of MCU for many tasks.

In a way cheap powerful MCU are biggest killers of the technique.

When I used FL during uni mid-90s it was used to optimize revolutions and torque for shifting loads. Another often described use case was a loaded cat on the arm of a crane, that moves along the arm without getting the load to swing.

Fuzziness means illogic ?

No.

There is a particular logic (“Classical Logic”) that people are usually thinking of when they mention logic. Many folks don’t know that there are many, many logics that have been invented by mathematicians and philosophers, and an infinity of possible logics. All are as mathematically sound as each other. They vary in how truth behaves (SQL’s true-false-null is a poor attempt at a 3-valued logic, for example; or you might include notions like necessity and possibility or time).

Fuzzy logic is another logic. It’s quite precise about how truth behaves, and truth is a real number on the range [0, 1]. And it does a nice job of providing a notion of partial truth.

To give an example: you can define "tall" for an adult human as being 0 if 150cm, 1 if 210cm, 0.7 if 190cm, etc.

Unlike boolean logic where you're either "tall" or "not tall".

I've learned fuzzy logic during my graduate control theory study, but never get what the point of it, at least in practice.

Using your example, if the control system can utilize non-binary, numerical input and do something with it in actuators, then just give it the actual height (in cm). If it can't, then introducing the concept of "semi-tall" isn't going to be helpful.

The example we learned in class is the classic thermostat of an air conditioner. Sure, it would be nice if it can tell between cold, warm and hot instead of a binary 0 and 1, but why not just use the actual numeral value (0.1, 0.22, etc.) then?

The most important thing it lets you do is combine and compare factors in nonlinear but intuitive ways.

The aircon example isn't really that great, to be honest, because there's only one rule and only one action you can take. If you add in "how full is the room" (and letting that be tuned for room size), an automated window vent, the outside temperature, and some sort of air quality reading, you can see how the job of making a room tolerable to be in starts to combine factors in interesting ways. https://en.wikipedia.org/wiki/Fuzzy_associative_matrix and https://en.wikipedia.org/wiki/Defuzzification are things I'm sure your course would have covered, but that's where I'd point at as a start.

Because when Fuzzy Logic for control was trying to gain popularity, processors were far more expensive. FL had the promise of being able to do complex control with really cheap hardware.

Reply on news.ycombinator.com