I am learning these languages for mind-expansion.
I am also studying AI, will learning Lisp be relevant? (e.g. Help me reason about and write AI systems)
Lisp is good to learn, but because it has "won" in a lot of ways I'm not convinced it's as important as it used to be. Lisp acquired its reputation in a world where a C programmer would pick up Lisp and holy shit dynamic typing, first-class functions, garbage collection, REPL, macros, powerful syntax-aware editors (which C got, but later), atoms, recursion, violent foaming at the mouth and falling down in sheer awe. If it's still your first language after C, sure. But if you learned, say, Python in school, the list comes back to "macros, which you really shouldn't make much use of in the more modern understanding, and a bigger focus on recursion". Less likely to make you pass out in awe.
(Since I find I sometimes piss people off by claiming Lisp is less than the bee's knees, remember, it's because Lisp won in a lot of ways. It blazed a trail that every modern language today has traveled to some extent or other.)
Haskell still has a counter-cultural element because of the fact that Lisp is quite dominant right now and most languages are about empowering the programmer to do anything at any time. The restrictions-based languages are just starting an ascent (currently probably being led in practice by Rust, I think, if it's not already bigger than Haskell it probably will be in less than a year), but it's still pretty early.
Best, briefest way I've seen it described. I think that's how I'll start describing it, too.
On the one hand, it doesn't seem to me to be strictly necessary to be "homoiconic" for macros to work. On the other hand, it seems like non-homoiconic languages always end up mitigating the power somehow. Maybe it's just correlation rather than causation, but unless you're planning to write a new language that doesn't matter much in practice.
Well, only if you're stuck with a broken macro system (cough cough Common Lisp cough cough). Here in scheme, we have er-macros and sc-macros, as well as syntax-rules. Hygene matters, and procedural macros rock, although you want declarative for the simple things. pg really made the wrong call on that, and it will probably haunt arc for years to come.
I haven't got a horse in this race.
maclisp -> Common Lisp
-> Eulisp (Eulisp took a lot of inspiration from Common Lisp)
-> Common Lisp
Scheme -> Racket
-> Common Lisp
This compatibility does not exist anymore. Interlisp is gone. Scheme is now its own language universe. PLT Scheme was renamed to Racket to prevent users from thinking Racket cares about compatibility with Scheme.
What was once a language group with dialects of a core language, is now not more than a bunch of ideas, where independent language camps pick/mix freely whatever they find important. For all practical work (!) they are now fully incompatible. Some languages still have a common core (Common Lisp, Emacs Lisp, ISLisp, Visual Lisp), while others have new cores: Scheme and Clojure are cores of new languages, with their own syntax, semantics and pragmatics. Neither of those cares about any form of compatibility or code sharing anymore.
No, PLT Scheme was renamed to Racket to reflect the fact that the base language does not (and had not even prior to the rename) really target compatibility with any of the scheme standards, though using its module language approach, Racket includes both R5RS and R6RS Scheme implementations, so its not at all the case that Racket doesn't care about Scheme compatibility, its just bigger than just that.
Racket, as an environment, has compatibility due to its module/language approach to a bunch of languages. The older Scheme R5RS / R6RS are amongst those.
> Racket is a full-spectrum programming language. It goes beyond Lisp and Scheme with dialects that support objects, types, laziness, and more. Racket enables programmers to link components written in different dialects, and it empowers programmers to create new, project-specific dialects. Racket's libraries support applications from web servers and databases to GUIs and charts.
The main language under development aren't those, it's Racket:
http://racket-lang.org/new-name.html
> They draw you in with the promise of a simple and polite little Scheme, but soon you'll find yourself using modules, contracts, keyword arguments, classes, static types, and even curly braces.
> programmers can now simply say “Racket” to refer to the specific descendant of Scheme that powers PLT's languages and libraries.
There is a specific new language, a descendant of Scheme, in which this stuff is written, and that is called Racket.
This Racket language and its main libraries is documented here:
https://docs.racket-lang.org/reference/index.html
See also:
> Shriram Krishnamurthi
> Racket is Racket. It's its own language.
> I'm not quite sure what "Lisp family" really means, given that even Common Lispers and Schemers seem to argue more than agree (maybe because they're close family members?). Given that Sussman argues that Scheme is also a descendent of Algol, doesn't that put Racket in the Algol family as well? So Racket is both an Algol and a Lisp; would you change your posting to say “picks Algol to teach son programming”? In short, do these labels tell us anything useful at all?
It really depends which Lisp you are talking about.
If this is true then why do I find Clojure so many orders of magnitude more powerful, more reasonable and more pragmatic than Python. I've done Python for a decade, but now days I would choose Clojure over it every single time.
Clojure over Python is so very much more than just macros. The default data structures are far superior, the time abstractions are fantastic, the concurrent and parallel programming features are better, the asynchronous support is miles ahead and it has parts you can use that just don't exist at all in Python like core.logic and core.match.
I also think the whole is greater than the sum of its parts. Yes, a lot of this stuff exists in Python, but not in the same elegant and idomatic way. In Python the onus is on you to get it right. And one rarely does. Clojure however steers you in the right direction, making it idomatic. When your code is right, it looks right. And when your code is wrong, it looks wrong.
Later on you can utilize your knowledge of Haskell and Prolog by studying Curry, which (sort of) marries the two together.
Depending on the area of AI you choose, Lisp can be a good choice but Prolog can be as well. The class I took focused on supervised learning with support vector machines, decision trees, and neural networks. These topics use fairly advanced mathematical concepts, so the language of choice is one in which libraries exist already. We used R, which the professor uses for his research. Python is also strong with its library support.
It's a good idea for you to narrow the area of AI you want to study and then find out the language people working in that area use.
If you don't have the cash for the very expensive softcover, you can grab the unoficial epub, which is a bit nicer than the online edition.
>Also pretty dry.
yep. I still haven't gotten all the way through the first section. It's a tough read.
Major nitpick: declarative thinking becomes the dominant way of thinking not pattern matching. When you write prolog declaratively (w/o cuts at the expense of efficiency) your code can answer multiple questions/queries.
With respect to answering "multiple questions/queries", is it fair to draw a parallel with SQL?
Simpsons = ['bart', 'lisa', 'homer'], member(X, Simpsons)?
will do a for each over the list, with X bound to each character.
and the same member definition can also be used to answer Simpsons = ['bart', 'lisa', 'homer'], member('brian griffin', Simpsons)?
Prolog and Lisp are interesting if you're into programming language design, as examples of their respective paradigms. They both used to be the "languages of AI" back in the day (as in ten or so years ago... ) before the modern wave of machine learning. Most Prologs for instance don't have any machine learning or even tensor manipulation libraries to speak of [1], so if you want to learn about machine learning algorithms you have to do it yourself from scratch. Which can be a great way to learn of course.
One very big exception to the above: if you want to learn about the original AI project, before the most recent machine learning wave, then to an extent you can't avoid Prolog or Lisp. Many of the GOFAI textbooks are also Lisp and Prolog textbooks (or the other way around) so by reading about the one you necessarily learn about the other.
One excellent such resource is George Luger's book, "AI Algorithms, Data Structures, and Idioms in Prolog, LISP, and Java":
http://wps.aw.com/wps/media/objects/5771/5909832/PDF/Luger_0...
______
[1] Though all it takes is someone with a bit of time in their hands. Watch this space.
More like machine learning is about machine learning. Remember that AI has always been a bunch of subfields. One or more are extremely popular at any given point. There's still usage of rule-based, agent-based, GA, NN, lots of heuristic planners, and this number-crunching stuff called ML. The old books I started with on it taught LISP, Prolog, Poplog, etc. These days, given all I've seen, I'm strongly against Prolog for real-world AI as the best approaches use a mix of techniques with often imprecise "facts" in their "heads." Watson architecture is a good example, although I think they still use Prolog in part of it.
Best approach, born out of old days, is to use a LISP for the AI but build-in Prolog, Standard ML, or others as DSL's for specific problems. Allegro CL, Racket, and sklogic's tool all do that. Gives you max flexibility plus ability to express solution in language closest to your mental model of the problem.
"Imprecise facts"? What do you mean?
Yes, Watson uses Prolog- for its pattern matching (unification, ie. Turing-complete pattern matching). They store facts retrieved from shallow parses over raw text as Prolog predicates, then let Prolog do its thing when it's time to find an answer.
>> More like machine learning is about machine learning.
I'm just finishing a Masters course at the University of Sussex. The subject is AI and most of the curriculum was machine learning. Machine learning for NLP, machine learning for Image Processing, Machine learning straight up, Neural Networks and so on, so forth.
In the UK I believe you'll find the same in any university. Also, if you keep up with latest research again it's all about employing statistical algorithms to build models of the data... in other words, machine learning. From the papers I've studied during my course that's been going on for a couple of decades at least now. For instance, if you pick up one of the staple NLP textbooks (Jurafsky and Martin, Manning and Schütze, Charniak) there's not a single line of Prolog or Lisp in them, or indeed any use for first order logic except to discuss its possible applications in representing meaning, but then nobody does meaning in NLP (because it's bloody hard).
Also, I should point out that Sussex in particular was one of the centres in Europe where logic programming was developed in the first place. They had the Poplog II suite and it's the alma matter of Christopher Mellish (of Clocksin & Mellish, from "Programming in Prolog"). There was one module in my course that had a bit of logic in it (one lecture for propositional logic, one for predicate logic, one for Bayes I think it went) but there was absolutely no Prolog to be found anywhere whatsoever.
So it's a big deal that Sussex seems to have completely given up on it. There's probably a bit of a backlash effect in that they rammed it down the throats of their students for a long time (I've talked with several Sussex alumni who have commented on that, and they don't have happy memories of Prolog). Even so though I think it's indicative of the attitude in the field in general.
I'm not sure what's the situation with Lisp.
But, really- AI is now machine learning. Nobody is willing to try anything else.
That most AI problems deal with uncertainty that's harder to deal with in first-order logic. You gave a good example with NLP where the early stuff I looked at tried logic approaches to find they fell flat due to context & language's probabilistic nature. All kinds of things in game AI turned out that way. Even some stuff that would seem true or false, like "am I being attacked?", wasn't so clear when bluffing was involved. I couldn't imagine how to handle a Poker game in Prolog with any success. That's significant given a combo of math, human BS, and luck it brings is common in many AI problems. For these reasons, people were moving from first-order logic even in my day to fuzzy logic, probabilistic models, neural networks, and machine learning techniques.
"The subject is AI and most of the curriculum was machine learning."
Now, remember that I said one or more subfields (or trends more accurrately) are usually very popular in AI. Anything now considered machine learning seems to be it these days. One other thing to factor in is the old adage in AI that any AI technique becoming very popular or effective is no longer called AI. So, as get up to date here, we have to consider that machine learning or AI labels might have changed in how they're applied + AI stuff getting non-AI names.
"Expert systems" were a huge thing back then but today they just call them business logic, business process management, workflow, whatever. Usually lots of If-Thens or Case Statements. Decision-making was AI but now the field is called planning, solving, optimization. Methods were largely search with heuristics and/or constraint solvers. Winners in timetabling at least were the same type last I checked. Game pathfinding another example. Automatic programming was a combo of high-level descriptions (eg 4GL), templates, synthesis, and so on. Still is although they call them IDE's with code generators now. Computer vision and pattern matching were definitely machine learning with statistical models and such. NN's were really different from most things in how they worked, have more resources now, and I usually hear deep learning but maybe they're called machine learning now. Chatterbots were highest performers in conversational style with many methods. Machine-to-machine and some to human communication was (and mostly still is) done with finite-state machines. Hardware synthesis tried all kinds of things but geometic methods with search + heuristics wins to this day. Knowledge storage, querying, and so on mostly went into database tech for best results & performance due to easier model and machine implementation. Some still did logic languages (eg Prolog, Datalog), encoding in neural nets, lots doing RDF/OWL/whatever, and so on.
So, that's off the top of my head different areas I learned about with techniques and new labels that appeared. Have any of these that weren't using machine learning techniques gone mostly into that in academia and/or commercial implement that you're aware of? While we're at it, do you have a link that lists the best AI labs so I can check on that and do a general update of my knowledge later on?
"So it's a big deal that Sussex seems to have completely given up on it. "
Normally I'd think it was chasing fads given the bandwagon effect that deep learning in particular is having. This time, though, I saw enough logic programming experts give up on it to know either the hardware or the approach itself isn't up to most AI challenges. Time showed it was probably approach itself. So, I agree Sussex moving on is very significant.
"But, really- AI is now machine learning. Nobody is willing to try anything else."
Non-ML is dominating in terms of highest performers in planning, synthesis, and databases. Those I know that are getting tons of research. They just don't call it AI anymore despite it being artificial tools replacing intelligence of humans that do such work. Genetic Programming's "Humies" awards probably deserve some mention given it's not really traditional AI or what you'd think of in machine learning. It's its own thing. There could be others. I agree that machine and deep learning are the big things getting vast majority of effort... damn near drowning everything else out... but others are still there. I already mentioned that name changes but I think it also depends on your university. I noticed in the past a lot and now a little that specific universities get big on specific approaches in general or subfields. Yours might have went all in on the trend more than another. A survey would be nice at this point that covered the activities I mentioned above that were called AI in the past.
"I'm not sure what's the situation with Lisp."
Me either. For AI and other research, most academics write tools with whatever they like best or others are using. For production, they usually code it in a common language for that like C++, Java, etc. One planner for timetabling college exams I recall was designed/prototyped in LISP then coded to C++ for release. The point of bringing up LISP is that DSL's, either executing or generating for specific languages, let you do every part of something as diverse as Watson in the same, common language with benefits of others selectively. For a Prolog example, you might parse in data with FSM's in C generated from LISP DSL, do simple filtering/preprocessing with safe 3GL, handle queries with Prolog style, and do user interaction with event-driven model. All integrated into same data structures, function calls, whatever through common language that's easy to parse/transform. Most apps don't need such power but a full AI might find it useful. Many machine learning techniques can just use a 3GL, though.
Unrelated, how far along are you on porting your prior work to open-source Prolog?
Modern AI is more of a numerical-statistical steamroller, where you're better off learning numpy or R and the like.
Yes?
For mind expansion, I'd probably add J. For Lisp specifically, I'd look at Racket over SICP because Racket is an ecosystem where a lot of interesting new work is happening and SICP is not really a book on Lisp programming (e.g. it does not address macros (the book on Lisp macros is Graham's On Lisp)).
The current main stream of AI is built on statistics and machine learning. It's psychological basis is behaviorism. Lisp (and Prolog) are more likely to be applied in symboloic (psychologically Jungian?/Freudian?/Lacanian?/etc?) approaches to AI. There are classic AI text books that use Common Lisp, and it's also an interesting language (there's a Rails like convention over configuration embedded in it).
Just don't box yourself into the Lisp best practices. Those are great for creating successful projects, but not for learning what programming can be. (Nowadays I think this advice will extend to Haskell too, but I'm still not certain.)
I don't think it will be relevant for any modern AI (ditto for Prolog). But knowing it does change the way you program - more yet if you are in some "domain specific language prone" environment, like Haskell.
I'll also agree with weavie in that you'll probably won't ever be "done" learning Haskell, nor taking lessons from Lisp. I don't think anybody has ever been.
Lisp or Scheme are good languages to learn, and I would reccomend them
Smalltalk is perhaps the only "real" object oriented language. It's a fantastic language to learn, and in some ways very similar to Lisp. While you're at it, learn CLOS or its cousins in Schemeland, which have some similarities. I've heard that some OO fans swear by them.
Prolog is the furthest from conventional programming of the three, and I'm not very familiar with it. I'd definitely reccomend learning it, though.
Racket also has a SICP language should one want to experience the original text without running into broken parts.
I'm not an expert on lisp and AI. From what I understand lisp has been the traditional go-to language for AI research, and for "classic" AI you probably cannot go wrong with learning at least the basics of lisp. However, a lot of recent AI research seems to be on neural networks, deep learning, and statistics. I don't know how important lisp is in those fields.
Prolog forces you to solve your problem with logic programming. You cannot take the easy way out and solve it (wholly or partially) in some other way that you already know. Likewise, Smalltalk forces you to solve the problem with OOP. Haskell forces you to deal with pure functions and lazy evaluation. APL forces you to solve the problem with vector operations.
Some of the approaches and solutions may be clumsy and worse than what you can do with a mix or some other programming paradigm. But I thought the point of the exercise was to learn the different programming paradigms, and in that context I prefer programming languages that do not offer alternative ways of solving the problem. I prefer a more restrictive approach and lisp is anything but restrictive (CLOS, kanren, etc).
Also I find that a different surface syntax helps me context switch to a different programming paradigm. For example, when I see s-expressions my mind switches to scheme and thinks accordingly, and that would be a distraction to me if I wanted to program imperatively, for example. When I see Prolog, I have a hard time thinking in terms of OOP. Maybe that's just me.
The reason for suggesting Smalltalk first was that it's farther away from Haskell than Prolog (Prolog clauses can appear similar to functional programming). Switching to Prolog after Smalltalk is another big jump. I like big jumps. They help me switch context. The OP may be different, I don't know.
$ lispworks
LispWorks(R): The Common Lisp Programming Environment
Copyright (C) 1987-2014 LispWorks Ltd. All rights reserved.
Version 7.0.0
CL-USER 1 > (require "prolog")
T
CL-USER 2 > (in-package "CP-USER")
#<The CP-USER package, 0/16 internal, 0/16 external>
CP-USER 3 > (erqp)
?- consult(appenda).
YES.
OK.
?- appenda([1,2,3],[14,15,16],L1).
L1 = [1,2,3,14,15,16]
OK.