Again, I don't have much experience with Scheme; I've read about half of SICP. So that's a huge caveat here.
It may have been misleading to say that Clojure was immutable by default. It is immutable; it does not have set! or the moral equivalent. You can use concurrency primitives to update in place (sort of? AFAICT you're mutating the var itself, not the data it contains) but each primitive comes with its own semantics.
As far as first-class is concerned: in Clojure, maps, sets, and vectors are functions. They're functions of their data as well as being data, and there are literal representations for each.
Examples:
user=> ({:foo 1} :foo) ;; maps are functions
1
user=> (:foo {:foo 1}) ;; keywords are functions, too
1
user=> (filter :name [{:name "foo"} {:surname "baz"}]) ;; filter by key
({:name "foo"})
user=> ([\a \b \c] 2) ;; access vector by index
\c
user=> (#{1 2 3} 4) ;; test set membership
nil
user=> user=> (filter #{2 4 6} (range 1 10)) ;; filter by set membership
(2 4 6)