I'm really focusing less on the idea that Dict the data type with it's associated methods is like a function, and more on the idea that a dictionary in the general sense is a mapping of input values to output values, and you can think of functions that way.

That said, there are some pretty reasonable analogies to be made between common dictionary operations and functions.

For example, adding and removing items can be done with function composition so long as you are okay with partial lookups. Here's a really short example I put together:

  module Example where
  import Control.Applicative

  type Dict a b = Eq a => a -> Maybe b

  emptyDict :: Dict a b
  emptyDict = const Nothing

  singleton :: a -> b -> Dict a b
  singleton k v target
    | k == target = Just v
    | otherwise = Nothing

  unionDict :: Dict a b -> Dict a b -> Dict a b
  unionDict dict1 dict2 k = dict1 k <|> dict2 k

  insertDict :: a -> b -> Dict a b -> Dict a b
  insertDict k v dict = singleton k v `unionDict` dict

  removeDict :: a -> Dict a b -> Dict a b
  removeDict k dict target
    | k == target = Nothing
    | otherwise = dict k

This particular representation of dictionaries isn't necessarily something you'd really want to do, but the general approach can be quite useful when you start working with something like GADTs and you end up with things like:

  data Smaller a where
    SmallerInt :: Smaller Int
    SmallerBool :: Smaller Bool
  
  data Larger a where
    LargerInt :: Larger Int
    LargerBool :: Larger Bool
    LargerString :: Larger String
  
  someLarger :: Larger x -> x
  someLarger l =
    case l of
      LargerInt -> 5
      LargerBool -> True
      LargerString -> "foo"
  
  embedLarger ::
    (forall x. Larger x -> Smaller x) ->
    (forall smallerI. Smaller smallerI -> r) ->
    (forall largerI. Larger largerI) -> r
  embedLarger mapping fromSmaller larger = fromSmaller (mapping larger)

(I'm actually co-authoring a talk for zurihac this year on this pattern, so I have quite a bit more to say on it, but probably not ideal to cram all of that into this comment).