Not quite. A borrow doesn't do cleanup because it's, well, a borrow, i.e. a temporary reference to some owned value. It's the owned value that's tasked with disposing of any resources/memory when it goes out of scope, just as in Rust.
The cost of borrowing (at least with heap types) is an increment upon creating the borrow, and a decrement upon disposing of the borrow. Over time I'd like to optimize those away, but that's not implemented at this time.