Is that memory increase seen in peak usage or average/idle usage?
For file servers, I could be willing to give up 350 MB of memory temporarily for large-scale indexing and thumbnailing operations if it doesn't happen often and makes the file serving/browsing more responsive.