Your exactly right. Once you unsubscribe from the default subreddits and start subscribing to subreddits that have more moderation (against memes, etc) reddit becomes a much better place. You end of with subreddit's like AskScience where you can have though provoking conversations without memes.

That's what makes reddit great, if your not into the memes, you can easily "turn them off".