Yes, the later chapters are tough. But I'd definitely say you don't need much preliminary discrete math in the first couple chapters. Let's say up to chapter 6 (Special Numbers)