Yes, you have to poll CodePipeline every second or so to get the jobs and perform your task. Currently, web hooks are not implemented or they are WIP.