The big fat joke is thinking that agile are the processes and methodologies developed from the manifesto.

Read the manifesto, apply some techniques, methodologies, frameworks that you hear about but always keep the manifesto motto of "constant communication and improvement of the process" in mind. You and your team should try different approaches and learn from them on what is working for you while keeping: 1. Communication 2. Constant feedback and improvement 3. Knowledge sharing

If only 20% of the people can contribute you have a problem with your team, not with the agile process to be honest.