I've messed around with Duolingo a few times, just to get a feel for how useful it is. I came away feeling that it was about the same level as flashcard vocabulary learning - useful, but only a small part of the pie.
Vocabulary is a necessary, but not sufficient, condition to be able to carry on a conversation. Mandarin in particular is more difficult, since you need to grind through hours and hour writing/rote learning characters to improve your vocabulary.
The one thing that noticeably accelerated my proficiency was having an actual conversation with a native speaker (host family/tutor/etc). That's not something that Duolingo really replicates.
Self-plug - coincidentally, earlier today I just released a small tool for Mandarin learners to practise their tone recognition! I've been working on a larger app and this seemed like something that would have helped me when I first started out. App is currently live @ https://fluent.academy - I'll be introducing more functionality for Mandarin learners in the near future, so would really love to hear from students of any level.