It's better to say you want to speak with your lawyer before you will answer questions. The fact you remained silent after being asked a question can be used against you in certain situations, especially with a non-custodial questioning. (the recent 2013 Supreme Court decision, Salinas v. Texas, has more on this)