Sounds like you are planning to break my browser's back functionality. If I were evaluating your site, you'd need to be pretty compelling to get to do that more than once.

There is a huge visual clue to the user that they are doing a "back" instead of a scroll (the whole page moves off to reveal the previous page behind it). It can be done accidentally, but a user will know what they have done and return. (In the static case, you have to move the page at least a quarter of the way out of the window. If you do a dynamic release it looks like less depending on how fast you are going. A "flick" could get you an accidental "back".)

Without looking at your site, I'll suggest that the bigger issue is that either "back"-"forward" isn't a no-op, or you leave a string of strange things in the back list. Both are common and both annoy users.