Sam has stated for many years that Marvin is not allowed into his marketplace unless by written invitation. Marvin, being a citizen of his own town, can seek a marketplace in his own town, work with others to create one for himself like Sam did, or violate Sams borders knowing the consequences (as Sam has already told him what will happen). Just like standing at the bottom of the ocean will drown you, Sam has stated the consequences of violating his border. Sam detaining Marvin from Sams marketplace isn't violating Marvin's rights any more the sea denies your right to breath if you stand beneath it--Marvin is responsible for the risks he takes when the consequences are known. If Sam was Sam Jung Il, you may be shot dead. If it's uncle Sam, you can expect to be returned to your town.

Edited because I was distracted by children.