If I remember correctly its also not so difficult to take an arbitrary zip file, append some noise so that it becomes prime (and will still decode). So even illegal primes are not difficult.
It's not considered a "number" when society thinks of it in a higher conceptual abstraction than a number. It may sound like tautology and circular reasoning but that's basically it.
A 16-byte number is a range of 0 to 256^16. Can some value within that small range be illegal? Maybe if someone doxxed somebody with GPS coordinates of their house and embedded it in those 16 bytes, they might run in to trouble with the law. Examples: https://www.google.com/search?q=laws+illegal+doxxing
For larger examples... a 200k jpeg of a child in a porn scene is just a number less than 256^200000. A 2 gigabyte mp4 video file of a copyrighted Marvel Avengers movie is just a number less than 256^2000000000. Can those accused of a crime convince the following groups in society that those are just very large numbers in a mathematical sense:
- police arresting you
- jury convicting you
- judge sentencing you to prison
- employers rejecting your job application after you've served prison time
Which just shows that "number" itself is a conceptual abstraction, no higher or lower than the others.
Totally subjective and differs per person and doesn't have a definitive answer.