So then, what is the one liner to preserve the filename rather than get a new deduped.txt?
Also how do you apply that command to the next file using shell history?
Also how do you apply that command to the next file using shell history?
awk '!n[$0]++' fileName | sponge fileName dedupe.awk <file >file.tmp && mv file.tmp file
Multifile versions vary, I'd prefer listing them out, alternatively you could read from a command output (ls, find, etc.) with a 'while read; do ... done' loop: for f in file1 file2 file3
do
dedupe.awk <$f > ${f}.tmp && mv ${f}.tmp $f
done
If you want to apply to specific files on an ad hoc basis, you could wrap the whole thing in a shell function with filename or list as a parameter.Or 'gawk -i' as suggested.
Properly using tempfile would also be an improvement.