Digitize MiniDV Tapes with Linux | Hacker News Reader

Digitize MiniDV Tapes with Linux

arsouyes.org

67 pointsarsouyes37 comments

Once the tapes have been digitized, the excellent and open source DV Analyzer tool [1] can be used to inspect the digitized file and produce a report indicating sections where there may be issues with the video and / or audio of the tape.

I digitized a large number of family MiniDV tapes a few years ago, since the quality of the footage on the original tapes is better than the DVD recordings made at the time (due to both the DVD MPEG2 recording scheme and the fact that the camera was connected to the DVD recorder over composite video). But some sections of a few of the tapes had suffered serious deterioration, rendering the resulting footage unwatchable.

Using DV Analyser was a great and quick way to identify the clips with a higher than expected number of errors, so I could try to re-grab those tapes, and (if that failed) go back to the DVD recordings. Much quicker than watching every video through in real-time!

There's also the DV Rescue tool [2] (also open source), which seems to actively assist in fixing detected errors, but I haven't got any experience with that.

[1] https://mediaarea.net/DVAnalyzer

[2] https://mediaarea.net/DVRescue

Thank you, this is very helpful.

I have a hundred or so MiniDV tapes awaiting ingestion...first need to figure out where to store the data.

Something I learned from floppies is that different drives (or decks, in this case) have different read characteristics, so the "bad" segments of tape are different depending on what you're playing it back with. I once was able to use that random fact to restore some data off an old floppy - it took three drives/PCs, but I was able to get a clean read of each block, and piece them together.

Extending that to DV tapes: the DV format is interesting in that every frame is a keyframe, and encodes an entire image. That's in contrast to H.264 and friends, where frames can be predicted, and/or just the difference between a prior and future frame can be encoded. DV also encodes a timecode into each frame. So if you had multiple DV decks and were able to get a bunch of unclean imports, it seems like it'd be trivial to write a small tool that grabs all of the clean frames from each and stitches them together over the gaps in others, according to that timecode.

(Disclaimer: I've never done that, this is a pure spitball.)

This is really interesting and not something I was aware of. (Although now the fact that an hour of MiniDV footage is 12GB makes more sense!).

I wonder if this is what DV Rescue (linked in my above post) is doing? If it's not doing this, it's certainly something that it should be doing.

You're also 100% spot on with the idea of using different drives to recover "tricky" tapes - I have two MiniDV camcorders exactly for that purpose. But I was doing it the "long" way of trying a tape in one, and if it didn't work well, trying it in the other. It didn't occur to me that I could try combining the results of both.

DVRescue sounds promising — thank you. From https://github.com/mipops/dvrescue

> dvmerge A script that takes multiple transfers of the same tape containing errors and combines them to create one file with the best information available for each problematic frame. dvmerge is part of dvrescue. See dvrescue -h on a recent build.

> dvplay A script that plays back and visualizes the DV errors as a stack of images. Running with the -x flag will produce JPEGs instead of just playing them.

> Snapshot daily builds are at https://mediaarea.net/download/snapshots/binary/dvrescue/.

> Digitizing a MiniDV cassette is therefore a misnomer. We should rather talk about data transfer.

Yeah, I was about to say it’s kind of strange to refer to this as digitization, since MiniDV holds digital data on it already.

I think transferring MiniDV tapes with Linux would be a better title. But I kind of get what they mean I guess.

A while back I posted a picture to social media of the CDs I was listening to while cleaning. A friend responded "Don't you think it's time you digitize your music?" to which I could only say "...but CDs are digital..."

'digitize'? we called it ripping. Not even Boomers used the term digitize.

This was a Xennial (born in '81).

"File-ize" would be accurate. DV tapes are streams of keyframes and are not computer files. CDs are tracks and subcode data and are not files. Importing these data media into a computer usually ends with saving a file, which you can store on HDD, SSD, CD (with a file system and not as audio), flash card, etc.

As a sibling comment mentioned the verb "to rip" is the colloquial term I'm familiar with.

This winter I transferred a ton of VHS tapes and a bunch of MiniDV tapes. The process is vastly different (obviously analogue vs digital). VHS was a lot more fun with the super cool deinterlacing/FPS boost stuff you can do with AviSynth.

For MiniDV I ended up using iMovie on my Macbook Pro and it worked surprisingly well. It automatically imported and spliced the different "cuts" of a single tape into different files. It even timestamped the videos with what the camera thought the time was (spoilers, tons of the times were clearly wrong).

I found a guide online somewhere, but I'd like to present the dongle beast I used between my Macbook and my camera: https://i.imgur.com/wZ9rJnH.png

Thunderbolt 3 -> Thunderbolt 2 -> Firewire

I just handed off my MiniDV hardware setup to a friend to try out on Windows for some of his tapes. What a blast!

Did something similar on my Mac. Dongle for Thunderbolt 2 to Firewire and then got the raw .mov files via iMovie. De-interlacing and encoding with ffmpeg. For a raw file with a size of say 600 MB you can get down to something like 30 MB with h264, h265 can reduce this further to perhaps 15 MB. All of a sudden those old memories will be easy to keep and share :)

For anyone trying to transfer MiniDV tapes on Windows, I recommend the super-lightweight freeware WinDV: http://windv.mourek.cz/

For some reason it worked a lot better for me than the commercial video capture software I tried.

Depending on how you feel about the quality, cheap ($12) chinese dongles exist that take composite RCA and plug into USB and present as a standard windows compatible webcam type interface. Through this, you can just pump it into OBS studio and save it however you want. This is how I saved 30 hours of VHS tapes The actual hard part is choosing compression settings that give you good quality without eating through gigabytes of disk space per hour.

Technology Connections on youtube provides a much more complicated and expensive setup, but it might be a hair higher quality.

These systems will ingest anything that spits out composite video; Videogame consoles, VHS, DVD, old computers, TV set top boxes etc

That would be a poor choice in this case.

The data on the MiniDV tape is already digital. Any FireWire interface (cheap dongles also available) will allow most operating systems an easy way to read the tape, and as well as retaining the original video quality, metadata like dates and times and each scene (each startrecord-stoprecord bit) are also transferred.

I transferred family MiniDV tapes in this way. It took about three sessions, spread over 3-4 years, to complete the task, and I used Linux, Windows and MacOS to transfer the data, depending on what computers were available. The output is the same regardless.

Can you recommend a cheap firewire dongle?

If you have a thunderbolt 3 port then the apple adapters are reasonable second-hand, but you need a tb3->tb2, and tb2->firewire adapters. They work fine in Windows and Linux. OWC used to make TB docks with firewire, but they're rare and expensive now.

Anything that claims to adapt plain USB to firewire is bait for unsuspecting buyers, or maybe supported by some ancient obscure sony camcorder models.

The cheapest options for firewire these days are pci, cardbus, pcie, and expresscard in about that order. G3/G4 era macs up to 2012 models are common firewire machines. I know Sony and IBM had laptops with the interface in the core duo times.

I was rather sloppy with my words there. I used a cheap PCI card, some years ago, with Windows and Linux. I had a Thunderbolt adapter for the Mac.

The same process works with Digital8, Sony's attempt at finding a way to squeeze some more life out of their Video8 format by fusing the digital components of DV with the mechanics of Video8.

2 years ago I bought a firewire pcie card exactly for the purpose of digitizing old footage from both Digital8 and analog tapes by old Sony camera is able to read. Reminded me of the old days when you had to wait for the whole tape to load on my Amstrad CPC to play a game :) The result was great, a lot of memories, some tape fixing on the old analog tapes and some troubleshooting to start with. HINT: I had to boot with the camera already connected to the firewire card otherwise the capture failed

I need to get firewire into a newish system and do the same. Haven't done this since agp video cards were a thing. Guess I'll need to start with that the modern motherboards have available.

I'd love to find an ultra-wide-scsi adapter to pull data from an old 4drive in 1 - setup as well.

Think I've got an old scsi card. not sure it'd match up with the newer motherboard connections though.

> These camcorders use FireWire to communicate with the computer, so you will need a suitable cable.

dvgrab will work with USB via the -V flag:

  -V, -v4l2     capture DV from V4L2 USB device (linux-uvc)

USB was mostly fine for interactive capture. Firewire has been smooth for unattended transfer of whole tapes.

Obtw - My Firewire port and Sony MiniDV didn't work with dvgrab until I explicitly commented out

    blacklist raw1394

in /etc/modprobe.d/blacklist-firewire.conf

dvgrab is awesome! I used it quite a bit recently, transferring old tapes on to more reliable media. Sometimes it the tape had an error and I had to rewind a bit and restart capture, but it mostly just worked.

Be warned, it does produce pretty large raw files, though, so after import, I ran ffmpeg to convert them all to mp4 files. No visible loss in quality, and about 90% size reduction.

I did something similar, except I ran it as dvgrab | ffmpeg to transfer and transcode to mp4 at the same time, one step. It worked great.

dvgrab is great for set-and-forget archiving MiniDV tapes.

betamaxthetape mentioned[0] DVRescue[1] which looks very useful for error correction. The tools expect the raw .dv files, so check before compressing.

[0] https://news.ycombinator.com/item?id=27971246 [1] https://github.com/mipops/dvrescue

First let's agree on word terms. There is nothing to 'digitize' MiniDV is already digital. It's literally the first word in the technology. So now we are talking about transferring data from tape to local HD storage, in MPEG2-TS format where we can further transcode it to a newer format.

Can I use this to read my (data) backup tapes? I have am 8mm that I can't find a scsi drive for

That depends on how that tape was formatted. There are plenty [1] of articles [2] out there [3] on the use of digital video tape as a backup medium, if your tape happens to have been created through one of these processes you should be able to read it in a similar fashion. Instead of dvgrab these seem to use dvconnect, time to start experimenting I'd say.

[1] https://www.linux.com/news/using-camcorder-tapes-back-files/

[2] https://www.networkworld.com/article/2302563/using-a-camcord...

[3] https://www.linuxquestions.org/questions/linux-software-2/us...

It was written either on a DEC Ultrix or SunOS system in the late 1990s probably by dd of tar blocks

A quick online search leads me to Data8 [1], which I admit I've never heard of. Do you think that could be the format you're thinking of?

I know that SCSI drives were made for DDS [2], which (at least, initially) used the same tapes as the DAT audio format. But those used 4mm tapes, so probably aren't what you have.

Alternatively, it could have been a custom built system that recorded data onto video 8mm tapes. That will probably be a bit harder to get the data off unless you know the details of the setup used.

[1] https://en.wikipedia.org/wiki/Data8

[2] https://en.wikipedia.org/wiki/Digital_Data_Storage

It was quite likely written on an Exabyte device [1], you may be able to find out which by looking at the type of tape used - MP tape was used on standard Exabyte drives, ME (shiny, evaporated metal) tape on 'Mammoth' drives. It could also be AIT [1], a Sony-proprietary format which competed with Exabyte.

[1] https://web.archive.org/web/20070226174943/http://www.exabyt...

[2] https://en.wikipedia.org/wiki/Advanced_Intelligent_Tape

Yes. it's an Exabyte.

I don't know how helpful this will be to you, but I vividly recall the brand Exabyte using 8mm tapes for storage/backup as my father used these extensively for work. The backup format was identical to the video as we have home movies recorded on the failed data tapes just fine.

So all I need to do is find a generation of 8mm exabyte enabled camera with fireworks, and play maybe. Thanks

Nice article, at least it's a smooth procedure not involving trying to find specific FireWire cards -in fact finding something with a suitable firmware- as I did with SGI O2s and Mac G4s.

I do not think so. dvgrab is made to capture videos and audios from tapes in MiniDV format

Reply on news.ycombinator.com