LTO Tape Archiving on the Cheap

We’ve been told by Larry Jordan and others that archiving our one-of-a-kind media files on hard drives is just waiting for disaster to strike. Eventually the hard drive will fail. Most people recommend using LTO tape as an archiving medium since it’s estimated to last 30 years or more. But even with new hardware like the mLogic mTape Thunderbolt drive, you’re still talking about $4,000 for the hardware and software. I needed a cheaper solution.

In response to an article by  called “Affordable, Easy LTO Archiving” on CreativeCow I posted this reply:

I’ve been archiving original camera files to Blu-Ray, but needed some way to archive FCPX projects which are about one TB each, on something other than a pile of hard drives.

I just took the plunge into the murky world of LTO and cobbled together a system for under $1500. Since we’re talking about “affordable” I thought I’d share my journey, since I couldn’t find an off-the-shelf LTO 5 or 6 Thunderbolt solution for under $3,500.

I found a Tandberg LTO 5 tape drive on eBay for $750 (probably a one of a kind deal), bought a used ATTO PCIe SAS card from B&H for $199, installed the card into a Akitio Thunder2 Thunderbolt to PCIe box ($219 on Amazon, including a Thunderbolt cable), found a 2 meter HP mini-SAS cable on eBay for $10, and downloaded the demo version of BRU PE for the Mac.


Then I found a guy selling new HP LTO 5 tapes in bulk on eBay for $15 each so I bought 20 of those. Imagine my surprise when I plugged the system into my 5K iMac and it all worked!

So far I’ve copied about 16 TB of data onto tape. I found out that the 1.5 TB tapes will actually hold about 1.35 TB before BRU asks for another tape. And it’s really slow — it takes about 3 hours to record that much to one tape, and another 3 hours if you want to verify data. I don’t know if the standalone systems are faster, but I won’t have a ton of archiving to do after the first pass, so I can live with it. I’m storing the tapes off-site and will keep the hard drive collection for quick restoring until they fail.

The only other expense will be the BRU license for $499, if I decide to keep using it. BRU kindly allows demo users to restore files even after the 30 day trial. I appreciate its ease of use but wish they offered a “light” version for us freelancers who just need to record a few dozen tapes per year. I tried LTFS but so far found software that will recognize the drive but it won’t format a tape. If anyone knows of a reliable, cheap or free LTFS solution that will work with my hardware setup on a Mac I’d appreciate hearing about it.

So it’s not elegant and it took a lot of research, but I’ve got my data archived on tape at last. Good luck to all in finding your own solutions.

Canon EOS 70D – Noise Test to ISO 12800

I picked up a Canon EOS 70D body to see if it would work for shooting video of musical performances.  I was a little wary of the APS-C sensor, since my previous bad experience with the Canon T3i.  But the promise of Live View continuous autofocus, touch screen and face detection on the 70D was enticing, so I’m trying it out.  Preliminary tests are very good, with the camera locking onto a face and following it as it moves.  Just what I need for stage work.

To check what the native ISO is on this camera, I did a test of the noise generated by the sensor itself.  I also wanted to see the nature of the noise at high-ISO settings.  Here’s the test (it’s best viewed at full screen at 1080p):

Noise looks good at the ISO 160 and multiples, like with previous Canon DSLRs.  The noise pattern is more noticeable than on a full-frame, but we’ll have to see how it shoots and intercuts in actual use.  More to come.

Notes:  Shot with body cap on at 1/60 sec, 1920 30 IPB. High ISO Noise Reduction set to Standard.

The noise has been enhanced in Final Cut Pro 7 with Brightness/Contrast.  Left side shows image as it came from the camera with Brightness boosted +1, Right side  +20.  Contrast -100 on both.


Mac Pro Upgrade – Mountain Lion, SSD, eSata, USB 3

My main editing computer has been an early 2009 Mac Pro 8-core Intel machine with 14gb RAM and an NVIDIA GeForce GT 120 graphics card, running OSX 10.6.8 Snow Leopard.  It’s been a workhorse but it was getting tired, taking a long time to boot up and do disk-intensive tasks.

I decided it was time to look into some upgrades to speed things up.  The first thing I did was to try using USB 3 on the Mac Pro to get some speedy external hard drives.  I found a Sonnet Allegro PCIe USB 3 card on Amazon.  It works with a 3 tb WD MyBook USB 3 drive getting up to 92mbs sequential write and 85mbs read.  But the two-slot card will only work with one device at a time and the drive will mysteriously disappear sometimes, only reappearing when I reboot.  I give the setup a 5 out of 10 because it works but is a dead-end since I can’t add a hub or another drive.

Next I tried a eSata card to get better speed from my Newer Voyager and NexStar external  drive docks which I’ve been running with Firewire 800.  I decided on a NewerTech MaxPower eSATA 6G PCIe 2.0 RAID Capable Controller Card from OWC because it supports RAIDs and they claimed it would run on the Mac Pro.  Speed increase was not dramatic, but I’m getting about 22% faster read/write from external drives.   I give it 7 out of 10.

I’m kind of late to the party with Solid State Drives (SSD) but from many accounts putting the system and main software on an SSD boot drive would be the best and speediest upgrade.  I decided to upgrade to OS 10.8 Mountain Lion first to see if it was stable, then install an SSD.

I ran Mountain Lion on a separate drive for a couple of weeks, only copying over software that I actually used during that time to see if I could eliminate a lot of the clutter in my Applications folder that I had accumulated over the years.  Plus Mountain Lion was not compatible with some of the old programs so I wanted to see what needed to be updated.  All the main programs I was using — Final Cut Pro 7 and X, Photoshop 5.1, After Effects, Compressor, ClipWrap, MPEG Streamclip, MS Office — seemed to work fine under Mountain Lion.  I had to upgrade Pluraleyes to the new Red Giant version, and I had to update drivers for Matrox Max, the Sonnet Allegro USB 3 card and a few other utilities.

All was good except that if the system went to sleep it would not wake up when you hit the keyboard or mouse, and I’d have to do a hard restart.  I didn’t lose much work because I usually save everything before getting up from the computer, but it was really annoying.  After checking a lot of posts by people with the same problem, I stumbled on a solution posted by Bonfire to upgrade the Nvidia graphics card drivers.  After downloading and installing the QUADRO & GEFORCE MAC OS X DRIVER RELEASE 304.00.05, everything is working fine.

Next the SSD.  I  picked up a Sandisk Extreme 480gb SSD on eBay for about $300.  Mounted it in an Icydock 2.5 to 3.5 converter box.  Loaded Mountain Lion and essential applications and swapped out my old boot drive in bay 1.  Startup time went from almost 2 minutes down to 45 seconds.  It’s reading/writing about twice as fast as my old drive according to Xbench.  9 out of 10 with just a knock for the expense.  I hope SSDs will start to drop in price like SD cards have.

So I’m happy with the upgrades.  For about $400 in parts it feels like a new machine, which will suffice until or if Apple decides to update their Mac Pro line.


Hacked and Fixed

This blog was hacked last month.  Google said it was blocked because of malware.  I never heard of this before, but I downloaded the files and checked them with ClamXav, a free virus checker, and found that 673 out of 1670 files were infected with something called Trojan.PHP-33.  The support desk at Bluehost where my website is hosted suggested using a malware scanning service to help clean out the bad code.  I decided to contact since it got some good reviews and was considerably cheaper than the alternatives.

Thomas J. Raef at We Watch Your Website had the site cleaned up in one day and now we’re back up and running.  The service monitors the site and will clean it as many times as necessary.  I also installed the anti-malware plug-in Anti-Malware by ELI (Get Off Malicious Scripts) on WordPress to prevent re-infection.  Finally, I contacted Google to unblock the site and it was cleared in a few hours.  All in all it was a pain to deal with.  I’m not sure why anyone would want to hack this little blog, but I hear that it’s pretty common.  I’m glad I was able to get rid of it.   If you are using WordPress I’d suggest at least installing the Anti-Malware plug-in, and subscribe to We Watch Your Website if you want to be proactive.

Shooting Video at Yoshi’s Oakland

We recently had the opportunity to shoot a performance at Yoshi’s Jazz Club in Oakland’s Jack London Square, one of the oldest and most respected jazz venues in the Bay Area.  It’s a 330-seat jazz club with a state-of-the-art sound system.

I talked to the contact person there a few days before the show to just check in and see if there were any issues to be aware of.  He said the floor manager would help us find places to set up cameras and that we could get a stereo feed from the sound board.

The night of the show we loaded-in at 5:30pm at the side entrance and found street parking.  (There’s a large paid parking lot above the club too, but at sound-check time there’s plenty of street parking.)

The club has a semi-circular stage with small cafe tables and some circular upholstered seating. Stage lighting is all tungsten which makes color balancing easy.  The house manager cleared one table area next to the sound board for our stationary wide camera and audio recorder.  We were given another table spot on the left for a manned stationary camera.

I put the small Canon HF-G10 on a tripod without opening the legs and gaffer-taped it to one of the table dividers on the right.  Finally I had the Canon 5D mkII on a monopod with a 70-200 f2.8 lens that I could use to roam on the right.

The lighting operator agreed to keep the light levels constant through the performance, and she gave us a few minutes to meter the actual stage lighting after sound-check.  The Sony AX2000s were set at f3.2 @ 1/30th with 6db gain.  The Canon G10 needed about 15db gain at the same settings.  The 5DmkII was at f3.2 @ 1/30th at 1250 ISO.

The audio feed from the board was line-level through XLR cables.  My standard audio recording setup now is to run the line-level signal through a portable Shure FP-33 mixer and send mic-level signals split with Y-cables to the camera and to a Zoom H4n.  This solves the problem of the Zoom distorting from a hot feed, and also gives me automatic backups to the camera inputs.  I had plenty of time during sound-check to set levels and make some test recordings.

As performance time approached we started rolling sound and the stationary cameras.  Luckily the house was about 3/4 full so there was some room to roam on the right side, mostly in the back row and against the wall.  The distance from there was a little long for the 70-200mm.  I was getting some medium close-ups of the performer, but couldn’t get in to tight head shots.  Getting physically closer would have blocked some of the audience’s views, which we had promised we wouldn’t do, so I just shot what I could.  The manned Sony on the left could get in closer with a longer zoom, so we’d have some tight shots from that angle.

If the show was sold-out it would have been very difficult to shoot with any kind of mobility, and I probably would have manned a stationary camera on the right.

We didn’t eat dinner at Yoshi’s that night, but they have a pretty good Japanese restaurant attached, and a decent bar menu with sushi, chicken skewers, etc., and a full bar.  There are also a lot of other restaurants in the area like Scott’s Seafood, Miss Pearl’s, Bocanova, etc.

All in all it was an excellent shooting experience.  The management and technicians were very accommodating and helpful, and treated us like professionals.  They charge the performers a fee to allow video shooting, so that could be an issue for some people.  We’re looking forward to shooting there again, or at Yoshi’s other location in San Francisco.

Studio Portraits: Tethered Shooting

I had the opportunity to shoot some corporate portraits this week in a temporary conference room studio.  My setup is pretty simple — 4 Alien Bees strobe heads: key, fill, background light, and a hair light from behind the background.  Canon 5dMkII with a 70-200 f2.8 lens.

Since it was a static setup I thought I would try tethering the Canon to a MacBook Pro so the clients could see the results immediately.  I used the Canon EOS Utilities software for remote shooting and it worked great.

First, it helped with the set up because you can take test shots without writing to your CF card or disk.  There’s an integrated histogram to help with exposure settings.  You get a nice big image to evaluate.  And you can change the camera settings remotely from within the software, including ISO, speed, aperture.  The only things you can’t change are the physical zoom and framing of course.

During shooting it helps with the large image to monitor shots.  The clients can then check the images and select the best ones, or we can talk about trying different poses.

Finally, and best of all, you can set it to write both to the CF card and your hard drive, so you get instant redundancy for safe backups.

I only wish there was a more portable version of this.  Lee Morgan of showed a way to wirelessly monitor photos instantly on an iPad a while back, but it’s just for monitoring, not controlling the camera.  I wouldn’t mind a wired connection if it added controls, but I don’t think it’s possible with Apple’s connector.

There is promising beta software called DSLR Controller available for Android tablets as reported by Petapixel last year.

Archiving Video Projects to Blu-Ray

I’ve had a few hard drives fail in the past but luckily didn’t lose anything too valuable.  But I get paranoid that something I put a lot of time into will be lost forever with a drive crash. I routinely back up original media to a separate drive, and the Final Cut Autosave Vault has saved me many times, but when I’m done with a project I needed a way to archive everything.  A long-form video project can eat up from 300 to 600gb so I needed something that can handle that amount of data, be relatively stable and cheaper than just buying more 3tb hard drives.

A lot of pros recommend archiving to LTO tape systems, but those drives are over $1000 and the tapes aren’t cheap either.  So I started archiving video projects to Blu-Ray discs as a cheaper alternative.  They hold up to 25gb per disk (50gb on double layer disks) and are rated to last up to 50 years — a lot longer than hard drives or tape.

I picked up an internal LG 12X Blu-Ray burner ($87 from and installed it in the second bay of my Mac Pro in about 10 minutes.  After researching different brands of media it looks like Sony Blu-Ray discs are consistently rated highly, and I found 50-disc packs for about $75.  I used Toast Titanium 10 to gather all the files from various on-line drives, and set them up for burning.  Toast will automatically span as many discs as needed for a large archive.

To keep things to a reasonable size I archive the original camera and audio files and everything from the project except render files and transcoded media since those can be recreated from the originals if needed.  A recent long-form project with about 3.5 hours of final video took 14 Blu-Ray discs to archive, costing about $21 for materials.   Each disc takes about 15 minutes to burn and verify, but you can work on other things while waiting.

A boring task, but I figure it’s good for clients to know you’ve got their material archived, and it’s good for your own peace of mind.


Faster YouTube Upload Success – Fiber Optic Internet

I have a potential client who needs one-hour HD videos uploaded to YouTube within a very short time window.  After compressing a test video with the Matrox CompressHD card to 720p h.264 with Compressor (45 minutes) the file size was 2.32gb.  Uploading through my Comcast residential connection at about 4Mbps took over 4 hours.

I tried a Comcast business connection which said was uploading at 9.1 Mbps, but that still took almost 3 hours.  I tried at different times of the day but although Speedtest said some upload speeds were faster than others it didn’t make a lot of difference in the time it took to upload.

After an extensive search and posting on several forums for help I finally found a local office building which is home to many non-profit organizations, and which has a fiber optic connection to the internet.  Plus they have free public wi-fi.  The upload speed is still only 9.9 Mbps, but the YouTube upload of the same file took only 38 minutes!

I don’t know why there is such a huge difference in speed.  Is Comcast somehow choking the upload process, or does a fiber optic connection make it faster even at the same upload speed?  More research is needed, but at least now I know I can meet the client’s need for speed.

Project Management for Video Files: Post Haste+DiskTracker+Voyager

I’ve been using Post Haste since I read a tip from EditingWhiz about keeping your video files organized so you can find what you’re looking for.  It’s a free program from Digital Rebellion that has templates which automatically set up folders for different media from FCP, After Effects, Photoshop, etc., and creates a dummy file in each folder with a consistent naming convention based on customer name and date for that project. It’s customizable for your own situation. I’m on a Mac but I think they have a Windows version too.  Coupled with a disk cataloging system like DiskTracker, you can find files easily across multiple internal and external drives.

I’m using a Newer Technologies Voyager quad interface external drive dock and about a dozen (and growing) 1 to 3tb drives that I use to archive projects.  The Voyager has Firewire 800 & eSATA which speeds up write times a lot over USB.