Wednesday

DNG workflow ideas

When I come back from a wedding, I've got hundreds and hundreds of RAW files that have to be dealt with. It's not uncommon to have 20-30 gigabytes of data on several CompactFlash (CF) cards. Here's what I do to manage all those photos.

When I put a CF card in to my reader I like to copy the files directly to an external hard drive rather than having some program (Lightroom or Photo Mechanic) do the transferring for me. Call me paranoid, 'cuz that's what I am. I like to have my original image source (the CF card) plugged in to my cranky computer for as little time as possible.

Here's my secret recipe (how corny is that?)..

First I create a folder with a unique filename for the event (like "2008/02/29_Karen & Paul") so that I can keep everything organized. As I work with the files in the days ahead, I always keep that folder structure. For the past couple years I’ve been starting the name with the year, because after several years of doing weddings I found that I actually had some redundant names and dates: in fact I’ve been called “The Official Photographer of Jennifers and Mikes.” This folder sits on an external firewire hard drive so that if I have any sort of system problem I'll have a set of files that are, again, independent of the computer.



Then I use the Adobe DNG Converter to process all those proprietary RAW files on the first external hard drive to another external drive.

The first option you see when you open the Adobe Digital Negative Converter is the select the images you want to convert. Here's where I pick that first folder where I brought in the files from the CF cards.



Just down from there you have the area where you can choose where you're going to put the images once you've converted them. In my case they are going in to a second folder (but with that same name as the first folder, “2008/02/29_Karen & Paul) on a different drive where I'm going to end up importing them later in to Lightroom. I have numerous external drives, and this second one is an eSATA drive that I have set up as a RAID array.)

The renaming option is next, and many photographers may want at this point to name their files with something that is unique to the event or assignment. I hold off on renaming until the last step of my workflow, because I may end up editing out lots of photos from the final set that goes to a client and I like all the filenames to be continuous. (They may never notice, but I don't like to have a gap in my numbering in files that go to clients.. "I've got number 16, and number 19, but where are 17 & 18?" is a question I want to avoid.)

Lastly you find a place where you can set some preferences for how you want the files handled during the process. What I've chosen to do is to leave the files uncompressed (more on this in the next post) and to preserve the original raw file rather than convert it to linear data. This allows me to refer to the original raw data later if I need to. And I choose not to embed the original raw file in with the DNG, because I already have a backup of those files anyway and for now I get as much out of the DNGs as I need. Also, if you choose to embed the original raw files you end up with a MUCH larger set of files, because you've got a "box" with the original file PLUS the converted file. If you do choose to embed the file, you can also later extract that with the "extract" button options at the bottom of the converter.



Now the converter purrs away, transmogrifying all those files at breakneck speed.

So, now I have my dngs all sitting on the second drive, and it's at this point that I launch Lightroom and import the images in place. This happens very quickly. In fact, bringing in 1100 images to Lightroom from last week's wedding took less than 8 minutes, including building the standard previews and writing the keywords and iptc information to each file. That first drive, where I brought everything in from the CF cards, really just serves as a backup at this point, and I leave the files there until the job is delivered. Once that’s done, weeks from now, I’ll clear them off to make room for new images.

Once all the images are safely off the CF cards, I'll do a quick pass through the images in Lightroom and delete any obvious screw-ups like super-underexposed photos, out-of-focus shots and photos I accidentally shot of my feet. This eliminates, oh, half of the pictures right off the bat. I select all the images and choose SAVE, which writes any Develop or Metadata changes in to the files in case I’ve played with tweaking some of them (and I always have). Then I burn DVD's of the remaining images and put the discs in my safe in the basement, which is supposed to be fireproof. I'm not testing that anytime soon.

Now I'm free to do my major editing in Lightroom or, if I need to do some detailed work I can take the images in to Photoshop. I've got the original RAW files if I need them, and a set of DNG's from those RAW files on another drive and a set of those files backed up to DVD. At this stage, I can erase and format those CF cards and stick them back in my camera for the next shoot, knowing that short of a major lightning strike that sets fire to my house I should be able to work with last week’s pictures until well in to the 21st Century.


3 comments:

Minotaurus007 said...

Interesting article. I understand you are using the Fuji FinePix S5 Pro. I still haven't found how to extract those RAF plus JPG-Files from the CF-Card *without* Fuji FinePix Studio. Can you help me?

Of course, I get JPGs when shooting in JPG-Mode. And RAFs when shooting in RAW-Mode. What I like to do is to shoot in combined mode: RAF+JPG!

However, when looking at the CF directory there is a strange listing in windows explorer and other tools:

DSCF1234 - FastStone RAF-File
DSCF1234 -
DSCF1235 - FastStone RAF-File
DSCF1235 -
etc. ...

... even when I'm dragging those files to my harddrive. I get two identical 25 MB Files, and no JPG can be found. I have to process these files with Fuji FinePix Studio, i.e. import them via this slow piece of software to my HD.

Back from a shooting with 500 or 1000 pix (SanDisk Extreme III 16 GB) this process via the Fuji S5 Pro takes longer than it's battery lasts!

Any ideas or a solution?

Thomas said...

Hi back! I have to make two admissions here..

1) I don't shoot in RAW+JPG mode, and I kinda don't understand why anyone does. If you're using any photo software at all, you get a better file with your RAW image and you can make jpegs almost instantly with much better control and quality than your camera can. Why you'd want to fill your CF card with a bad jpeg that you have to get off your computer anyway is a mystery to me, and I'd love to hear a good rationalization for this process.

2) I very quickly got frustrated with the Fuji software and don't use it at all. It is terribly cumbersome, and as you point out quite slow as well, so I use the Adobe products instead (the free DNG converter, Lightroom or Camera Raw.)

There are a number of free applications (and as many ones that you pay for) that you can use to extract a jpeg from your RAW files. I don't have much experience with them, because I've pretty much always used Adobe products.

So, since you're not able to extract the jpegs you shoot, I'd save your CF card space and just shoot in RAW, and then convert them to DNG's so you save a little space and can get away from the Finepix Studio software.

Minotaurus007 said...

@Thomas

ad1) Why JPG+RAW?

Fuji FinePix S5 Pro has - like any other camera - the in-built RAW-converter that produces JPGs. According to the situation, you can choose the kind of JPG you want by preselecting F1 F1a F2b F2. I think this is a unique feature of the camera. This produces different but constant results in the JPGs. You can immediately put them on an USB-stick or DVD without any "RAW-games".

95% of these pix go to the trash anyway and so do the RAWs. From the remaining I select the top pix and THEN - if necessary - try to optimize from the RAW-files.

For me and my hardware it is much more conveniant to handle 500 pix รก 2.5 MB (M setting) than 500 pix a 25 MB (RAW).

Speed and no need for special hard-/software to get the first results is very important. And you must admit that the S5 produces decent colors, sharpness and little noise in most situations anyway. With JPGs out of the camera you have little time-to-print and can use pictures right after the shoot.

- - -

I found the solution to the downloading problem. Just insert the CF into a card reader. Then JPGs and RAWs will be recognized by the operating system. While in camera only the RAWs are recognized (see my former comment).