Using Deep Learning to Remember Pearl Harbor

I was inspired today by this article from Newsweek titled "25 Striking Photos of the Hawaii Attack." The first cover photo was striking indeed! It had been colorized somehow, which really brought it to life!

Sailors in a motor launch rescue a survivor from the water alongside the sunken USS West Virginia (BB-48) during or shortly after the Japanese air raid on Pearl Harbor. LIBRARY OF CONGRESS

All the rest were in black and white, but the content was amazing! I wondered what they would look like in color as well. That's where Artificial Intelligence and Deep Learning come in. I'd seen a method for using Deep Learning to perform Neural Style Transfer, like this one by Adrian Rosebrock on PyImageSearch.

Neural Style Transfer by Adrian Rosebrock at PyImageSearch.com

Basically, you're training the network to take the style of one image and transfer it to another. Pretty cool! I was also pretty sure I'd seen someone using Deep Learning to colorize video, so I looked it up.

Sure enough, I found this great paper by Richard Zang titled, Real-Time User-Guided Image Colorization with Learned Deep Priors. No need to go into the details, but he has a github repo with a handy jupyter notebook you can follow to try it out yourself.

The hardest part was getting Caffe installed since I usually work with TensorFlow. It took several hours to get everything configured correctly, but the result was these stunning images shown below. Well worth the effort!

The network was pre-trained, but initial guesses weren't quite creating the dramatic pop that the cover photo did, so I used the colorized cover photo as a global reference, and that refined the color predictions on the other images. I think they turned out pretty nice!

View of Battleship Row from the head of 1010 dock, during or immediately after the Japanese raid on Pearl Harbor. Original photo from Library of Congress.

USS Tern (AM-31) fighting fires aboard the sunken USS West Virginia (BB-48), on December 7, 1941, immediately after the Japanese raid on Pearl Harbor. Original photo from Library of Congress.

View looking up Battleship Row on December 7, 1941, after the Japanese attack. Original photo from Library of Congress.

Crew abandoning the damaged USS California (BB-44) as burning oil drifts down on the ship on the morning of December 7, 1941, shortly after the end of the Japanese raid. Original photo from Library of Congress.

The scene on the southeastern part of Ford Island, looking northeasterly, with USS California (BB-44) in right center, listing to port after being hit by Japanese aerial torpedoes and bombs during the raid on Pearl Harbor. Original photo from Library of Congress.

One of the seven Vought SB2U-3 Vindicators destroyed on the field at Ewa during the attack on Pearl Harbor. Original photo from Library of Congress.

I'm grateful to those who served and fought in WWII. I wish I'd gotten to know more veterans to understand their stories better, but I'm glad we have some pictures to help remind us of their sacrifices.

Credit for photo captions goes to Newsweek and their excellent article. Be sure to check it out for some historical content behind the images.