Taking the example of color and mask propagation, she pre-trained the model on synthesized frame pairs generated from the MS-COCO dataset and then fine-tuned the network on the ACT dataset which contains 7260 video sequences with about 600,000 frames. Using NVIDIA TITAN XP GPUs, Liu and her colleagues trained this hybrid network on hundreds of videos from multiple datasets for color, HDR, and mask propagation. What makes this work unique is that the consequent colorization can be achieved via an interactive method in which the user annotates a part of the image, resulting in the finished product. The convolutional neural network infers what the colors should be from just one colorized frame and fills in the color in the remaining frames. “Now, colorizing a full video can be easily achieved by annotating at sparse locations in only a few key-frames,” Liu stated. Such redundancy has been extensively studied in video compression and encoding, but is less explored for more advanced video processing such as colorizing a video,” said Sifei Liu, Researcher at NVIDIA and the author of this paper. “Videos contain highly redundant information between frames. But now, a new deep learning based algorithm developed by NVIDIA researchers promises to make the process a lot easier - the new framework allows visual artists to simply colorize one frame in a scene and the AI goes to work by colorizing the rest of the scene in real time. ![]() Manually colorizing black and white video is labor intensive and a tedious process.
0 Comments
Leave a Reply. |