Generation of realistic tree barks using deep learning

Abstract

With the increase in demand for high-quality visual content in video games, movies, and simulators, it is of paramount importance to create realistic 3D models of trees, which are ubiquitous and find application in many areas such as urban modeling, movies, and gaming. In this work, we propose a methodology to create realistic 3D models of tree barks using a hand-held camera. There exist many computer graphics techniques which can achieve high quality tree generation; however, only a few works focus on realistic modeling of tree bark. The difficulties in generating realistic tree barks is mainly because of the complex appearance of the bark surfaces. The complexity arises because of the wide variety of barks and their intricate details. Consequently, many methods for realistic tree modeling are being researched. A majority of the work focuses on using the traditional methods in 3D modeling to generate the tree models. In this work, we explore a deep learning based methodology to generate high quality 3D models of tree barks. To the best of our knowledge, this is the first attempt at generating realistic tree barks using deep learning. We designed a pipeline to generate realistic-looking tree barks using multi-view 3D Reconstruction and Generative Adversarial Networks (GANs). 3D Reconstruction was used to generate tree barks of real trees, while GANs were used to generate tree barks of fake trees. As part of the pipeline, we developed an efficient method to perform the 3D reconstruction faster and at the same time, generate better quality 3D point clouds. We also analyzed different GAN architectures and loss functions to generate high-quality surface geometry and color of the tree barks. We designed a GAN architecture to generate the surface of the tree barks along with the bark color concurrently. Finally, we developed a GAN architecture to tile small images into a larger and a continuous image. To test the scope of application of our method, we generated oak and beech tree barks, which have contrasting tree bark structures, using our proposed pipeline. Our experimental results show that our approach generates realistic looking tree barks for both smooth and deeply ridged surfaces. The generated tree barks look realistic for both the tested bark types and our method was able to capture the fine details in the tree bark surface. Ultimately, this method could be used to generate thousands of 3D tree bark models for several types of trees.