With the help of deep-learning and segmentation maps, Nvidia’s GauGAN technology is capable of creating realistic images using nothing more than sketches.
Although the technology is not perfected by any means, it’s a big step towards digital transformation and allowing us to create our very own synthetic scenery.
By taking advantage of Nvidia’s GauGAN model, the following professions will be able to quickly and effortlessly create synthetic images:
– Game designers
– Urban planners
To get the technology this far, their team of researchers trained it by analysing well over a million images, including the 41,000 they got from Flickr. According to their words, it acts as a “smart paintbrush”, since it takes the sketch and fills in the details.
Bryan Catanzaro, Vice President of Applied Deep Learning Research at Nvidia, describes the process like a colouring book the GauGAN takes in and adds details to it – including shadows and reflections, things it learned by analysing real images.
Catanzaro further explained that the technology is far more advanced than simply cutting images and stitching them together, as it draws new ones in a similar manner to how blood-and-bones artists would do it.
Journalists were able to experience the first demonstration in person at the GTS conference. Reportedly, it can also switch styling to represent different seasons and tweak the reflections accordingly.
Nvidia is also working on technology that will greatly benefit the field of medicine. To illustrate an example, they used generative adversarial networks for the purpose of creating artificial brain MRI imagery to train networks on.