The skies epitomized

Art + Machine Learning showing how we see the sky.

The Skies Epitomized is a series of artworks exploring the essence of the sky from the perspective of humans gazing at it. The works were created in collaboration between Maja Petrić, an artist, and Nebojša Jojić, a machine learning researcher. The artwork is derived from big data through a machine learning algorithm that is used to create visual summarizations (epitomes) of sky images people posted on the Internet.

Exhibited: December 2015 – March 2016

Maja Petrić explains the motivation behind the art: I find it fascinating to see a visualization of how we see the world. Nebojša Jojić’s epitome algorithm actually paints the picture of the sky in the eyes of the Internet community, which includes most of us. I believe that the image of “the sky in our eyes” can unlock understanding about human relationship towards our origin, and our surroundings.

Nebojša Jojić’s epitome model is a generative model of images that assumes that all images of interest come from a large panorama in which individual pixels can change slightly. The individuals are small cutouts with small local perturbations. The panorama is much larger than the images in the set, but much smaller than a tiling of all images would be. The panorama is thus referred to as an epitome of the set, as it captures the variation seen in the data in a miniature form, but also expands on individual images by placing them in the context of other images. The learning algorithm iteratively maps each of the images from the set to the best matching locations in the epitome and then re-estimates the panorama and the variation parameters until it achieves convergence.

Artist statement

People have been turning their eyes towards the sky for meaning and guidance since prehistoric times. Our gaze directed towards the sky has shaped astronomy, mythology, religion, agriculture, art, physics, and world-views of cultures across this globe. As an artist, I am interested in the meaning of that gaze at the sky as an element that is both mundane and archetypal for the history of our civilization.

I was guided by the notion that on Earth, we share one sky, but each person looking at it has its’ own experience of it that depends on time and place they are observing it, but mostly it depends on their “eyes”, subjective view. The multitude of mental images of the sky reflects large numbers of perspectives about the space that surrounds us and our place in the universe.

I wonder what is the significance of our way of perceiving the sky and how is that perception of the sky shaping the environment and our behavior towards the surroundings, and in so other people. In this project, we used Nebojša Jojić’s epitome model to construct the visualization (epitome) of sky images people posted on the Internet. Here, a machine learning technique allows us to go inside the head of the Internet and have a peek of it’s mental image of the sky, and also of the process of constructing that image.

Epitome of earth skies

A world map of epitomized skies

The machine learning algorithm sifts through big data for representational images of the sky for each climate and uses it to create visual summarization (epitome) of people’s perception of the sky changing across the globe. It is a visualization of the sky from the human perspective. Each location in the world is covered by the most recognizable kind of sky in that area. It is meant to be reminiscent of the transient nature of the experience people have when they gaze up. The goal is to allow the viewer to grasp the wide range of experiences and memories from these geographic locations.

Epitome of local skies

This series epitomizes the online images of the sky retrieved through web search for keywords corresponding to specific times and places. Through these epitomes, we depict the collective view of the sky across several locations in time and space. These views are facilitated by the vast collection of images people around the world have posted on the web, and are thus also a reflection of how the web might “comprehend” the sky in its boundless variation: Epitomes reveal archetypical skies defined by a global Internet community. With this series we are both inquiring about what people see when they gaze into the sky, and exploring ways in which human views of the sky shape our landscapes.

Maja posed a question: What is a collective perception of the polar sky?

In search of an answer, we used big data that was processed by a machine learning algorithm to construct a visual summarization (epitome) of the polar sky.

To create the epitome of the polar sky, the algorithm sifted through big data in search of the images under the keywords sky, skyscape, north+pole+sky, south+pole+sky and used Nebojša Jojić’s epitome model to extrapolate parts in found images into an epitomized data set representing what people perceive to be a typical polar sky.

Next to each sky epitome is a photograph of a person gazing at the sky. The photographs are retrieved from the web in the same manner: by web search for a person that represents that location, climate, or time point.

Epitome of the Polar Skies

Epitome of the Tropical Monsoon Skies

Epitome of the Arid Climate Skies

Epitome of the Moderate, Hot and Humid Skies

Epitome of the Moderate, Mild and Rainy Skies

Epitome of the Tropical Wet Skies

Epitome of the 9/11 Skies

Epitome of the Baghdad Skies