Humans have always expressed their appreciation for beauty and are prepared to pay to live in beautiful places. Communities protect areas of outstanding natural beauty and fund grand palaces and gardens, cathedrals, and art. Funds are always limited though, and the case for beauty needs to be made. To help protect what is beautiful, we need to start measuring beauty.

Previous research has focused on aspects of our environments that are large and easy to observe. This includes aerial footage to measure the percentage of green land cover; satellite imagery to measure tree density and different types of land cover; or census data to measure population density (e.g., White et al, 2013; Kardan et al, 2015; Alcock et al, 2015; Mitchell and Popham, 2007).

Some of that research has thrown up surprises: Mitchell and Popham (2007), for example, find that greater amounts of green space are actually associated with higher rates of ill health in low-income suburban neighbourhoods. The authors suggest that in such places, the quality of verdancy might also matter, not just the amount of green. We need a refined understanding of beauty.

How can we gather large-scale measurements to help evaluate what people find beautiful and how that beauty influences their wellbeing? For help, we have turned to large-scale data from the internet and deep learning methods that have proven very effective in many artificial intelligence tasks, such as facial recognition and object detection in photos, speech recognition and translation between languages (LeCun, Bengio & Hinton, 2015).

In a recent paper (Seresinhe, Preis, Moat et al, 2017), we explored 1.5 million ratings of over 200,000 images of Great Britain from the online game Scenic-Or-Not. In that online game, visitors to a site had to rate a particular picture of a landscape on the degree of scenic beauty.

We then employed a deep learning model that was trained to identify what the average visitor to the site thought of as beautiful. This involved two steps: first, the program had to recognise visual attributes, such as ‘trees’, ‘mountain’, ‘hospital’ and ‘highway’. In the second step, it had to learn which features respondents found to be ‘scenic’.

Across Great Britain as a whole, we find many results that accord with intuition. Features such as ‘valley’, ‘coast’ and ‘mountain’ are most associated with higher scenic ratings, whereas features such as ‘construction site’, ‘industrial area’, ‘hospital’ and ‘parking lot’ are most associated with lower scenic ratings.

In urban areas, we find that ‘natural canal’, ‘forest path’ and ‘river’ are associated with higher scenic ratings. But beautiful places are not only composed of natural elements; man-made bridge-like structures such as ‘viaduct’ and ‘aqueduct’, and characterful buildings such as ‘castle’, ‘tower’ and ‘cottage’ can also increase the beauty of a scene.

We also find that in urban areas, as well as across Great Britain as a whole, features reflecting large areas of grass, such as ‘grass’ and ‘athletic field’, lead to lower scenic ratings. This might be because areas that contain the most grass lack other features, such as trees or valley contours, resulting in an uninteresting scene.

In a third step, we trained a new deep learning model to rate whole photographs for scenic beauty. This enables us to generate scenic ratings automatically at a high resolution, making it possible to find out how people evaluate the local areas in which they live, something that is particularly useful in dense cities.

When identifying the top 5% of scenic views in London, our model again picks up that parks known for their scenery, such as Hampstead Heath and Richmond Park, have large clusters of scenic imagery. We also find highly scenic imagery in attractive built-up areas, such as Trafalgar Square, and around characterful buildings, such as Big Ben and the Tower of London.

We have thus built a new tool that helps us to evaluate how beautiful an area is and how beautiful individuals would find an area yet to be built. Importantly, not all green space is considered to be beautiful since it appears people want to have something in the green (such as trees, hill contours or water features). Our findings also reveal that buildings and bridges, too, can in fact be features of environments that we appreciate and in which we might enjoy spending time.

Our findings also reveal that buildings and bridges, too, can in fact be features of environments that we appreciate and in which we might enjoy spending time. In addition, our findings demonstrate that the availability of large crowdsourced datasets coupled with recent advances in deep learning can help us develop a deeper understanding of what environments we might find beautiful. Technological advances such as these open up new avenues through which researchers might develop evidence necessary for policymakers, and that will enable urban planners and architects to design spaces that will most enrich the wellbeing of their inhabitants.