Are you smarter than an ATLAS algorithm?

Article: ‘That looks weird’ — evaluating citizen scientists’ ability to detect
unusual features in ATLAS images of LHC collisions
Authors:  A.J. Barra, A. Haasb, C.W. Kalderon
Reference: arXiv:1610.02214v1

As it turns out, non-expert citizen scientists can match or beat out ATLAS algorithms in identifying features in images of LHC collisions.

The ability of the general public to identify long-lived particles and other unusual features in images of LHC collisions recorded by the ATLAS experiment was studied using data from the Higgs Hunters project. The Higgs Hunters project was launched by NYU scientists and colleagues in 2014, and allows members of the general public to study LHC images to help search for previously unobserved particles. Writing computer algorithms to identify “weird looking things” in these images can be difficult, and human eyes, be they expert or non-expert, can help with the hunt. The Higgs Hunters project scientists are specifically searching for previously unobserved particles that could be created via decay of the Higgs. In some cases, the tracks left in the ATLAS experiment could be picked out better by human eyes than computer programs.

This isn’t the first time the scientific community has reached out to non-experts to classify images or aid in scientific pursuits. Through the Galaxy Zoo project, for example, citizen scientists contributed to the results of 48 scientific papers by classifying galaxy shapes and spotting unusual objects in images from the Sloan Digital Sky Survey and other image datasets. In a more indirect way, the public has also been previously invited to contribute to CERN’s science by donating their personal computer’s idle time to help simulate proton-proton collisions. The Higgs Hunters project builds on this history by being the first to allow the general public to take an active role in searching for new particles at the LHC, which not only has the practical benefit of helping scientists out in their search, but also inspires the non-scientist public to take an interest in the field of particle physics.

The task selected for the Higgs Hunters project was that of identifying new particles, \phi, dubbed “baby bosons,” as they decay within the ATLAS detector. Such particles are predicted in theories in which an additional scalar mixes weakly with the Higgs boson. These processes were chosen because they had not been previously unobserved and would generate a signature that is fairly easily identifiable by eye, rendering the citizen scientists competitive with standard reconstruction algorithms. The discovery of these particles would be a high impact scientific discovery, a key motivating feature when employing citizen scientists.

Before being presented to volunteers, images were pre-selected to include those containing a muon and an antimuon with invariant mass consistent with the mass of the Z boson. Such events are consistent with the Z boson decay processes Z \rightarrow \mu^+ + \mu^- and have an increased probability of also containing a Higgs boson, since virtual Z bosons may emit Higgs bosons through the ‘Higgs-strahlung’ process, Z^* \rightarrow Z+H. Data were selected from the 2012 data-taking period between April and December. The ability of the volunteers to identify the desired events was calibrated using test images which showed Monte Carlo simulations of the process of interest, H \rightarrow \phi + \phi. All of the images presented to volunteers, be they simulations or real data, were processed using the ATLAS reconstruction software.

An example ATLAS detector image presented to citizen scientists, generated from a computer simulation of the process H → φ+φ. The green lines emanating from the center indicate the reconstructed muon and antimuon used to select the event. The red dotted line indicates the direction of missing momentum transverse to the beam. Source: arXiv:1610.02214v1

As of October 2016, classifications had been performed by ~32,000 citizen scientists of a wide arrange of ages and backgrounds from 179 countries. Peaks in volunteer activity occurred soon after the project launch and when CERN news stories were published about the project. Each image was classified by ~60 people. The number of classifications completed by each citizen scientist follows an approximate power-law behavior, with most volunteers classifying just a handful of images, but ~1,000 people providing 100+ classifications each. The most dedicated enthusiast provided nearly 20,000 classifications. In total, 1,200,000 features of interest were classified on ~39,000 distinct images.

Left: Start date of new citizen scientists joining the project for the first time. Right: Number of classifications per citizen scientist, displaying an approximate power-law behavior. Source: arXiv:1610.02214v1

It was found that the citizen scientists’ performance competed very well with that of the computer algorithm, even beating it for events with low mass (8 GeV) baby bosons regardless of image view and boson lifetime value. As the mass of the boson increased, the algorithm began beating out the human volunteers. A “weird thing” was also spotted: an image showing a collision apparently containing a jet of multiple collimated muons. In the Standard Model, jets are always of hadrons, not muons. After further investigation by the science team, it was found that the event was due to an unusual interaction of a known particle with the detector, rather than an unusual or new particle. While this didn’t represent a discovery of new physics, the potential of citizen scientists to pick out “weird stuff” could lead to the future identification of interesting features in LHC collision data.

The volunteers responded positively to the project, with the overwhelming majority (>97%) interested in continuing their participation in a future CERN physics project. 47% of respondents said they were more likely to go on to study physics as a result of participating in the project, and 80% felt that their knowledge of particle physics had been improved.

In the future, relying on non-expert citizen scientists could help scientific collaborations classify data as well or better than computer algorithms could, while engaging the public in physics in a valuable and influential way. Just last month, LIGO launched its “Gravity Spy” citizen science program, in which participants search LIGO data for “glitches” that can help LIGO scientists distinguish between the signals they observe.

Background reading:

The following two tabs change content below.

Eve Vavagiakis

Eve Vavagiakis is a Ph.D. student and NSF Graduate Research Fellow in the experimental cosmology group at Cornell University, working on the Advanced ACTPol (AdvACT) project. She won the Cranson W. and Edna B. Shelley Award for her undergraduate research at Cornell developing a cryogenic scanning Fabry-Perot interferometer for mid-IR to submillimeter astronomical observations.

Leave a Reply

Your email address will not be published. Required fields are marked *