Wednesday, November 4, 2015

How We Monitor Data Collection with Advanced LIGO

The first Advanced LIGO observing run (O1) started in mid-September and will end in mid-January.  Today I want to tell you about how we collect our data.  On the surface this is obvious: with computers and sensitive electronics.  But how do we keep the detector working so that we can collect data and how do we know that our data is good?

The LIGO Livingston control room on 3 November 2015 (during O1).


The most important step in collecting data is that the detector is working.  This is the primary job responsibility of the roughly 10 operators who work at the site.  There are 3 10-hour shifts a day, each one overlapping with the previous operator's shift by 2 hours so that the incoming operator can be brought up to speed on any issues that may be ongoing.  Since O1 will last into mid-January, that means that there will be at least the operator in the control room every night, weekend, and holiday - even during Thanksgiving dinner, Christmas morning, and New Years at midnight!

During their shift, they monitor various things like the power of the laser, local vibrations, and a multitude of other readings from all over the detector that tend to drift over time.  This work is mainly to prevent a fault in one of the systems that would interrupt data collection.  When everything is working the way it is supposed to, this part of their job can be boring - and we love boring days and nights. 

Excitement happens when we are no longer able to keep the light bouncing back and forth between the mirrors (we call this "breaking lock").  The operator's job now is to respond by discovering if the lock was lost due to an environmental issue we can't control (like an earthquake anywhere on the planet) or due to an detector issue.  If there is a malfunction in the detector, the operator identifies what subsystem caused the problem and then uses their training to fix it and get the detector up and running again.  Through my conversations with them, one of the harder parts of their job is identifying which part of the detector isn't working properly since there are so many subsystems that need to work all at the same time for us to be able to collect data.


During the Initial LIGO science runs, there were always 2 people in the control room: the operator and the "scimon" (short for science monitor).  The scimon's job was to ensure the quality of the data that was being collected and give feedback to the operator.  Scimons came from institutions across the country who would usually spend a week or two at the observatory before returning to their home.  This meant that there were a lot of people passing though the observatory (which isn't bad) and by the time they really got comfortable in their job it was time for them to go home (this isn't good). 

We are doing the science monitoring differently for Advanced LIGO: we have longer-term (several months) visiting scientists (LSC Fellows) working on site to monitor the data as it is collected and we have data quality scientists (we call them "DQ Shifters") who remotely monitor the properties of the data for a period of 3-4 days.


These scientists are on-site to monitor the data as it is collected and they also each have a project related to improving the instrument.  There is almost always a fellow on-site except for the earliest hours of the day (they are not as necessary as the operator and their instrument research is best done when other scientists are also around).  The fellows work with the operators to identify subsystems that may be causing issues and they work to resolve them.  Basically, these are the Advanced LIGO version of the scimon but with the benefit of having the visiting scientist being able to apply what they learn while on site.


The DQ shifter is a scientist who monitors the quality of the data that has already been taken (within about a day or so).  Sometimes, patterns only become evident after a significant amount of data has been collected.  Because this work is not expected to have immediate feedback to the operators and fellows, this work can be done remotely.  We have created automated web pages that have all the plots needed to look at how the different parts of the detector are working.  There are about 40 or so of us (including me) who have been trained on how to interpret all of the graphs that appear on these pages and what specific things we should be watching for.  We communicate with the fellows at the site we are monitoring on a daily basis so that they can use the feedback to improve the quality of the data.  When our shift is done (we usually cover 3-4 days in a shift), we document our findings, report to the data quality group who specializes in studying collected data, and we enter an entry in the detector log with a summary of our shift.

Summary pages used by DQ Shifters to evaluate the quality of data already taken.  These plots specifically show how the ground was moving in different frequency bands throughout the day on 2 November 2015.


  1. Thanks for the peek into aLIGO. As an interested civilian (I scored a B- in high school physics) I'm tickled a prominent part of the project is being done in my back yard. And, I can't wait for the results of O1 (whether it observes anything or doesn't find something).

    1. I'm glad you're excited! I sure am! I can't wait until we start making regular detections and getting to do astronomy with the information we collect. I want to "see" something we never knew of before...

  2. Hey, what version of Ubuntu is that you're running there? Do you guys generally stick to LTS releases? How many different OSes do you use in your environment?