The Everest Pipeline

Below are the basic instructions for reproducing the everest catalog. For large photometric surveys like K2, processing all light curves will take a lot of time and a lot of computing power. It took on the order of 100,000 CPU hours to generate the catalog for K2 campaigns 0-8.


Users do not need to follow these instructions, unless they wish to tweak model parameters and generate catalogs of their own. Most users will simply want to interface with the existing catalog. See the user interface section.

1. Download the data

Each everest mission implements its own functionality for interfacing with the raw data. For K2, individual raw light curves can be downloaded by running

everest.missions.k2.GetData(EPIC, download_only = True)

Batch downloading for entire campaigns is implemented for individual missions. For K2, you can submit a PBS job to a cluster with everest.missions.k2.pbs.Download().

2. De-trend the data

Individual light curves can be de-trended by instantiating one of the everest models:

everest.nPLD(EPIC, **kwargs)

Currently, the available models are everest.nPLD, everest.sPLD, everest.pPLD, and the experimental everest.iPLD. These are all subclasses of the everest.detrender.Detrender class, which accepts a bunch of keyword arguments.

De-trending generates a log log file, a npz model file, and a pdf data validation summary (DVS) file. These are all stored in the directory given by the TargetDirectory() function of the mission (for K2, see everest.k2.TargetDirectory).

Batch de-trending is also implemented for individual missions. For K2, see everest.missions.k2.pbs.Run().


You can check the status of the de-trending of each K2 campaign with the everest.k2.Status function. This is also implemented as the everest-status command line utility in everest/bin.

3. Compute CBVs

Co-trending basis vectors (CBVs) can be computed for each of the K2 campaigns once they are done running. To do this, run everest.k2.GetCBVs on each campaign. This will compile all of the de-trended light curves and compute the top signals that are shared among all light curves using SysRem.

4. Publish

FITS files can be generated by instantiating an everest model and calling its publish method. This step also corrects light curves with the CBVs and updates the DVS figure.