Preprocessing Modules

These modules contain functions to reduce the number of candidate sources associated with a transient, calculate relevant attributes, and combine the output of association pipeline into a final Pandas DataFrame.

sourceCleaning

astro_ghost.sourceCleaning.check_dict(dic, df)[source]
Check to make sure that the list of all potential hosts

in the dataframe matches all potential hosts in the dictionary - that the two describe the same hosts.

Parameters:

  • dic (dictionary) – key,value pairs of transient name, list of candidate host PS1 objIDs.

  • df (Pandas DataFrame) – PS1 properties for candidate hosts.

astro_ghost.sourceCleaning.clean_df_from_dict(dic, df)[source]
Remove sources from PS1 object DataFrame if not in the

dictionary matching transients to their candidate hosts.

Parameters:

  • dic (dictionary) – key,value pairs of transient name, list of candidate host PS1 objIDs.

  • df (Pandas DataFrame) – PS1 properties for candidate hosts.

Returns:

PS1 properties, after removing sources.

Return type:

Pandas DataFrame

astro_ghost.sourceCleaning.clean_dict(dic, df, to_keep=[], bestDetectionCut=False)[source]
Clean the dictionary to match the dataset.

note that if we’re cutting by bestDetection, we find that we cut out many true hosts. So we only overwrite the list of potential hosts in this case if our new list isn’t empty.

Parameters:

  • dic (dictionary) – key,value pairs of transient name, list of candidate host PS1 objIDs.

  • df (Pandas DataFrame) – PS1 properties for candidate hosts.

  • to_keep (array) – List of PS1 objIDs for candidates to keep in the dictionary (even if they’re not in the dataframe).

  • bestDetectionCut (bool) – If True, bestDetection==1 was a selection cut. Don’t remove hosts from the dictionary if this would remove all of a transient’s potential hosts.

Returns:

key,value pairs of transient name, list of candidate host PS1 objIDs, after removing PS1 objects not in the dataframe.

Return type:

dictionary

astro_ghost.sourceCleaning.getColors(df)[source]
Calulate observer-frame colors for PS1 sources, and make some cuts

from bad photometry.

Parameters:

df (Pandas DataFrame) – PS1 properties for candidate hosts.

Returns:

PS1 properties, with color and photometry cuts.

Return type:

Pandas DataFrame

astro_ghost.sourceCleaning.makeCuts(df, cuts=[], dict='')[source]

Make a series of quality cuts on the candidate host galaxies in the dataframe.

Parameters:

  • df (Pandas DataFrame) – PS1 properties for candidate hosts.

  • cuts (array) – List of cuts to apply. Options are: ‘n’ - remove objects without at least 10 detections. ‘quality’- remove objects with PS1 qualityFlag > 128 (suggesting bad photometry). ‘coords’ - remove objects with missing position information. ‘mag’ - remove objects with missing photometry (aperture magnitudes). ‘primary’ - remove objects with primaryDetection = 0. ‘best’ - remove objects with bestDetection = 0. ‘duplicate’ - remove all completely duplicated rows.

  • dic (dictionary) – key,value pairs of transient name, list of candidate host PS1 objIDs.

Returns:

PS1 properties, with quality cuts applied.

Return type:

Pandas DataFrame

astro_ghost.sourceCleaning.removePS1Duplicates(df)[source]
Because there are many duplicate entries in PS1 host table, follow this hierarchy for

prioritizing which ones to keep: 1. If duplicate, remove non-primary detections 2. If still duplicate, remove NANs in yKronFlux, yskyErr, and yExtNSigma 3. If still duplicate, take the value with the smallest yKronFluxErr

Parameters:

df (Pandas DataFrame) – PS1 properties for candidate hosts.

Returns:

PS1 properties, after removing duplicates.

Return type:

Pandas DataFrame

stellarLocus

astro_ghost.stellarLocus.calc_7DCD(df)[source]
Calculates the color distance (7DCD) of objects in df from the

stellar locus from Tonry et al., 2012.

Parameters:

df (Pandas DataFrame) – Dataframe of PS1 objects.

Returns:

The same dataframe as input, with new column 7DCD.

Return type:

Pandas DataFrame

astro_ghost.stellarLocus.plotLocus(df, color=False, save=False, type='', timestamp='')[source]
Plots the color-color distribution of objects in df along with the Tonry et al., 2012

PS1 stellar locus.

Parameters:

  • df (Pandas DataFrame) – Dataframe of PS1 objects.

  • color (bool, optional) – If True, color objects by their distance from the stellar locus.

  • save (bool, optional) – If True, saves the image.

  • type (str, optional) – Can be “Gals” for galaxies or “Stars” for stars. Only relevant for coloring the distributions.

  • timestamp (str, optional) – Timestamp to append to the saved plot filename.

starSeparation

astro_ghost.starSeparation.separateStars_STRM(df, model_path='.', plot=False, verbose=False, starcut='gentle')[source]
Star-galaxy separation, using a random forest trained on the PS1-STRM-classified star

and galaxy labels given in Beck et al., 2021.

Parameters:

  • df (Pandas DataFrame) – Dataframe of PS1 sources.

  • model_path (str, optional) – Filepath to the saved random forest model.

  • plot (bool, optional) – If True, shows the separated stars and galaxies in Ap - Kron vs Ap Mag space.

  • verbose (bool, optional) – If true, print details of the classification routine.

  • starcut (str) – Labels corresponding to the classification thresholds required to classify a star as such. Options are ‘gentle’ (P>0.8), normal (P>0.5), and aggressive (P>0.3).

Returns:

A dataframe of sources classified as galaxies.

Return type:

Pandas DataFrame

Returns:

A dataframe of sources classified as stars.

Return type:

Pandas DataFrame

astro_ghost.starSeparation.separateStars_South(df, plot=0, verbose=0, starcut='gentle')[source]
Star-galaxy separation in the southern hemisphere, using a simple

SkyMapper_StarClass threshold cut.

Parameters:

  • df (Pandas DataFrame) – Dataframe of PS1 sources.

  • plot (bool, optional) – If True, shows the separated stars and galaxies in Ap - Kron vs Ap Mag space.

  • verbose (bool, optional) – If true, print details of the classification routine.

  • starcut (str) – Labels corresponding to the classification thresholds required to classify a star as such. Options are ‘gentle’ (P>0.8), normal (P>0.5), and aggressive (P>0.3).

Returns:

A dataframe of sources classified as galaxies.

Return type:

Pandas DataFrame

Returns:

A dataframe of sources classified as stars.

Return type:

Pandas DataFrame

hostMatching

astro_ghost.hostMatching.build_ML_df(dic, hostDF, transientDF)[source]

Consolidates the final host associations into a single dataframe.

Parameters:

  • dic (dictionary) – key,value pairs of transient name, list of associated host PS1 objIDs (should be one-to-one except where the association failed).

  • hostDF (Pandas DataFrame) – PS1 properties for all host galaxies.

  • transientDF (TNS properties for all transients.) – Pandas DataFrame

Returns:

The final consolidated DF of transient & host galaxy properties.

Return type:

Pandas DataFrame