parejkoj

# pick your own favorite catalog and exposure file
import lsst.afw.table
file = "DATA_REPO/demo_collection/src/20130617/r/HSC-R/903342/src_HSC_r_HSC-R_903342_1_36_demo_collection.fits"
catalog=lsst.afw.table.SourceCatalog.readFits(file)

# What is the result of this?
catalog[0].getFootprint()

import lsst.afw.detection
catalog[0].getFootprint()

parejkoj

APDB preloading in general

• Configuration: separate pipeline/config to run on preload, analogous to existing PIPELINES_CONFIG?
• Run ap.assocaition.LoadDiaCatalogsTask as a standalone task inside the Prompt Processing code directly, make it a PipelineTask, or run it inside a new DatabasePreloadTask(PipelineTask)?
  • Either way, run LoadDiaCatalogsTask in preload, write diaObjects to a butler dataset ("{fakesType}{coaddName}Diff_diaObject_input"?), add that as an Input to DiaPipelineTask.
• Would need APDB access in DiaPipelineTask, LoadDiaCatalogsTask, PP itself [and future Solar System task, assumed analogous to LoadDiaCatalogsTask].
• Currently the Apdb object is a "pseudo-subtask" of DiaPipelineTask, set up via config fields and a "marker" Output conneciton to signal that the task completed it's APDB access.
• LoadDiaCatalogsTask currently takes apdb as run argument.

Strategy: passing apdb to __init__:

PipelineTasks must be constructable from just (config, log, initInputs); no such restriction fo

parejkoj

h1. Background

Exposure masks consist of a bitmask array, with a mapping of bit plane name to bit id. Mask plane definitions are kept in a single global state, with static members on {{Mask}} to modify that global name->bit number mapping. The documentation for the meaning of our current default planes is only available from the {{Mask.h}} header on our [C++ Doxygen page|http://doxygen.lsst.codes/stack/doxygen/x_masterDoxyDoc/classlsst_1_1afw_1_1image_1_1_mask.html#details]. Users cannot ask a Mask what the docs for a given plane are, nor is there any facility for documenting newly added planes (e.g. {{INJECTED_CORE}}).

After multiple false starts and dead ends, we ([~jbosch] and I) decided that the only reasonable way to manage mask plane definitions that include docstrings is to remove that global state and static member API. I've now implemented such an API in afw, but have not fully cleaned it up, and have not made changes to the rest of the Science Pipelines to reflect this API; I want to make sure thi

parejkoj

/**
 * An immutable label for a filter in an exposure or coadd.
 *
 * This class is agnostic to whether the stored label is for an abstract or
 * physical filter.
 * Details about the filter bandpass and transmission curve are stored in a TransmissionCurve object, which
 * can be looked up with the label via `butler.get("transmission_filter")`.
 */
class FilterLabel final : public typehandling::Storable {
public:

parejkoj

New proposal for handling calibrations in gen3

Definitions

Some definitions, since precision in this terminology can matter a lot.

• ingest: move/copy/symlink files from outside of a butler repository and butler.ingest them to a butler repository adding Dimension data to the registry as necessary.
• put: write in-memory objects to a butler repository using butler.put: this is distinct from ingest because the butler handles all registry updates and the objects have to be in memory, not as on-disk files.
• export/import: read data from one butler repository and put it into a different butler repository.
• raw: an un-modified image from a camera (including bias, dark, flat, science, etc.).

parejkoj

numbapro

Install via anaconda:

https://store.continuum.io/cshop/academicanaconda

and click the "Free" button next to "Anaconda Academic License" in the upper-right. Put in your contact information, and be sure to use your .edu email address. You will receive an email (possibly immediately) with your academic license and instructions, the gist of which are:

Copy the license file attached to the email into ~/.continuum/ conda update conda

parejkoj

#!/usr/bin/env python
"""
Merge many jpegs into a movie using ffmpeg.

Since ffmpeg requires all the files to numbered sequentially, with no gaps,
we sequentially named symlinks to the files, run ffmpeg on those files, then
delete the temporary symlinks.
"""
import glob
import subprocess