Skip to content

Instantly share code, notes, and snippets.

@nevencaplar

nevencaplar/fiber

Created April 30, 2019 14:09
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save nevencaplar/e194f448262b76b4f961fc53a5580015 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
fiber
# Build fiber trace (takes ``normal'' flats)
constructFiberTrace.py $TARGET --calib $TARGET/CALIB --rerun $RERUN/fiberTrace \
--id visit=11530..11534 $batchArgs -c || exit 1
+ constructFiberTrace.py /tigress/ncaplar/ReducedData/HgArFeb_2019_flat --calib /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB --rerun Apr30_2019/fiberTrace --id visit=11530..11534 --batch-type=smp --cores 8 --doraise -c
root INFO: Loading config overrride file '/home/ncaplar/custom_pipeline/obs_pfs/config/fiberTrace.py'
CameraMapper INFO: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
root INFO: Loading config overrride file '/home/ncaplar/custom_pipeline/obs_pfs/config/fiberTrace.py'
CameraMapper INFO: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
root INFO: Running: -c /tigress/ncaplar/ReducedData/HgArFeb_2019_flat --calib /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB --rerun Apr30_2019/fiberTrace --id visit=11530..11534 --doraise -c --noExit
CameraMapper INFO: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389528 INFO 2019-04-29T19:41:34.126-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389528 INFO 2019-04-29T19:41:34.127-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389532 INFO 2019-04-29T19:41:34.128-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389532 INFO 2019-04-29T19:41:34.128-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389530 INFO 2019-04-29T19:41:34.133-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389530 INFO 2019-04-29T19:41:34.133-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389527 INFO 2019-04-29T19:41:34.134-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389531 INFO 2019-04-29T19:41:34.134-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389527 INFO 2019-04-29T19:41:34.134-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389531 INFO 2019-04-29T19:41:34.134-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389533 INFO 2019-04-29T19:41:34.150-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389533 INFO 2019-04-29T19:41:34.150-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389529 INFO 2019-04-29T19:41:34.156-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389529 INFO 2019-04-29T19:41:34.156-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
fiberTrace INFO: Scatter processing
389528 INFO 2019-04-29T19:41:34.373-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389532 INFO 2019-04-29T19:41:34.379-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389531 INFO 2019-04-29T19:41:34.380-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389530 INFO 2019-04-29T19:41:34.381-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389527 INFO 2019-04-29T19:41:34.383-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389533 INFO 2019-04-29T19:41:34.398-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389529 INFO 2019-04-29T19:41:34.405-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389528 INFO 2019-04-29T19:41:34.604-0400 fiberTrace: Processing DataId(initialdata={'visit': 11531, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11531, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 14.998, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set()) on tiger2-sumire.princeton.edu:389528
389528 INFO 2019-04-29T19:41:34.604-0400 fiberTrace.isr: Performing ISR on sensor DataId(initialdata={'visit': 11531, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11531, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 14.998, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set())
389527 INFO 2019-04-29T19:41:34.616-0400 fiberTrace: Processing DataId(initialdata={'visit': 11530, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11530, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 15.004, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set()) on tiger2-sumire.princeton.edu:389527
389527 INFO 2019-04-29T19:41:34.616-0400 fiberTrace.isr: Performing ISR on sensor DataId(initialdata={'visit': 11530, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11530, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 15.004, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set())
389531 INFO 2019-04-29T19:41:34.620-0400 fiberTrace: Processing DataId(initialdata={'visit': 11534, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11534, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 15.004, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set()) on tiger2-sumire.princeton.edu:389531
389530 INFO 2019-04-29T19:41:34.620-0400 fiberTrace: Processing DataId(initialdata={'visit': 11533, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11533, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 14.998, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set()) on tiger2-sumire.princeton.edu:389530
389531 INFO 2019-04-29T19:41:34.621-0400 fiberTrace.isr: Performing ISR on sensor DataId(initialdata={'visit': 11534, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11534, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 15.004, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set())
389530 INFO 2019-04-29T19:41:34.621-0400 fiberTrace.isr: Performing ISR on sensor DataId(initialdata={'visit': 11533, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11533, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 14.998, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set())
389529 INFO 2019-04-29T19:41:34.637-0400 fiberTrace: Processing DataId(initialdata={'visit': 11532, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11532, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 15.005, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set()) on tiger2-sumire.princeton.edu:389529
389529 INFO 2019-04-29T19:41:34.638-0400 fiberTrace.isr: Performing ISR on sensor DataId(initialdata={'visit': 11532, 'dateObs': '2019-02-01', 'site': 'L', 'category': 'A', 'expId': 11532, 'arm': 'r', 'spectrograph': 1, 'field': 'FLAT', 'ccd': 1, 'filter': 'r', 'expTime': 15.005, 'dataType': 'flat', 'taiObs': '2019-02-01', 'pfiDesignId': 4296015889, 'slitOffset': 0.0}, tag=set())
389528 INFO 2019-04-29T19:41:38.640-0400 fiberTrace.repair: Identified 8 cosmic rays.
389529 INFO 2019-04-29T19:41:38.690-0400 fiberTrace.repair: Identified 19 cosmic rays.
389527 INFO 2019-04-29T19:41:38.715-0400 fiberTrace.repair: Identified 11 cosmic rays.
389531 INFO 2019-04-29T19:41:38.744-0400 fiberTrace.repair: Identified 9 cosmic rays.
389530 INFO 2019-04-29T19:41:38.806-0400 fiberTrace.repair: Identified 2 cosmic rays.
fiberTrace INFO: Scale on tiger2-sumire.princeton.edu:389526
CameraMapper INFO: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389529 INFO 2019-04-29T19:41:41.635-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389529 INFO 2019-04-29T19:41:41.635-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389528 INFO 2019-04-29T19:41:41.636-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389528 INFO 2019-04-29T19:41:41.636-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389527 INFO 2019-04-29T19:41:41.637-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389527 INFO 2019-04-29T19:41:41.637-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389533 INFO 2019-04-29T19:41:41.639-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389533 INFO 2019-04-29T19:41:41.640-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389531 INFO 2019-04-29T19:41:41.640-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389530 INFO 2019-04-29T19:41:41.641-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389531 INFO 2019-04-29T19:41:41.641-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389530 INFO 2019-04-29T19:41:41.641-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389532 INFO 2019-04-29T19:41:41.642-0400 CameraMapper: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
389532 INFO 2019-04-29T19:41:41.642-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389529 INFO 2019-04-29T19:41:41.920-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389528 INFO 2019-04-29T19:41:41.924-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389533 INFO 2019-04-29T19:41:41.926-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389527 INFO 2019-04-29T19:41:41.928-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389530 INFO 2019-04-29T19:41:41.933-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389531 INFO 2019-04-29T19:41:41.941-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
389532 INFO 2019-04-29T19:41:41.956-0400 CameraMapper: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
fiberTrace INFO: Scatter combination
fiberTrace INFO: Combining 5 inputs {'arm': 'r', 'spectrograph': 1, 'ccd': 1, 'filter': 'r', 'calibDate': '2019-02-01', 'visit0': 11530} on tiger2-sumire.princeton.edu:389526
fiberTrace INFO: 11 fiber traces found on combined flat
fiberTrace INFO: Median relative transmission of fiber 619 is 81073.445312
fiberTrace INFO: Median relative transmission of fiber 586 is 76244.093750
fiberTrace INFO: Median relative transmission of fiber 517 is 85138.164062
fiberTrace INFO: Median relative transmission of fiber 463 is 89397.390625
fiberTrace INFO: Median relative transmission of fiber 417 is 92393.156250
fiberTrace INFO: Median relative transmission of fiber 400 is 81488.718750
fiberTrace INFO: Median relative transmission of fiber 339 is 85402.187500
fiberTrace INFO: Median relative transmission of fiber 288 is 80907.445312
fiberTrace INFO: Median relative transmission of fiber 222 is 67279.945312
fiberTrace INFO: Median relative transmission of fiber 191 is 68082.906250
fiberTrace INFO: Median relative transmission of fiber 31 is 37314.148438
fiberTrace INFO: Writing {'arm': 'r', 'spectrograph': 1, 'ccd': 1, 'filter': 'r', 'calibDate': '2019-02-01', 'visit0': 11530} on tiger2-sumire.princeton.edu:389526
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_INPUT_2' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_CREATION_DATE' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_INPUT_0' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_INPUT_1' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_INPUT_4' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_INPUT_3' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
/tigress/HSC/PFS/stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/pyfits/3.4.0+11/lib/python/pyfits-3.4-py3.6-linux-x86_64.egg/pyfits/card.py:261: VerifyWarning: Keyword name 'CALIB_CREATION_TIME' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created.
keyword, VerifyWarning)
ingestCalibs.py $TARGET --calib $TARGET/CALIB --validity 1000 \
$TARGET/rerun/$RERUN/fiberTrace/FIBERTRACE/*.fits || exit 1
+ ingestCalibs.py /tigress/ncaplar/ReducedData/HgArFeb_2019_flat --calib /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB --validity 1000 /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/rerun/Apr30_2019/fiberTrace/FIBERTRACE/pfsFiberTrace-2019-02-01-011530-r1.fits
root INFO: Loading config overrride file '/home/ncaplar/custom_pipeline/obs_pfs/config/ingestCalibs.py'
CameraMapper INFO: Loading exposure registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/registry.sqlite3
CameraMapper INFO: Loading calib registry from /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/calibRegistry.sqlite3
ingestCalibs INFO: /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/rerun/Apr30_2019/fiberTrace/FIBERTRACE/pfsFiberTrace-2019-02-01-011530-r1.fits --<move>--> /tigress/ncaplar/ReducedData/HgArFeb_2019_flat/CALIB/FIBERTRACE/pfsFiberTrace-2019-02-01-011530-r1.fits
#( $CLEANUP && rm -r $TARGET/rerun/$RERUN/fiberTrace) || true
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment