Skip to content

Instantly share code, notes, and snippets.

@nevencaplar

nevencaplar/gist:08d6532809090a9053b4a36b68ad38b0

Created November 9, 2021 17:45
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save nevencaplar/08d6532809090a9053b4a36b68ad38b0 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
reduceExposure.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/ --rerun ncaplar/PIPE2D-918_2 --id visit=67714 -j 12 -c doWriteCalexp=True isr.doFlat=False isr.doDefect=False repair.cosmicray.keepCRs=False --clobber-config
root INFO: Loading config overrride file '/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/obs_pfs/w.2021.45/config/reduceExposure.py'
CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
root INFO: Running: /tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/bin/reduceExposure.py /projects/HSC/PFS/Subaru --calib /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/ --rerun ncaplar/PIPE2D-918_2 --id visit=67714 -j 12 -c doWriteCalexp=True isr.doFlat=False isr.doDefect=False repair.cosmicray.keepCRs=False --clobber-config
CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
CameraMapper INFO: Loading exposure registry from /projects/HSC/PFS/Subaru/registry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /projects/HSC/PFS/Subaru/CALIB-PFI-20211108/calibRegistry.sqlite3
reduceExposure INFO: Processing [{'visit': 67714, 'dateObs': '2021-09-18', 'site': 'S', 'category': 'A', 'arm': 'r', 'spectrograph': 1, 'field': 'HEADER_TEST', 'ccd': 1, 'filter': 'r', 'expTime': 15.0, 'dataType': 'FLAT', 'taiObs': '2021-09-18T05:35:57.902', 'pfsDesignId': 512368543363980592, 'slitOffset': 0.0}]
reduceExposure INFO: Processing [{'visit': 67714, 'dateObs': '2021-09-18', 'site': 'S', 'category': 'A', 'arm': 'b', 'spectrograph': 1, 'field': 'HEADER_TEST', 'ccd': 0, 'filter': 'b', 'expTime': 15.0, 'dataType': 'FLAT', 'taiObs': '2021-09-18T05:35:57.957', 'pfsDesignId': 512368543363980592, 'slitOffset': 0.0}]
reduceExposure.isr INFO: Performing ISR on sensor {'visit': 67714, 'dateObs': '2021-09-18', 'site': 'S', 'category': 'A', 'arm': 'b', 'spectrograph': 1, 'field': 'HEADER_TEST', 'ccd': 0, 'filter': 'b', 'expTime': 15.0, 'dataType': 'FLAT', 'taiObs': '2021-09-18T05:35:57.957', 'pfsDesignId': 512368543363980592, 'slitOffset': 0.0}
reduceExposure.isr INFO: Performing ISR on sensor {'visit': 67714, 'dateObs': '2021-09-18', 'site': 'S', 'category': 'A', 'arm': 'r', 'spectrograph': 1, 'field': 'HEADER_TEST', 'ccd': 1, 'filter': 'r', 'expTime': 15.0, 'dataType': 'FLAT', 'taiObs': '2021-09-18T05:35:57.902', 'pfsDesignId': 512368543363980592, 'slitOffset': 0.0}
reduceExposure.isr INFO: Converting exposure to floating point values
reduceExposure.isr INFO: Converting exposure to floating point values
reduceExposure.isr INFO: Assembling CCD from amplifiers
reduceExposure.isr INFO: Assembling CCD from amplifiers
reduceExposure.isr INFO: Applying bias correction.
reduceExposure.isr INFO: Applying bias correction.
reduceExposure.isr INFO: Masking NAN value pixels.
reduceExposure.isr INFO: Masking NAN value pixels.
reduceExposure.isr INFO: Widening saturation trails.
reduceExposure.isr INFO: Widening saturation trails.
reduceExposure.isr INFO: Applying dark correction.
reduceExposure.isr INFO: Applying dark correction.
reduceExposure.isr INFO: Interpolating masked pixels.
reduceExposure.isr INFO: Interpolating masked pixels.
reduceExposure.isr WARN: No rough magnitude zero point set for filter b
reduceExposure.isr INFO: Setting rough magnitude zero point: 30.940228
reduceExposure.isr WARN: No rough magnitude zero point set for filter r
reduceExposure.isr INFO: Setting rough magnitude zero point: 30.940228
reduceExposure.repair INFO: Identified 104 cosmic rays.
reduceExposure.readLineList INFO: No lamps on; assuming sky.
reduceExposure.readLineList INFO: Filtered line lists for lamps: NaI,OH,OI
reduceExposure.centroidLines INFO: Measured 29587 line centroids
reduceExposure WARN: DetectorMap adjustment failed: Insufficient discrete wavelengths (1 vs 3 required)
reduceExposure FATAL: Failed on dataIds=[{'visit': 67714, 'dateObs': '2021-09-18', 'site': 'S', 'category': 'A', 'arm': 'b', 'spectrograph': 1, 'field': 'HEADER_TEST', 'ccd': 0, 'filter': 'b', 'expTime': 15.0, 'dataType': 'FLAT', 'taiObs': '2021-09-18T05:35:57.957', 'pfsDesignId': 512368543363980592, 'slitOffset': 0.0}]: RuntimeError: No good points
Traceback (most recent call last):
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
result = self.runTask(task, dataRef, kwargs)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
return task.runDataRef(dataRef, **kwargs)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/reduceExposure.py", line 251, in runDataRef
calibs = self.getSpectralCalibs(sensorRef, exposure, pfsConfig)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/reduceExposure.py", line 508, in getSpectralCalibs
phot = self.photometerLines.run(exposure, lines, detectorMap, pfsConfig, fiberTraces)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/photometerLines.py", line 99, in run
phot = self.photometerLines(exposure, referenceLines, detectorMap, pfsConfig, fiberTraces)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/photometerLines.py", line 175, in photometerLines
apCorr = self.apertureCorrection.run(exposure, pfsConfig, detectorMap, lines)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/apertureCorrections.py", line 121, in run
maxWavelength=maxWavelength)
File "/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/fitFocalPlane.py", line 109, in run
raise RuntimeError("No good points")
RuntimeError: No good points
reduceExposure.repair INFO: Identified 1961 cosmic rays.
reduceExposure.readLineList INFO: No lamps on; assuming sky.
reduceExposure.readLineList INFO: Filtered line lists for lamps: NaI,OH,OI
reduceExposure.centroidLines INFO: Measured 209510 line centroids
reduceExposure.adjustDetectorMap INFO: Final fit: chi2=104.773664 dof=32 xRMS=0.112057 yRMS=0.192259 (0.016600 nm) from 28/583 lines
reduceExposure.adjustDetectorMap INFO: Fit quality from reserved lines: chi2=1103665.054715 xRMS=3.497688 yRMS=3.480721 (0.300527 nm) from 65 lines (10.0%)
reduceExposure.adjustDetectorMap INFO: Softening errors by x=0.569162, y=0.581371 pixels (0.050196 nm) to yield chi^2/dof=1
reduceExposure.adjustDetectorMap INFO: Softened fit: chi2=30.527497 dof=32 xRMS=0.479383 yRMS=0.473752 (0.040904 nm) from 28 lines
reduceExposure.adjustDetectorMap INFO: Softened fit quality from reserved lines: chi2=15201.060910 xRMS=3.550787 yRMS=3.464628 (0.299138 nm) from 65 lines
reduceExposure.photometerLines.apertureCorrection.fit INFO: Fit focal plane function: chi^2=3950.021131 length=90/90 numSamples=613 numGood=6712 numBad=48286 numRejected=172
/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/apertureCorrections.py:313: RuntimeWarning: overflow encountered in power
ratio = 10.0**(-0.4*result.values)
/tigress/HSC/PFS/stack/20190925/stack/miniconda3-4.5.12-1172c30/Linux64/drp_stella/w.2021.45/python/pfs/drp/stella/apertureCorrections.py:314: RuntimeWarning: overflow encountered in multiply
newFlux = flux/ratio if invert else flux*ratio
reduceExposure.photometerLines INFO: Photometered 209510 lines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment