Skip to content

Instantly share code, notes, and snippets.

View Athanaseus's full-sized avatar

Athanaseus Javas Ramaila Athanaseus

  • South African Radio Astronomy Observatory
  • Cape Town
  • 12:53 (UTC +02:00)
View GitHub Profile
2024-08-23 19:51:16 STIMELA.tron.lightcurves.extract INFO: selecting recipe steps for (sub)recipe: [bold green]extract[/bold green]
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: ### validated inputs
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: recipe 'extract':
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: catalog = ar1-output-004/T16R02C04-MFS-image.mastercat.ecsv
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: ncpu = 36
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: nlc = 100
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: dir-out = ar1-output-004
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: cds-list = ['ar1-output-004/cubes/cube-1-252s.zarr']
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: running recipe 'extract'
2024-08-23 20:06:36 STIMELA.tron.lightcurves.extract INFO: for loop iteration 0: cds = ar1-output-004/cubes/cube-1-252s.zarr
aiobotocore==2.12.3
aiohttp==3.9.5
aioitertools==0.11.0
aiosignal==1.3.1
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
asciitree==0.3.3
astLib==0.11.10
astro-kittens==1.4.6
astro-tigger-lsm==1.7.3
This file has been truncated, but you can view the full file.
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: ### validated inputs
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: cab /home/aramaila/.virtualenvs/DDF/bin/DDF.py:
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Data.MS = ['/data6/aramaila/hii-bubbles/mgpls-reductions/msdir/T16R02C02-mgps_concat-ncorrs.ms']
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Data.ColName = FSLOPE_CORRECTED_DATA
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Data.Sort = True
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Predict.ColName = POINTING_MODEL
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Mode = Clean
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Name = mgpls_im_04/T16-concatinate-di1/im-di1
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Cubes = all
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Images = all
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: ### validated inputs
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: recipe 'MGPLS-HII-bubbles':
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: ms-base = /mnt/beegfs/mktplus/msdir
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: dir-out-base = mgpls-output-000
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: pa-rotate = False
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: suffix =
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: ms = /mnt/beegfs/mktplus/msdir/T16R02C04-corr-scan4-chan512-t16-XXYY.ms
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: running recipe 'MGPLS-HII-bubbles'
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: processing step 'flagsummary-stats-0'
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: (report initial flagging statistics)
# Test ms
5.2G ./T16R02C04-corr-scan4-chan512-t16-XXYY.ms
3.9G ./T16R02C06-corr-scan5-chan512-t16-XXYY.ms
5.2G ./T16R02C02-corr-scan3-chan512-t16-XXYY.ms
# Profiling
2024-02-01 19:25:52 STIMELA.loop-pointings INFO: run successful after 22:54:41
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: ### validated inputs
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: cab /home/aramaila/.virtualenvs/DDF/bin/DDF.py:
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Data.MS = ['/data6/aramaila/hii-bubbles/mgpls-reductions/msdir/T16R02C02-mgps_concat-ncorrs.ms']
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Data.ColName = FSLOPE_CORRECTED_DATA
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Data.Sort = True
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Predict.ColName = POINTING_MODEL
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Mode = Clean
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Name = mgpls_im_06/T16-concatinate-di1/im-di1
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Cubes = all
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Images = all
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: ### validated inputs
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: cab /home/aramaila/.virtualenvs/DDF/bin/DDF.py:
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Data.MS = ['/data6/aramaila/hii-bubbles/mgpls-reductions/msdir/T16R02C02-mgps_concat-ncorrs.ms']
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Data.ColName = FSLOPE_CORRECTED_DATA
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Data.Sort = True
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Predict.ColName = POINTING_MODEL
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Mode = Clean
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Name = mgpls_im_05/T16-concatinate-di1/im-di1
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Cubes = all
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Images = all
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ### validated inputs
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: cab flagdata:
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: spw =
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: mode = summary
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ms = /scratch3/users/athanaseus/msdir/T16R02C02-corr-scan3-chan512-t16-XXYY.ms
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: singularity image /scratch3/users/athanaseus/images/quay.io-stimela2-casa:cc0.0.2.simg exists
### running /opt/slurm/bin/srun -J loop-pointings.mgpls-reductions.flagsummary-stats-0 /software/common/singularity/3.11.3/bin/singularity exec --containall --pwd /scratch3/users/athanaseus/runner --bind /scratch3/users/athanaseus/runner:/scratch3/user2024-01-31 11:06:57
@Athanaseus
Athanaseus / slum_1.log
Created January 31, 2024 09:04
NODE=1
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ### validated inputs
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: cab flagdata:
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: spw =
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: mode = summary
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ms = /scratch3/users/athanaseus/msdir/T16R02C02-corr-scan3-chan512-t16-XXYY.ms
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: singularity image /scratch3/users/athanaseus/images/quay.io-stimela2-casa:cc0.0.2.simg exists
### running /opt/slurm/bin/srun -J loop-pointings.mgpls-reductions.flagsummary-stats-0 /software/common/singularity/3.11.3/bin/singularity exec --containall --pwd /scratch3/users/athanaseus/runner --bind /scratch3/users/athanaseus/runner:/scratch3/user2024-01-31 10:59:39
@Athanaseus
Athanaseus / DDF.md
Last active February 1, 2024 09:24
DDF MosaicKing

Mosaicking for the MeerKAT GPLS using Direction Dependent Faceting

Mosaicking 3 single scan pointings of 3 minutes each.

DDFacet command: ` DDF.py --Data-MS T16R02C02-corr-scan3-chan512-t16-XXYY.ms,T16R02C04-corr-scan4-chan512-t16-XXYY.ms,T16R02C06-corr-scan5-chan512-t16-XXYY.ms --Data-ColName DATA --Data-ChunkHours 3.0 --Data-Sort 1 --Predict-ColName POINTING_MODEL --Output-Mode Clean --Output-Name mgpls_im_01/T16R02-avg-mosaic-di1/im-di1 --Output-Cubes all --Output-Images all --Image-NPix 18000 --Image-Cell 1.0 --Facets-NFacets 9 --Weight-Robust -1.0 --Weight-SigmoidTaperInnerRolloffStrength 0 --Weight-SigmoidTaperOuterRolloffStrength 0 --RIME-DecorrMode FT --Parallel-NCPU 60 --Cache-Reset 1 --Beam-Model FITS --Beam-NBand 3 --Beam-CenterNorm 1 --Beam-Smooth 1 --Beam-FITSFile mktat_beams/new_mkt_pb_avg_$(xy)_$(reim).fits --Beam-FITSFeed xy --Beam-DtBeamMin 4.0 --Beam-FITSParAngleIncDeg 0.5 --Freq-NBand 3 --Freq-NDegridBand 12 --Deconv-Mode SSD2 --Deconv-MaxMajorIter 4 --Deconv-RMSFactor 3.0 --Deconv-P