Skip to content

Instantly share code, notes, and snippets.

View Athanaseus's full-sized avatar

Athanaseus Javas Ramaila Athanaseus

  • South African Radio Astronomy Observatory
  • Cape Town
  • 06:33 (UTC +02:00)
View GitHub Profile
aiobotocore==2.12.3
aiohttp==3.9.5
aioitertools==0.11.0
aiosignal==1.3.1
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
asciitree==0.3.3
astLib==0.11.10
astro-kittens==1.4.6
astro-tigger-lsm==1.7.3
This file has been truncated, but you can view the full file.
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: ### validated inputs
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: cab /home/aramaila/.virtualenvs/DDF/bin/DDF.py:
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Data.MS = ['/data6/aramaila/hii-bubbles/mgpls-reductions/msdir/T16R02C02-mgps_concat-ncorrs.ms']
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Data.ColName = FSLOPE_CORRECTED_DATA
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Data.Sort = True
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Predict.ColName = POINTING_MODEL
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Mode = Clean
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Name = mgpls_im_04/T16-concatinate-di1/im-di1
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Cubes = all
2024-01-27 11:39:15 STIMELA.mosaic.mosaic-di1 INFO: Output.Images = all
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: ### validated inputs
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: recipe 'MGPLS-HII-bubbles':
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: ms-base = /mnt/beegfs/mktplus/msdir
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: dir-out-base = mgpls-output-000
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: pa-rotate = False
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: suffix =
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: ms = /mnt/beegfs/mktplus/msdir/T16R02C04-corr-scan4-chan512-t16-XXYY.ms
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: running recipe 'MGPLS-HII-bubbles'
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: processing step 'flagsummary-stats-0'
2024-02-01 22:39:45 STIMELA.loop-pointings.mgpls-reductions INFO: (report initial flagging statistics)
# Test ms
5.2G ./T16R02C04-corr-scan4-chan512-t16-XXYY.ms
3.9G ./T16R02C06-corr-scan5-chan512-t16-XXYY.ms
5.2G ./T16R02C02-corr-scan3-chan512-t16-XXYY.ms
# Profiling
2024-02-01 19:25:52 STIMELA.loop-pointings INFO: run successful after 22:54:41
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: ### validated inputs
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: cab /home/aramaila/.virtualenvs/DDF/bin/DDF.py:
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Data.MS = ['/data6/aramaila/hii-bubbles/mgpls-reductions/msdir/T16R02C02-mgps_concat-ncorrs.ms']
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Data.ColName = FSLOPE_CORRECTED_DATA
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Data.Sort = True
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Predict.ColName = POINTING_MODEL
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Mode = Clean
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Name = mgpls_im_06/T16-concatinate-di1/im-di1
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Cubes = all
2024-02-01 10:58:46 STIMELA.mosaic.mosaic-di1 INFO: Output.Images = all
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: ### validated inputs
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: cab /home/aramaila/.virtualenvs/DDF/bin/DDF.py:
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Data.MS = ['/data6/aramaila/hii-bubbles/mgpls-reductions/msdir/T16R02C02-mgps_concat-ncorrs.ms']
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Data.ColName = FSLOPE_CORRECTED_DATA
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Data.Sort = True
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Predict.ColName = POINTING_MODEL
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Mode = Clean
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Name = mgpls_im_05/T16-concatinate-di1/im-di1
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Cubes = all
2024-01-30 20:35:04 STIMELA.mosaic.mosaic-di1 INFO: Output.Images = all
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ### validated inputs
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: cab flagdata:
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: spw =
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: mode = summary
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ms = /scratch3/users/athanaseus/msdir/T16R02C02-corr-scan3-chan512-t16-XXYY.ms
2024-01-31 11:06:41 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: singularity image /scratch3/users/athanaseus/images/quay.io-stimela2-casa:cc0.0.2.simg exists
### running /opt/slurm/bin/srun -J loop-pointings.mgpls-reductions.flagsummary-stats-0 /software/common/singularity/3.11.3/bin/singularity exec --containall --pwd /scratch3/users/athanaseus/runner --bind /scratch3/users/athanaseus/runner:/scratch3/user2024-01-31 11:06:57
@Athanaseus
Athanaseus / slum_1.log
Created January 31, 2024 09:04
NODE=1
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ### validated inputs
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: cab flagdata:
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: spw =
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: mode = summary
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: ms = /scratch3/users/athanaseus/msdir/T16R02C02-corr-scan3-chan512-t16-XXYY.ms
2024-01-31 10:59:31 STIMELA.loop-pointings.mgpls-reductions.flagsummary-stats-0 INFO: singularity image /scratch3/users/athanaseus/images/quay.io-stimela2-casa:cc0.0.2.simg exists
### running /opt/slurm/bin/srun -J loop-pointings.mgpls-reductions.flagsummary-stats-0 /software/common/singularity/3.11.3/bin/singularity exec --containall --pwd /scratch3/users/athanaseus/runner --bind /scratch3/users/athanaseus/runner:/scratch3/user2024-01-31 10:59:39
@Athanaseus
Athanaseus / DDF.md
Last active February 1, 2024 09:24
DDF MosaicKing

Mosaicking for the MeerKAT GPLS using Direction Dependent Faceting

Mosaicking 3 single scan pointings of 3 minutes each.

DDFacet command: ` DDF.py --Data-MS T16R02C02-corr-scan3-chan512-t16-XXYY.ms,T16R02C04-corr-scan4-chan512-t16-XXYY.ms,T16R02C06-corr-scan5-chan512-t16-XXYY.ms --Data-ColName DATA --Data-ChunkHours 3.0 --Data-Sort 1 --Predict-ColName POINTING_MODEL --Output-Mode Clean --Output-Name mgpls_im_01/T16R02-avg-mosaic-di1/im-di1 --Output-Cubes all --Output-Images all --Image-NPix 18000 --Image-Cell 1.0 --Facets-NFacets 9 --Weight-Robust -1.0 --Weight-SigmoidTaperInnerRolloffStrength 0 --Weight-SigmoidTaperOuterRolloffStrength 0 --RIME-DecorrMode FT --Parallel-NCPU 60 --Cache-Reset 1 --Beam-Model FITS --Beam-NBand 3 --Beam-CenterNorm 1 --Beam-Smooth 1 --Beam-FITSFile mktat_beams/new_mkt_pb_avg_$(xy)_$(reim).fits --Beam-FITSFeed xy --Beam-DtBeamMin 4.0 --Beam-FITSParAngleIncDeg 0.5 --Freq-NBand 3 --Freq-NDegridBand 12 --Deconv-Mode SSD2 --Deconv-MaxMajorIter 4 --Deconv-RMSFactor 3.0 --Deconv-P

@Athanaseus
Athanaseus / log-qc.txt
Created December 7, 2023 07:37
QUARTICAL
2023-11-22 15:15:23 | INFO | parser:log_final_config | Final configuration state:
------------------------------------input_ms------------------------------------
input_ms.path msdir/PicA-PictorA-corr.ms
input_ms.data_column DATA
input_ms.sigma_column None
input_ms.weight_column WEIGHT_SPECTRUM
input_ms.time_chunk 32
input_ms.freq_chunk 0
input_ms.is_bda False