Skip to content

Instantly share code, notes, and snippets.

@stevenjswanson
Last active August 21, 2023 05:48
Show Gist options
  • Save stevenjswanson/b49d9571dbf6b9160555db3a0867410f to your computer and use it in GitHub Desktop.
Save stevenjswanson/b49d9571dbf6b9160555db3a0867410f to your computer and use it in GitHub Desktop.
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | ---> Starting the MUNGE Authentication service (munged) ...
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | * Starting MUNGE munged
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | ...done.
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | ---> Starting the Slurm Database Daemon (slurmdbd) ...
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | ERROR 2002 (HY000): Can't connect to server on 'mysql' (111)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | -- Waiting for database to become active ...
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | ERROR 2002 (HY000): Can't connect to server on 'mysql' (111)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | -- Waiting for database to become active ...
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | ERROR 2002 (HY000): Can't connect to server on 'mysql' (111)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | -- Waiting for database to become active ...
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | -- Database is now active ...
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug: Log file re-opened
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug: auth/munge: init: Munge authentication plugin loaded
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: init: mysql_connect() called for db slurm_acct_db
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: Attempting to connect to mysql:3306
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: accounting_storage/as_mysql: _check_mysql_concat_is_sane: MySQL server version is: 5.7.43
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: _check_database_variables: innodb_buffer_pool_size: 134217728
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: _check_database_variables: innodb_log_file_size: 50331648
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: _check_database_variables: innodb_lock_wait_timeout: 50
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: error: Database settings not recommended values: innodb_buffer_pool_size innodb_lock_wait_timeout
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: accounting_storage/as_mysql: init: Accounting storage MYSQL plugin loaded
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: Not running as root. Can't drop supplementary groups
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: ArchiveDir = /tmp
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: ArchiveScript = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: AuthAltTypes = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: AuthAltParameters = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: AuthInfo = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: AuthType = auth/munge
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: CommitDelay = 0
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: CommunicationParameters = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DbdAddr = slurmdbd
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DbdBackupHost = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DbdHost = slurmdbd
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DbdPort = 6819
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DebugFlags = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DebugLevel = 6
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DebugLevelSyslog = 10
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DefaultQOS = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: LogFile = /var/log/slurm/slurmdbd.log
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: MessageTimeout = 10
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: Parameters = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PidFile = /var/run/slurmdbd/slurmdbd.pid
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PluginDir = /usr/lib/x86_64-linux-gnu/slurm-wlm
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PrivateData = none
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeEventAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeJobAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeResvAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeStepAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeSuspendAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeTXNAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: PurgeUsageAfter = NONE
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: SlurmUser = slurm(990)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StorageBackupHost = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StorageHost = mysql
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StorageLoc = slurm_acct_db
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StorageParameters = (null)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StoragePort = 3306
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StorageType = accounting_storage/mysql
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: StorageUser = slurm
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: TCPTimeout = 2
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: TrackWCKey = 0
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: TrackSlurmctldDown= 0
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: acct_storage_p_get_connection: acct_storage_p_get_connection: request new connection 1
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: Attempting to connect to mysql:3306
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: slurmdbd version 21.08.5 started
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: running rollup at Mon Aug 21 05:45:59 2023
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_roll_usage: Everything rolled up
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug: REQUEST_PERSIST_INIT: CLUSTER:linux VERSION:9472 UID:990 IP:10.0.39.3 CONN:8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: acct_storage_p_get_connection: acct_storage_p_get_connection: request new connection 1
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: Attempting to connect to mysql:3306
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_REGISTER_CTLD: called in CONN 8 for linux(6817)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: slurmctld at ip:10.0.39.3, port:6817
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: dropping key time_start_end from table "linux_step_table"
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug: sending updates to linux at 10.0.39.3(6817) ver 9472
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: _slurm_connect: failed to connect to 10.0.39.3:6817: Connection refused
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: Error connecting slurm stream socket at 10.0.39.3:6817: Connection refused
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: error: update cluster: Connection refused to linux at 10.0.39.3(6817)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_GET_TRES: called in CONN 8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_GET_QOS: called in CONN 8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_GET_USERS: called in CONN 8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_GET_ASSOCS: called in CONN 8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_GET_RES: called in CONN 8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_FLUSH_JOBS: called in CONN 8 for linux
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_CLUSTER_TRES: called in CONN 8 for linux(1=2,2=2000,3=0,4=2,5=2,6=0,7=0,8=0)
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug: accounting_storage/as_mysql: as_mysql_cluster_tres: We don't have an entry for this machine linux most likely a first time running.
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_GET_FEDERATIONS: called in CONN 8
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_NODE_STATE_UP: NODE:c2 REASON:(null) TIME:1692596762
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_NODE_STATE_UP: NODE:c1 REASON:(null) TIME:1692596763
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_STEP_START: StepId=1.0 NAME:sleep SUBMIT:1692596790
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_start: as_mysql_job_start: called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_STEP_START: StepId=2.0 NAME:sleep SUBMIT:1692596790
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_start: as_mysql_job_start: called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_START: START CALL ID:1 NAME:interactive INX:0
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_start: as_mysql_job_start: called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_START: START CALL ID:2 NAME:interactive INX:0
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_start: as_mysql_job_start: called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_START: ELIGIBLE CALL ID:3 NAME:interactive
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_start: as_mysql_job_start: called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_STEP_COMPLETE: StepId=2.0 SUBMIT:1692596790
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_STEP_COMPLETE: StepId=1.0 SUBMIT:1692596790
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_COMPLETE: ID:1
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_complete: as_mysql_job_complete() called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_START: START CALL ID:3 NAME:interactive INX:5
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_start: as_mysql_job_start: called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_STEP_START: StepId=3.0 NAME:sleep SUBMIT:1692596790
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_STEP_COMPLETE: StepId=3.0 SUBMIT:1692596790
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_COMPLETE: ID:2
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_complete: as_mysql_job_complete() called
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: DBD_JOB_COMPLETE: ID:3
slurm-stack_slurmdbd.0.siis8b7lkt6h@142dev | slurmdbd: debug2: accounting_storage/as_mysql: as_mysql_job_complete: as_mysql_job_complete() called
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment