andre-merzky

#!/usr/bin/python

import troy

def main () :
    try:
        cpd = troy.pilot.ComputePilotDescription ()
        cpd['queue'] = 'large'

        cps = troy.pilot.ComputePilotService ()

andre-merzky

#include<saga/saga.hpp>


void work (int jobno, saga::session s, saga::job::service js)
{
  saga::url bfast_base_dir ("sftp://india.futuregrid.org/N/u/oweidner/software/bfast/");

  try
  {

andre-merzky

__author__    = "Ole Weidner"
__copyright__ = "Copyright 2012-2013, The SAGA Project"
__license__   = "MIT"


""" This examples shows how to run a job on a remote SLURM cluster
    using the 'SLURM' job adaptor.

    More information about the saga-python job API can be found at:
    http://saga-project.github.com/saga-python/doc/library/job/index.html

andre-merzky

import os
import sys
import saga
import getpass

REMOTE_HOST = "login1.stampede.tacc.utexas.edu"
USERNAME   = os.environ.get('USER')

def main():
    try:

andre-merzky

import os
import sys
import saga
import getpass

REMOTE_HOST = "login1.stampede.tacc.utexas.edu"
USERNAME    = os.environ.get('USER')
WORKINGDIR  = "/home1/02554/sagatut/XSEDETutorial/%s/SAGA" % USER_NAME

def main():

andre-merzky

import os
import sys
import time
import logging
import uuid
import saga
import pilot
import traceback
 

andre-merzky

Request #302541
inconsistent backend state ?

Andre Merzky
Jul 03 06:18

I have trouble to understand the semantics of various globus online commands, as the results do not reflect my expectations. In particular, I would expect that an operation which returns no error has actually *completed* on the backend, i.e. that the respective changes are committed onto the storage system. But in fact I get a different impression, and I am not sure if that is because of aggressive caching, or because of different, conflicting code paths, or something else. My best guess is that GO uses different protocols for different operations, and state is getting out of sync?

As example I include a globus online shell session below, which is an exemplary for the kind of problem I am encountering in different contexts (i.e. with different operations), too. [Some output lines of ls which refer to other people's files have been omitted in the session log, for clarity]

andre-merzky

{
    "agent_layout": {
        "agent.master": {
            "bridges": {
                "pubsub": {
                    "agent_state_pubsub": "local", 
                    "agent_unschedule_pubsub": "local"
                }, 
                "queue": {
                    "agent_executing_queue": "local", 

andre-merzky

#!/usr/bin/env python

import time
import subprocess
import multiprocessing

class A(multiprocessing.Process):

    def __init__(self):
        multiprocessing.Process.__init__(self)

andre-merzky

merzky    6053  4004  0 13:16 pts/15   00:00:00      \_ bash
merzky   15824  6053  4 13:27 pts/15   00:00:01          \_ python ./examples/getting_started_local.py
merzky   15856 15824  0 13:27 pts/16   00:00:00              \_ /bin/bash -i
merzky   16110 15824  0 13:27 pts/17   00:00:00              \_ /bin/sh -i
merzky   16376 15824  0 13:27 pts/19   00:00:00              \_ /bin/sh -i
merzky   16762 16376  0 13:27 pts/19   00:00:00              |   \_ /bin/sh /home/merzky/.saga/adaptors/shell_job/wrapper.sh
merzky   16505 15824  0 13:27 pts/22   00:00:00              \_ /bin/sh -i
--
merzky   16984     1  0 13:27 pts/19   00:00:00 /bin/sh /home/merzky/.saga/adaptors/shell_job//monitor.sh true && export RADICAL_PILOT_PROFILE=TRUE && mkdir -p /home/merzky/radical.pilot.sandbox/rp.session.cameo.merzky.016703.0009-pilot.0000/ && cd /home/merzky/radical.pilot.sandbox/rp.session.cameo.merzky.016703.0009-pilot.0000/ && ( /bin/bash -l bootstrap_1.sh  -d 'radical.utils-v0.35-38-gd0ead1b-work-am.tar.gz:saga-python-v