Minimal example for a profiled SnailTrail computation

main.md

Preparing the target computation

Add following dependencies to your Cargo.toml:

[dependencies]
logformat = { git = "..." }
timely_adapter = { git = "..." }

Profile your computation (load_balance_factor can also be replaced by 1):

fn main() {
    let mut args = std::env::args();
    let _ = args.next(); // bin name
    let load_balance_factor = args.next().expect("missing load balance factor").parse::<usize>().unwrap();
    let _ = args.next(); // "--"

    let args = args.collect::<Vec<_>>();
    timely::execute_from_args(std::env::args(), move |worker| {
        timely_adapter::connect::register_logger::<logformat::pair::Pair<u64, std::time::Duration>>(worker, load_balance_factor);

        // <dataflow logic>

        let timely_logger = worker.log_register().get::<timely::logging::TimelyEvent>("timely");
        if let Some(timely_logger) = &timely_logger {
            timely_logger.log(TimelyEvent::Text(String::from("")));
        }

        for i in 0..10 {
            // <provide input>
            
            input.advance_to(i + 1);
            input.flush();
            while probe.less_than(input.time()) {
                worker.step();
            }

            if let Some(timely_logger) = &timely_logger {
                timely_logger.log(TimelyEvent::Text(String::from(""));
            }
        }
    })
    .unwrap();
}

Profiling the target computation

Offline (write to file): Just run the profiled computation. Online (write to socket): Start a SnailTrail instance (follow https://github.com/li1/snailtrail#getting-started). Pass SNAILTRAIL_ADDR=<IP:8000> as environment variable and run the profiled computation.