GSoC 2017

0001-Allow-setting-asynchronous-file-prefetching-in-job-o.patch

From 0ba801e0c4f9e4d877f7ce926fc5ea1c7b917252 Mon Sep 17 00:00:00 2001
From: George <gpapadrosou@gmail.com>
Date: Thu, 20 Jul 2017 00:20:46 +0300
Subject: [PATCH] Allow setting asynchronous file prefetching in job options

This commit will allow the user to enable/disable asynchronous file prefetching in ROOT for a RootDomain.
The option can be set inside a job options file using InputPoolAttributes like so:
`ServiceMgr.AthenaPoolCnvSvc.InputPoolAttributes += [ "ASYNC_PREFETCHING = 1" ]`
---
 Database/APR/RootStorageSvc/src/RootDomain.cpp | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Database/APR/RootStorageSvc/src/RootDomain.cpp b/Database/APR/RootStorageSvc/src/RootDomain.cpp
index 907afa2..c58666b 100755
--- a/Database/APR/RootStorageSvc/src/RootDomain.cpp
+++ b/Database/APR/RootStorageSvc/src/RootDomain.cpp
@@ -19,6 +19,7 @@
 #include "TError.h"
 #include "TFile.h"
 #include "TROOT.h"
+#include "TEnv.h"
 #include "TTree.h"
 #include "TVirtualStreamerInfo.h"
 
@@ -58,6 +59,11 @@ DbStatus RootDomain::setOption(const DbOption& opt)  {
         }
         return sc;
       }
+      else if ( !strcasecmp(n, "ASYNC_PREFETCHING") )  {
+        gEnv->SetValue("TFile.AsyncPrefetching", 1);
+        int asyncPrefetching = gEnv->GetValue("TFile.AsyncPrefetching", 0);
+        return opt._getValue(asyncPrefetching);
+      }
       break;
     case 'D':
       if ( !strcasecmp(n, "DEFAULT_COMPRESSION") )  {
@@ -142,6 +148,9 @@ DbStatus RootDomain::getOption(DbOption& opt) const   {
       if ( !strcasecmp(n, "ABORT_LEVEL") )  {
         return opt._setValue(int(gErrorAbortLevel));
       }
+      else if ( !strcasecmp(n, "ASYNC_PREFETCHING") )  {
+        return opt._setValue((int)gEnv->GetValue("TFile.AsyncPrefetching", 0));
+      }
       break;
     case 'C':
       if ( !strcasecmp(n, "CLASS") )  {
-- 
2.10.1 (Apple Git-78)

README.md

During GSoC 2017, I worked for the ATLAS experiment at Cern in order to study the effect of asynchronous data prefetching mechanism implemented in the ROOT framework.

Firstly, some ROOT macro's were created in order to create a sample ROOT file and a macro to allow reading the ROOT file with different data prefetching methods. For the benchmarks, the sample file was accessed both locally and remotely through Cernbox's http server. Here is the repository for the ROOT benchmarks.

As a next step, the benchmarks were run by using the Athena framework, which embeds ROOT. Athena is an ATLAS simulation and data analysis software framework. Here is the repository which contains the scripts for the athena benchmarks.

A presentation about the results of the benchmarks can be found here.

In conclusion, the benchmarks showed that asynchronous prefetching leads to generally better performance than standard prefetching, and can lead up to 90% CPU utilization when the network speed gets closer to local disk read speed. For this reason, a merge request was made to the Athena framework, which allowed it's users to enable asynchronous prefetching inside a job options file. The code patch can be found here.