Skip to content

Instantly share code, notes, and snippets.

@ashrithr
Last active November 16, 2021 09:48
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save ashrithr/58ce0be83be9c2a3c6c3 to your computer and use it in GitHub Desktop.
Save ashrithr/58ce0be83be9c2a3c6c3 to your computer and use it in GitHub Desktop.
YarnClient example for deleteing user applications
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.hadoop.yarn.api.records.ApplicationId;
import org.apache.hadoop.yarn.api.records.ApplicationReport;
import org.apache.hadoop.yarn.api.records.YarnApplicationState;
import org.apache.hadoop.yarn.client.api.YarnClient;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.exceptions.YarnException;
public class YarnClientTest {
public static void main(String[] args) throws YarnException, IOException {
if (args.length != 2) {
System.err.println("Expecting 2 arguments");
System.exit(1);
}
String type = args[0];
String app_id = null;
String user_id = null;
List<ApplicationId> user_applications = new ArrayList<ApplicationId>();
YarnApplicationState running_app = YarnApplicationState.RUNNING;
YarnConfiguration conf = new YarnConfiguration();
YarnClient c = YarnClient.createYarnClient();
c.init(conf);
c.start();
if (type.equalsIgnoreCase("app_id")) {
app_id = args[1];
for (ApplicationReport application: c.getApplications()) {
if (application.getApplicationId().equals(app_id) && application.getYarnApplicationState().equals(running_app))
c.killApplication(application.getApplicationId());
}
} else if (type.equalsIgnoreCase("user_id")) {
user_id = args[1];
for (ApplicationReport application: c.getApplications()) {
if (application.getUser().equals(user_id))
user_applications.add(application.getApplicationId());
if (!user_applications.isEmpty()) {
for (ApplicationId application_id: user_applications) {
if (application.getYarnApplicationState().equals(running_app))
c.killApplication(application_id);
}
} else {
System.out.println("The user_id " + user_id + " does not have any applications associated");
}
}
} else {
System.err.println("Expecting either 'app_id' or 'user_id' as parameter 1");
System.exit(1);
}
}
}
@iSultan
Copy link

iSultan commented Sep 20, 2016

Hi Ashrith,

Quick question regarding YarnClient. Does every user create only one YarnClient for all his/her applications? Or every application is associated with a YarnClient object?

I am trying to find a way to solve a problem I have.
There is user submits multiple jobs at the same time. I need to run an algorithm against all submitted applications before launching there Application Masters.

@Sathiyarajan
Copy link

@iSultan

Hi Ashrith,

Quick question regarding YarnClient. Does every user create only one YarnClient for all his/her applications? Or every application is associated with a YarnClient object?

I am trying to find a way to solve a problem I have.
There is user submits multiple jobs at the same time. I need to run an algorithm against all submitted applications before launching there Application Masters.

Any approaches did you find for the above scenario? I got stuck up in the same step in yarn-client api YarnClient.getApplications() method?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment