Last active
January 12, 2016 19:40
-
-
Save pdurbin/814fd29916749523db9a to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html> | |
<head> | |
<title>2016-01-12 Dataverse API Testing</title> | |
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> | |
<style type="text/css"> | |
@import url(http://fonts.googleapis.com/css?family=Yanone+Kaffeesatz); | |
@import url(http://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic); | |
@import url(http://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic); | |
body { font-family: 'Droid Serif'; } | |
h1, h2, h3 { | |
font-family: 'Yanone Kaffeesatz'; | |
font-weight: normal; | |
} | |
.remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; } | |
.remark-slide-content { font-size: 25px; } | |
.title h1 { font-size: 70px; } | |
.agenda { font-size: 26px; } | |
.footnote { | |
font-size: 15px; | |
position: absolute; | |
bottom: 1em; | |
left: 3em; | |
right: 1em; | |
} | |
.quote { font-size: 18px; } | |
</style> | |
</head> | |
<body> | |
<textarea id="source"> | |
class: center, middle, title | |
# API Testing with REST Assured | |
<img src="dataverse-logo.png" width="300px"/></br> | |
<img src="rest-assured-logo.png" width="300px"/><br/> | |
</br> | |
.left[ | |
<img src="philipdurbin.jpg" width="60" align="left" style="border:5px solid transparent" /> | |
Philip Durbin ([@philipdurbin](https://twitter.com/philipdurbin)) | |
http://www.iq.harvard.edu/people/philip-durbin | |
] | |
.right[<img src="iqss-logo.png" width="100" align="left" style="border:5px solid transparent" />] | |
2016-01-12 | |
--- | |
# Agenda | |
- How to write and run API tests from Netbeans. | |
- How to execute tests on a server such as http://phoenix.dataverse.org | |
- Tips and tricks on using REST Assured. | |
- Overview of automated testing ideas: https://github.com/IQSS/dataverse/issues/2746 | |
--- | |
# What is an API? | |
- Application Programming Interface. | |
- Technically, APIs are everywhere. We're talking about testing REST (HTTP) APIs. | |
<img src="ojs.png" width="700px"/> | |
--- | |
# The good and bad of testing APIs | |
- Good | |
- Real world ("acceptance tests"). | |
- Can you create a user? | |
- Can you upload a file? | |
- Much easier than testing the UI. | |
- Is business logic centralized? Any GUI-only features? | |
- Bad | |
- Database, etc. must be running. | |
- Where are these tests going to run? | |
- Many steps required to test certain scenarios (i.e. deleting a file). | |
- Hard to measure coverage, unlike unit tests: https://coveralls.io/github/IQSS/dataverse | |
--- | |
# How do you test an API? | |
- One off tests | |
- Hit it with `curl` a few times in development. Hit the "up" arrow as needed. | |
- Automate the tests | |
- Use REST Assured or your favorite testing library. | |
- How often do you run them? Where? On your laptop? | |
- Run the automated tests on a real server | |
- Use Jenkins or similar to trigger the tests after a build. | |
--- | |
# Is there an API Test already? | |
<img src="navigate.png" width="600px"/> | |
--- | |
# DataversesIT ("integration test") | |
<img src="dataversesit.png" width="600px"/> | |
src/test/java/edu/harvard/iq/dataverse/api | |
--- | |
# Creating an Integration Test Step 1 | |
<img src="nav-to-ss.png" width="600px"/> | |
--- | |
# Creating an Integration Test Step 2 | |
<img src="create-ss.png" width="400px" align="right"/> | |
- FooTest vs. FooIT | |
- IT checkbox | |
- JUnit | |
- Code generation | |
--- | |
# Side Effects of API Testing | |
<img src="tests-ran-fine.jpg" width="350px"/> <img src="hippo-oath.jpg" width="350px"/> | |
- Do no harm. | |
- Clean up after yourself. | |
--- | |
# Running an integration test | |
<img src="run-test.png" width="700px"/> | |
mvn test -Dtest=DataversesIT | |
--- | |
# What is the test about? | |
### `testAttemptToCreateDuplicateAlias` | |
<img src="stay-saved.jpg" width="250px" align="right"/> | |
- Reads a bit like a user story. | |
- "Alice attempts to create a dataverse with an alias that's the upper case version of an existing alias. This should fail and report the error back to Alice." | |
<img src="only-one.png" width="150px"/> | |
cfa vs. CFA vs. CfA | |
https://github.com/IQSS/dataverse/issues/2598 | |
--- | |
# Is your API test doing too much? | |
#### `testCreateDatasetUploadFileDownloadFileEditTitle` | |
<img src="perl6.gif" width="300px" align="right"/> | |
Assert that: | |
- dataverse created | |
- dataset created via SWORD | |
- datasets listed via SWORD | |
- file uploaded | |
- download file by id | |
- delete file | |
- update title | |
- etc. | |
src/test/java/edu/harvard/iq/dataverse/api/SwordIT.java | |
--- | |
# JUnit Assertions | |
.center[ | |
<img src="good-bad.png" width="400px"/> | |
] | |
- `assertEquals(200, attemptToDoSomethingGood)` | |
- `assertEquals(400, attemptToDoSomethingBad)` | |
--- | |
# REST Assured | |
"Java DSL for easy testing of REST services" -- https://github.com/jayway/rest-assured | |
<img src="rest-assured-logo.png" width="200px"/> | |
"Jayway is a software innovation consultancy" -- http://jayway.com | |
<img src="jayway-logo.png" width="200px"/> | |
- status codes | |
- JSON response | |
- XML response | |
--- | |
# REST Assured Assertions | |
Response listDatasetsResponse = UtilIT.listDatasetsViaSword(dataverseAlias1, apiToken1); | |
listDatasetsResponse.prettyPrint(); | |
listDatasetsResponse.then().assertThat() | |
.statusCode(OK.getStatusCode()) | |
.body("feed.dataverseHasBeenReleased", equalTo("false")) | |
.body("feed.entry[0].title", equalTo(initialDatasetTitle)); | |
Assert that: | |
- Status code is OK (200). | |
- "dataverseHasBeenReleased" is false | |
- title matches expected string | |
Return type: `com.jayway.restassured.response.Response` | |
src/test/java/edu/harvard/iq/dataverse/api/SwordIT.java | |
--- | |
# Testing on a server | |
<img src="trend.png" width="700px"/> | |
https://build.hmdc.harvard.edu:8443 | |
--- | |
# Triggering tests from Jenkins | |
- Build | |
- Build a war file and package setup scripts. | |
- https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-build-4.2.3/ | |
- Deploy | |
- Drop database, deploy war file, setup, curl tests. | |
- https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-deploy/ | |
- Test | |
- REST Assured tests | |
- https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-4.2.3/ | |
--- | |
# Only a piece of the testing puzzle | |
Improve automated testing - https://github.com/IQSS/dataverse/issues/2746 | |
<img src="puzzle.jpg" width="350px" align="right"/> | |
- API Tests | |
- Unit Tests | |
- [The Testable Command](https://github.com/IQSS/dataverse/blob/1d7eadcb89ccebb4990c62757c84ec705f13853d/doc/theTestableCommand/TheTestableCommand.md) | |
- Selenium (browser) | |
- Performance | |
- Security | |
- etc. | |
.footnote[ | |
https://www.flickr.com/photos/horiavarlan/4273913228 | |
] | |
</textarea> | |
<script src="http://gnab.github.io/remark/downloads/remark-latest.min.js" type="text/javascript"> | |
</script> | |
<script type="text/javascript"> | |
var slideshow = remark.create(); | |
</script> | |
</body> | |
</html> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment