rbobbins/ci_code_coverage.md

## ci_code_coverage.md

      
    Raw
  

              ci_code_coverage.md
            
          
    #Continuous Integration and Code Coverage In Xcode#
Speaker: Matt Moriarity
###Agenda

What is Xcode Server
What's new in Xcode 7
Code coverage
Extending Xcode server

What is Xcode Server


Enables Continuous Integration

Enables collaboration and imporves software quality by regularly pulling and building app so you can find problems failures as soon as possible


Why xcode server?

Easy to setup
Deep integration with xcode
It knows what kinds of errors xcode can produce * surfaces more than just text logs
Can set it up to run on a suite of devices + simulators


Terminology

Scheme: Recipe for building your project. Important to Xcode server when stting up a bot
Bot: Builds your project on a schedule determined by you.
Integration: A single run of a bot


What's new in Xcode 7


Tabbed interface that allows you to edit bots w/o having to go through the whole setup flow
Source control improvements: surface more details. i.e, which repostiories your bot will checkout, you can decide which repositories and which branches to checkout. (how could you NOT do this before!?)
Security: Specifically SSH fingerprinting and (validating certificates)??. Requires you to explicitly trust any servers.
Updated reports: Test report is cleaned up, compacted. Easier to see assertion failures. Logs view has performance improvements! 1 log at a time, rather than all of them.
Improved issue tracking: compares test failures to previous integrations, so you can figure out when an issue was introduced. Users can "claim" issues * attaches your name to it, so that folks from your team will know you're working on it. You can silence issues for some period of time (i.e for known, temporary issues) so they don't clog logs.
Related to Xcode 7 features:

UI Testing creates a full window session for simulators. If you run tests on device, you'll see them running.
On demand resources: hosted by Xcode server.


Code coverage:


All about measuring the value of your tests. How do you know how good your tests are? Code coverage shoes which code is exercised by unit tests, and which areas are untested! Apple thinks this is really important for teams that are serious about testing.
Tightly integrated with LLVM. Countes the # of times that each expression is evaluated.
New tab in Xcode that shows code coverage data
Highlights expressions that have different test coverage on different devices (!!!)
History of your project that tracks changes in coverage. Also shows trends for code coverage (!!!). Helps you make informed decisions about how to spend development time.

Demo

(Basically a demo of the above-mentioned features)

Extending Xcode Server


Apple recognizes that you don't use just xcode server, so they've built integrations for external servivces
Triggers: custom actions, i.e. email notifications or scripts. Assumes Bash by default for scripts (but you can customize that)

Can run before or after integration.

Triggers that run before the integration actually run after source code is checked out.
Triggers that run after the integration can be gated based on success/failure of the build.
Lots of data about the integration is exposed through env variables. (i.e XCS_BOT_ID, XCS_INTEGRATION_ID)


Xcode server API

Build off "open, well-established standards".

Uses basic auth over HTTPS.
RESTful, uses JSON


Because of this, it's compatible with most scripting languages.
Communicates over port 20343. Examples:

GET https://my-server:20343/api/bots
GET https://my-server:20343/api/bots/:bot_id/integrations
Trigger an integration automatically: POST to /bots/:bot_id/integrations


Demo


I was distracted during this part.