Skip to content

Instantly share code, notes, and snippets.

@jhoguet
Last active January 12, 2017 17:41
Show Gist options
  • Save jhoguet/72da878e3aecd061025706b77233344b to your computer and use it in GitHub Desktop.
Save jhoguet/72da878e3aecd061025706b77233344b to your computer and use it in GitHub Desktop.
scraping data from thoughtworks
{
"radars" : {
"ThoughtWorks" : {
"quandrants" : {
"Languages And Frameworks" : [
{
"name": "adopt",
"blips": [
{
"name": "Ember.js",
"description": "If you are faced with building a single-page application (SPA) and trying to choose a framework to build with, <strong><a href=\"http://emberjs.com/\">Ember.js</a></strong> has emerged as a leading choice. Our teams praise Ember for its highly productive developer experience, with far fewer surprises than other frameworks such as <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. The Ember CLI build tooling is a haven in the storm of JavaScript build tools, and the Ember core team and community are highly active and responsive."
},
{
"name": "React.js",
"description": "In the avalanche of front-end JavaScript frameworks, <a href=\"http://facebook.github.io/react/\"><strong>React.js</strong></a> stands out due to its design around a reactive data flow. Allowing only one-way data binding greatly simplifies the rendering logic and avoids many of the issues that commonly plague applications written with other frameworks. We're seeing the benefits of React.js on a growing number of projects, large and small, while at the same time we continue to be concerned about the state and the future of other popular frameworks like <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. This has led to React.js becoming our default choice for JavaScript frameworks."
},
{
"name": "Redux",
"description": "With the increasing complexity of single-page JavaScript applications, we have seen a more pressing need to make client-side state management predictable. <a href=\"http://redux.js.org/\"><strong>Redux</strong></a>, with its <a href=\"http://redux.js.org/docs/introduction/ThreePrinciples.html\">three principles</a> of restrictions for updating state, has proven to be invaluable in a number of projects we have implemented. <a href=\"https://egghead.io/courses/getting-started-with-redux\">Getting Started with Redux</a> and <a href=\"https://egghead.io/courses/building-react-applications-with-idiomatic-redux\">idiomatic Redux</a> tutorials are a good starting point for new and experienced users. Its minimal library design has spawned a rich set of tools, and we encourage you to check out the <a href=\"https://github.com/markerikson/redux-ecosystem-links\">redux-ecosystem-links</a> project for examples, middleware and utility libraries. We also particularly like the testability story: Dispatching actions, state transitions and rendering can be unit-tested separately from one another and with minimal amounts of mocking."
},
{
"name": "Spring Boot",
"description": "A lot of work has gone into <a href=\"http://projects.spring.io/spring-boot\"><strong>Spring Boot</strong></a> to reduce complexity and dependencies, which largely alleviates our previous reservations. If you live in a Spring ecosystem and are moving to microservices, Spring Boot is now the obvious choice. For those not in Springland, <a href=\"/radar/languages-and-frameworks/dropwizard\">Dropwizard</a> is also worthy of serious consideration."
},
{
"name": "Butterknife",
"description": "<strong><a href=\"https://github.com/JakeWharton/butterknife\">Butterknife</a></strong> is a field&nbsp;and method&nbsp;binding view-injection library. It allows the injection of arbitrary objects, views and listeners, thereby ensuring cleaner code with reduced glue code for Android development. With Butterknife, multiple views can be grouped into a list or array with common actions applied to the views simultaneously, without heavy reliance on XML configurations. Our project teams have used this library and benefited from its simplicity and ease of use."
},
{
"name": "Dagger",
"description": "With the increased need for Android-based applications, <strong><a href=\"http://google.github.io/dagger/\">Dagger</a> </strong>offers a fully static, compile-time dependency-injection framework. Dagger's strictly generated implementation and nonreliance on reflection-based solutions addresses many of the performance and development issues,&nbsp;thereby making it suitable for Android development. With Dagger, there is full traceability with easy debugging because the entire call stack for provision and creation is made available."
},
{
"name": "Dapper",
"description": "<a href=\"https://github.com/StackExchange/dapper-dot-net\"><strong>Dapper</strong></a> is a minimal, lightweight ORM of sorts for .NET. Rather than trying to write the SQL queries for you, Dapper maps SQL queries to dynamic objects. Though it's not brand new, Dapper has steadily gained acceptance from ThoughtWorks teams working in .NET. For the C# programmer, it removes some of the drudgery of mapping relational queries to objects while still allowing complete control over the SQL or stored procedures."
},
{
"name": "Elixir",
"description": "Interest in the <a href=\"http://elixir-lang.org/\"><strong>Elixir</strong></a> programming language continues to build. Increasingly, we see it used in serious projects and hear feedback from developers who find its Actor model to be robust and very fast. Elixir, which is built on top of the Erlang virtual machine, is showing promise for creating highly concurrent and fault-tolerant systems. Elixir has distinctive features such as the Pipe operator, which allows developers to build a pipeline of functions as you would in the UNIX command shell. The shared byte code allows Elixir to interoperate with Erlang and leverage existing libraries while supporting tools such as the Mix build tool, the IEx interactive shell and the <a href=\"http://elixir-lang.org/docs/stable/ex_unit/ExUnit.html\">ExUnit</a> unit-testing framework."
},
{
"name": "Enzyme",
"description": "We’ve been enjoying the rapid component-level UI testing that <a href=\"http://airbnb.io/enzyme/\"><strong>Enzyme</strong></a> provides for <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> applications. Unlike many other snapshot-based testing frameworks, Enzyme allows you to test without doing on-device rendering, which results in faster and more granular testing. This is a contributing factor in our ability to massively reduce the amount of functional testing we find we have to do in React applications."
},
{
"name": "Immutable.js",
"description": "Immutability is often emphasized in the functional programming paradigm, and most languages have the ability to create immutable objects—objects that can't be changed once created. <strong><a href=\"https://facebook.github.io/immutable-js/\">Immutable.js</a></strong> is a library for JavaScript that provides many persistent immutable data structures, which are highly efficient on modern JavaScript virtual machines. Immutable.js objects are, however, not normal JavaScript objects, so references to JavaScript objects from immutable objects should be avoided. More teams are using this library for tracking mutation and maintaining state in production. We recommend that developers investigate this library, especially when it's combined with the rest of the Facebook stack."
},
{
"name": "Phoenix",
"description": "Some of our ThoughtWorks teams have had very positive experiences with <strong>Phoenix</strong>, a server-side web MVC framework written in <a href=\"/radar/languages-and-frameworks/elixir\">Elixir</a>. In addition to being streamlined and easy to use, Phoenix takes advantage of Elixir to be extremely fast. For some developers, Phoenix evokes the joy they experienced when first discovering Ruby and Rails. Although the ecosystem of libraries for Phoenix is not as extensive as for some more mature frameworks, it should benefit from the continuing success and growth of support for Elixir."
},
{
"name": "Quick and Nimble",
"description": "Most of our iOS teams are now using the <strong><a href=\"https://github.com/Quick/Quick\">Quick</a> and <a href=\"https://github.com/Quick/Nimble\">Nimble</a></strong> pairing for their unit tests. In the <a href=\"http://rspec.info/\">RSpec</a> family of behavior-driven development (BDD) testing tools, it provides very readable tests (with describe blocks) across <a href=\"/radar/languages-and-frameworks/swift\">Swift</a> and Objective-C and has good support for asynchronous testing."
},
{
"name": "React Native",
"description": "We are seeing continued success with <a href=\"https://facebook.github.io/react-native/\"><strong>React Native</strong></a> for rapid cross-platform mobile development. Despite some churn as it undergoes continuing development, the advantages of trivial integration between native and nonnative code and views, the rapid development cycle (instant reload, chrome debugging, Flexbox layout) and general growth of the React style is winning us over. As with many frameworks, care needs to be taken to keep your code well structured, but diligent use of a tool like <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> really helps here."
},
{
"name": "Robolectric",
"description": "In the Android application-development world, <strong><a href=\"http://robolectric.org/\">Robolectric</a></strong> is a unit-testing framework that has been used by multiple teams within our technical community. It offers the best option among those available for writing real unit tests that extend or interact directly with Android components and support JUnit tests. We caution, though, that because it is an implementation of the Android SDK, there might be device-specific issues for some tests that pass in Robolectric. To manually mock all the Android dependencies, ensuring only test of the system-in-test will require a lot of complex code, and this framework addresses this effectively."
},
{
"name": "Aurelia",
"description": "<a href=\"http://aurelia.io/\"><strong>Aurelia</strong></a> is considered the next-generation JavaScript client framework and was written using a modern version of JavaScript: ECMAScript 2016. Aurelia was created by Rob Eisenberg, the creator of <a href=\"http://durandaljs.com/\">Durandal</a>. He left the <a href=\"https://angular.io/\">Angular 2.0</a> core team to dedicate his time to this project. The great thing about Aurelia is that it's highly modular, contains simple small libraries and is designed to be customized easily. Aurelia follows the pattern of convention over configuration, which enables easier production and consumption of modules, but there are no strong conventions that you have to adhere to. Aurelia has a large community, and in the project website you can learn more by using the tutorials."
},
{
"name": "ECMAScript 2017",
"description": ""
},
{
"name": "Elm",
"description": "We have been prompted to reconsider <a href=\"http://elm-lang.org/\"><strong>Elm</strong></a> because of the rapid adoption of <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> framework. Elm—the original inspiration for Redux—offers the view componentization and reactiveness of <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> along with the predictable state of Redux in a compiled, strongly typed functional language. Elm is written in Haskell and has a Haskell-like syntax but compiles down to HTML, CSS and JavaScript for the browser. JavaScript programmers rushing to embrace React.js and Redux might want to also consider Elm as a type-safe alternative for some applications."
},
{
"name": "GraphQL",
"description": "When we look at REST implementations in the wild, we frequently see REST misused to naively retrieve object graphs through chatty interactions between client and server. Facebook's <a href=\"https://github.com/facebook/graphql\"><strong>GraphQL</strong></a> is an interesting alternative to REST that might be a better approach for this very common use case. As a protocol for remotely retrieving object graphs, GraphQL has received enormous attention recently. One of GraphQL's most interesting features is its consumer-oriented nature: The structure of a response is driven entirely by the client, not the server. This decouples the consumer and forces the server to obey Postel's law. Client implementations are now available in many programming languages, but we have seen a flurry of interest of Facebook's <a href=\"https://facebook.github.io/relay/\">Relay</a>, a JavaScript framework that was designed to support the <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> stateless component model."
},
{
"name": "JuMP",
"description": "<strong><a href=\"https://github.com/JuliaOpt/JuMP.jl\">JuMP</a></strong> is a domain-specific language for <a href=\"https://en.wikipedia.org/wiki/Mathematical_optimization\">mathematical optimizations</a> in <a href=\"http://julialang.org/\">Julia</a>. JuMP defines a common API called <a href=\"https://github.com/JuliaOpt/MathProgBase.jl\">MathProgBase</a> and enables users to write solver-agnostic code in Julia. Currently supported solvers include <a href=\"http://artelys.com/en/optimization-tools/knitro\">Artelys Knitro</a>, <a href=\"https://projects.coin-or.org/Bonmin\">Bonmin</a>, <a href=\"https://projects.coin-or.org/Cbc\">Cbc</a>, <a href=\"https://projects.coin-or.org/Clp\">Clp</a>, <a href=\"https://projects.coin-or.org/Couenne\">Couenne</a>, <a href=\"http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/\">CPLEX</a>, <a href=\"https://github.com/ifa-ethz/ecos\">ECOS</a>, <a href=\"http://www.fico.com/en/products/fico-xpress-optimization-suite\">FICO Xpress</a>, <a href=\"http://www.gnu.org/software/glpk/\">GLPK</a>, <a href=\"http://www.gurobi.com\">Gurobi</a>, <a href=\"https://projects.coin-or.org/Ipopt\">Ipopt</a>, <a href=\"http://www.mosek.com/\">MOSEK</a>, <a href=\"http://ab-initio.mit.edu/wiki/index.php/NLopt\">NLopt</a> and <a href=\"https://github.com/cvxgrp/scs\">SCS</a>. One other benefit is the implementation of automatic differentiation technique in reverse mode to compute derivatives so users are not limited to the standard operators like sin, cos, log and sqrt but can also implement their own custom objective functions in Julia."
},
{
"name": "Physical Web",
"description": "We have been intrigued by the <strong><a href=\"https://google.github.io/physical-web/\">Physical Web</a></strong> standard created by Google. The idea of Physical Web is simple—beacons broadcast a URL—but the possibilities are broad. Basically, this is a way to annotate the physical world, tying objects and locations into the digital realm. The current transport mechanism is <a href=\"https://github.com/google/eddystone/tree/master/eddystone-url\">Eddystone URLs</a> over Bluetooth LE, and sample clients are available. Although there are obvious security concerns with following randomly discovered links, we are most interested in use cases with customized clients where you can filter or proxy the URLs as required."
},
{
"name": "Rapidoid",
"description": "<strong><a href=\"http://www.rapidoid.org/\">Rapidoid</a></strong> is a collection of web framework modules, including a fast low-level HTTP server implemented from scratch on top of Java NIO. Clever usage of off-heap input/output buffers, object pools and thread-local data structures provide Rapidoid an edge over other NIO-based servers like <a href=\"http://netty.io/\">Netty</a>. Being a fairly new project, Rapidoid has yet to implement a few features like built-in cache and SSL support; we suggest you check the <a href=\"https://github.com/rapidoid/rapidoid\">roadmap</a> for updates."
},
{
"name": "Recharts",
"description": "We've been enjoying how <strong><a href=\"http://recharts.org/\">Recharts</a></strong> integrates <a href=\"/radar/tools/d3\">D3</a> charts into <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> in a clean and declarative manner."
},
{
"name": "ReSwift",
"description": "We are excited that the <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> paradigm has made its way to Swift-land in the form of <a href=\"http://reswift.github.io/ReSwift\"><strong>ReSwift</strong></a>. We’ve found real benefits in the simplicity and readability of codebases once state and state changes are managed in a central place and common idiom. This also helps with building \"offline first\" applications."
},
{
"name": "Three.js",
"description": "Despite the fervor surrounding the spate of new headsets, we believe there are many VR and AR scenarios that make sense in the browser, particularly on mobile. Given this trend, we have seen an uptick in usage of <a href=\"https://threejs.org/\"><strong>Three.js</strong></a>, a powerful JavaScript visualization and 3D rendering framework. The growth in support for WebGL, which it is based on, has helped adoption, as has the vibrant community supporting this open source project."
},
{
"name": "Vue.js",
"description": "In the ever-changing world of front-end JavaScript frameworks, <a href=\"https://vuejs.org/\"><strong>Vue.js</strong></a> has gained a lot of ground as a lightweight alternative to <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. It is designed to be a very flexible—and a less opinionated—library that offers a set of tools for building interactive web interfaces around concepts like modularity, components and reactive data flow. It has a low learning barrier, which makes it interesting for junior developers and beginners. Vue.js itself is not a full-blown framework; it is focused on the view layer only and therefore is easy to integrate with other libraries or existing projects."
},
{
"name": "WebRTC",
"description": "Widespread adoption of AR/VR as a collaboration and communication medium requires a modern and readily available video streaming platform. <strong><a href=\"https://webrtc.org/\">WebRTC</a></strong> is an emerging standard for real-time communication between browsers that enables video streaming within commonly available web technologies. The range of browsers that support this standard is increasing, but Microsoft and Apple have been slow to adopt WebRTC in their proprietary browsers. If momentum continues to build, WebRTC could form the future foundation for AR/VR collaboration on the web."
},
{
"name": "AngularJS",
"description": "<a href=\"https://angularjs.org/\"><strong>AngularJS</strong></a> helped revolutionize the world of single-page JavaScript applications, and we have delivered many projects successfully with it over the years. However, we are no longer recommending it (v1) for teams starting fresh projects. We prefer the ramp-up speed and more maintainable codebases we are seeing with <a href=\"/radar/languages-and-frameworks/ember-js\">Ember</a> and <a href=\"/radar/languages-and-frameworks/react-js\">React</a>, particularly in conjunction with <a href=\"/radar/languages-and-frameworks/redux\">Redux</a>."
},
{
"name": "JSPatch",
"description": "Many iOS developers are using <strong><a href=\"https://github.com/bang590/JSPatch\">JSPatch</a></strong> to dynamically patch their apps. When a JSPatch-enabled app runs, it loads a chunk of JavaScript (potentially via an insecure HTTP connection) and then bridges to the main Objective-C application code to change behavior, fix bugs, and so on. While convenient, we think monkey-patching live apps is a bad idea and should be avoided. When doing any amount of incremental patching, it's very important that your testing process matches what end users will experience, in order to properly validate functionality. An alternative approach is to use <a href=\"/radar/languages-and-frameworks/react-native\">React Native</a> for the app and <a href=\"https://apphub.io/\">AppHub</a> and <a href=\"https://microsoft.github.io/code-push/\">CodePush</a> to push small updates and new features."
}
]
},
{
"name": "trial",
"blips": [
{
"name": "Ember.js",
"description": "If you are faced with building a single-page application (SPA) and trying to choose a framework to build with, <strong><a href=\"http://emberjs.com/\">Ember.js</a></strong> has emerged as a leading choice. Our teams praise Ember for its highly productive developer experience, with far fewer surprises than other frameworks such as <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. The Ember CLI build tooling is a haven in the storm of JavaScript build tools, and the Ember core team and community are highly active and responsive."
},
{
"name": "React.js",
"description": "In the avalanche of front-end JavaScript frameworks, <a href=\"http://facebook.github.io/react/\"><strong>React.js</strong></a> stands out due to its design around a reactive data flow. Allowing only one-way data binding greatly simplifies the rendering logic and avoids many of the issues that commonly plague applications written with other frameworks. We're seeing the benefits of React.js on a growing number of projects, large and small, while at the same time we continue to be concerned about the state and the future of other popular frameworks like <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. This has led to React.js becoming our default choice for JavaScript frameworks."
},
{
"name": "Redux",
"description": "With the increasing complexity of single-page JavaScript applications, we have seen a more pressing need to make client-side state management predictable. <a href=\"http://redux.js.org/\"><strong>Redux</strong></a>, with its <a href=\"http://redux.js.org/docs/introduction/ThreePrinciples.html\">three principles</a> of restrictions for updating state, has proven to be invaluable in a number of projects we have implemented. <a href=\"https://egghead.io/courses/getting-started-with-redux\">Getting Started with Redux</a> and <a href=\"https://egghead.io/courses/building-react-applications-with-idiomatic-redux\">idiomatic Redux</a> tutorials are a good starting point for new and experienced users. Its minimal library design has spawned a rich set of tools, and we encourage you to check out the <a href=\"https://github.com/markerikson/redux-ecosystem-links\">redux-ecosystem-links</a> project for examples, middleware and utility libraries. We also particularly like the testability story: Dispatching actions, state transitions and rendering can be unit-tested separately from one another and with minimal amounts of mocking."
},
{
"name": "Spring Boot",
"description": "A lot of work has gone into <a href=\"http://projects.spring.io/spring-boot\"><strong>Spring Boot</strong></a> to reduce complexity and dependencies, which largely alleviates our previous reservations. If you live in a Spring ecosystem and are moving to microservices, Spring Boot is now the obvious choice. For those not in Springland, <a href=\"/radar/languages-and-frameworks/dropwizard\">Dropwizard</a> is also worthy of serious consideration."
},
{
"name": "Butterknife",
"description": "<strong><a href=\"https://github.com/JakeWharton/butterknife\">Butterknife</a></strong> is a field&nbsp;and method&nbsp;binding view-injection library. It allows the injection of arbitrary objects, views and listeners, thereby ensuring cleaner code with reduced glue code for Android development. With Butterknife, multiple views can be grouped into a list or array with common actions applied to the views simultaneously, without heavy reliance on XML configurations. Our project teams have used this library and benefited from its simplicity and ease of use."
},
{
"name": "Dagger",
"description": "With the increased need for Android-based applications, <strong><a href=\"http://google.github.io/dagger/\">Dagger</a> </strong>offers a fully static, compile-time dependency-injection framework. Dagger's strictly generated implementation and nonreliance on reflection-based solutions addresses many of the performance and development issues,&nbsp;thereby making it suitable for Android development. With Dagger, there is full traceability with easy debugging because the entire call stack for provision and creation is made available."
},
{
"name": "Dapper",
"description": "<a href=\"https://github.com/StackExchange/dapper-dot-net\"><strong>Dapper</strong></a> is a minimal, lightweight ORM of sorts for .NET. Rather than trying to write the SQL queries for you, Dapper maps SQL queries to dynamic objects. Though it's not brand new, Dapper has steadily gained acceptance from ThoughtWorks teams working in .NET. For the C# programmer, it removes some of the drudgery of mapping relational queries to objects while still allowing complete control over the SQL or stored procedures."
},
{
"name": "Elixir",
"description": "Interest in the <a href=\"http://elixir-lang.org/\"><strong>Elixir</strong></a> programming language continues to build. Increasingly, we see it used in serious projects and hear feedback from developers who find its Actor model to be robust and very fast. Elixir, which is built on top of the Erlang virtual machine, is showing promise for creating highly concurrent and fault-tolerant systems. Elixir has distinctive features such as the Pipe operator, which allows developers to build a pipeline of functions as you would in the UNIX command shell. The shared byte code allows Elixir to interoperate with Erlang and leverage existing libraries while supporting tools such as the Mix build tool, the IEx interactive shell and the <a href=\"http://elixir-lang.org/docs/stable/ex_unit/ExUnit.html\">ExUnit</a> unit-testing framework."
},
{
"name": "Enzyme",
"description": "We’ve been enjoying the rapid component-level UI testing that <a href=\"http://airbnb.io/enzyme/\"><strong>Enzyme</strong></a> provides for <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> applications. Unlike many other snapshot-based testing frameworks, Enzyme allows you to test without doing on-device rendering, which results in faster and more granular testing. This is a contributing factor in our ability to massively reduce the amount of functional testing we find we have to do in React applications."
},
{
"name": "Immutable.js",
"description": "Immutability is often emphasized in the functional programming paradigm, and most languages have the ability to create immutable objects—objects that can't be changed once created. <strong><a href=\"https://facebook.github.io/immutable-js/\">Immutable.js</a></strong> is a library for JavaScript that provides many persistent immutable data structures, which are highly efficient on modern JavaScript virtual machines. Immutable.js objects are, however, not normal JavaScript objects, so references to JavaScript objects from immutable objects should be avoided. More teams are using this library for tracking mutation and maintaining state in production. We recommend that developers investigate this library, especially when it's combined with the rest of the Facebook stack."
},
{
"name": "Phoenix",
"description": "Some of our ThoughtWorks teams have had very positive experiences with <strong>Phoenix</strong>, a server-side web MVC framework written in <a href=\"/radar/languages-and-frameworks/elixir\">Elixir</a>. In addition to being streamlined and easy to use, Phoenix takes advantage of Elixir to be extremely fast. For some developers, Phoenix evokes the joy they experienced when first discovering Ruby and Rails. Although the ecosystem of libraries for Phoenix is not as extensive as for some more mature frameworks, it should benefit from the continuing success and growth of support for Elixir."
},
{
"name": "Quick and Nimble",
"description": "Most of our iOS teams are now using the <strong><a href=\"https://github.com/Quick/Quick\">Quick</a> and <a href=\"https://github.com/Quick/Nimble\">Nimble</a></strong> pairing for their unit tests. In the <a href=\"http://rspec.info/\">RSpec</a> family of behavior-driven development (BDD) testing tools, it provides very readable tests (with describe blocks) across <a href=\"/radar/languages-and-frameworks/swift\">Swift</a> and Objective-C and has good support for asynchronous testing."
},
{
"name": "React Native",
"description": "We are seeing continued success with <a href=\"https://facebook.github.io/react-native/\"><strong>React Native</strong></a> for rapid cross-platform mobile development. Despite some churn as it undergoes continuing development, the advantages of trivial integration between native and nonnative code and views, the rapid development cycle (instant reload, chrome debugging, Flexbox layout) and general growth of the React style is winning us over. As with many frameworks, care needs to be taken to keep your code well structured, but diligent use of a tool like <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> really helps here."
},
{
"name": "Robolectric",
"description": "In the Android application-development world, <strong><a href=\"http://robolectric.org/\">Robolectric</a></strong> is a unit-testing framework that has been used by multiple teams within our technical community. It offers the best option among those available for writing real unit tests that extend or interact directly with Android components and support JUnit tests. We caution, though, that because it is an implementation of the Android SDK, there might be device-specific issues for some tests that pass in Robolectric. To manually mock all the Android dependencies, ensuring only test of the system-in-test will require a lot of complex code, and this framework addresses this effectively."
},
{
"name": "Aurelia",
"description": "<a href=\"http://aurelia.io/\"><strong>Aurelia</strong></a> is considered the next-generation JavaScript client framework and was written using a modern version of JavaScript: ECMAScript 2016. Aurelia was created by Rob Eisenberg, the creator of <a href=\"http://durandaljs.com/\">Durandal</a>. He left the <a href=\"https://angular.io/\">Angular 2.0</a> core team to dedicate his time to this project. The great thing about Aurelia is that it's highly modular, contains simple small libraries and is designed to be customized easily. Aurelia follows the pattern of convention over configuration, which enables easier production and consumption of modules, but there are no strong conventions that you have to adhere to. Aurelia has a large community, and in the project website you can learn more by using the tutorials."
},
{
"name": "ECMAScript 2017",
"description": ""
},
{
"name": "Elm",
"description": "We have been prompted to reconsider <a href=\"http://elm-lang.org/\"><strong>Elm</strong></a> because of the rapid adoption of <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> framework. Elm—the original inspiration for Redux—offers the view componentization and reactiveness of <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> along with the predictable state of Redux in a compiled, strongly typed functional language. Elm is written in Haskell and has a Haskell-like syntax but compiles down to HTML, CSS and JavaScript for the browser. JavaScript programmers rushing to embrace React.js and Redux might want to also consider Elm as a type-safe alternative for some applications."
},
{
"name": "GraphQL",
"description": "When we look at REST implementations in the wild, we frequently see REST misused to naively retrieve object graphs through chatty interactions between client and server. Facebook's <a href=\"https://github.com/facebook/graphql\"><strong>GraphQL</strong></a> is an interesting alternative to REST that might be a better approach for this very common use case. As a protocol for remotely retrieving object graphs, GraphQL has received enormous attention recently. One of GraphQL's most interesting features is its consumer-oriented nature: The structure of a response is driven entirely by the client, not the server. This decouples the consumer and forces the server to obey Postel's law. Client implementations are now available in many programming languages, but we have seen a flurry of interest of Facebook's <a href=\"https://facebook.github.io/relay/\">Relay</a>, a JavaScript framework that was designed to support the <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> stateless component model."
},
{
"name": "JuMP",
"description": "<strong><a href=\"https://github.com/JuliaOpt/JuMP.jl\">JuMP</a></strong> is a domain-specific language for <a href=\"https://en.wikipedia.org/wiki/Mathematical_optimization\">mathematical optimizations</a> in <a href=\"http://julialang.org/\">Julia</a>. JuMP defines a common API called <a href=\"https://github.com/JuliaOpt/MathProgBase.jl\">MathProgBase</a> and enables users to write solver-agnostic code in Julia. Currently supported solvers include <a href=\"http://artelys.com/en/optimization-tools/knitro\">Artelys Knitro</a>, <a href=\"https://projects.coin-or.org/Bonmin\">Bonmin</a>, <a href=\"https://projects.coin-or.org/Cbc\">Cbc</a>, <a href=\"https://projects.coin-or.org/Clp\">Clp</a>, <a href=\"https://projects.coin-or.org/Couenne\">Couenne</a>, <a href=\"http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/\">CPLEX</a>, <a href=\"https://github.com/ifa-ethz/ecos\">ECOS</a>, <a href=\"http://www.fico.com/en/products/fico-xpress-optimization-suite\">FICO Xpress</a>, <a href=\"http://www.gnu.org/software/glpk/\">GLPK</a>, <a href=\"http://www.gurobi.com\">Gurobi</a>, <a href=\"https://projects.coin-or.org/Ipopt\">Ipopt</a>, <a href=\"http://www.mosek.com/\">MOSEK</a>, <a href=\"http://ab-initio.mit.edu/wiki/index.php/NLopt\">NLopt</a> and <a href=\"https://github.com/cvxgrp/scs\">SCS</a>. One other benefit is the implementation of automatic differentiation technique in reverse mode to compute derivatives so users are not limited to the standard operators like sin, cos, log and sqrt but can also implement their own custom objective functions in Julia."
},
{
"name": "Physical Web",
"description": "We have been intrigued by the <strong><a href=\"https://google.github.io/physical-web/\">Physical Web</a></strong> standard created by Google. The idea of Physical Web is simple—beacons broadcast a URL—but the possibilities are broad. Basically, this is a way to annotate the physical world, tying objects and locations into the digital realm. The current transport mechanism is <a href=\"https://github.com/google/eddystone/tree/master/eddystone-url\">Eddystone URLs</a> over Bluetooth LE, and sample clients are available. Although there are obvious security concerns with following randomly discovered links, we are most interested in use cases with customized clients where you can filter or proxy the URLs as required."
},
{
"name": "Rapidoid",
"description": "<strong><a href=\"http://www.rapidoid.org/\">Rapidoid</a></strong> is a collection of web framework modules, including a fast low-level HTTP server implemented from scratch on top of Java NIO. Clever usage of off-heap input/output buffers, object pools and thread-local data structures provide Rapidoid an edge over other NIO-based servers like <a href=\"http://netty.io/\">Netty</a>. Being a fairly new project, Rapidoid has yet to implement a few features like built-in cache and SSL support; we suggest you check the <a href=\"https://github.com/rapidoid/rapidoid\">roadmap</a> for updates."
},
{
"name": "Recharts",
"description": "We've been enjoying how <strong><a href=\"http://recharts.org/\">Recharts</a></strong> integrates <a href=\"/radar/tools/d3\">D3</a> charts into <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> in a clean and declarative manner."
},
{
"name": "ReSwift",
"description": "We are excited that the <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> paradigm has made its way to Swift-land in the form of <a href=\"http://reswift.github.io/ReSwift\"><strong>ReSwift</strong></a>. We’ve found real benefits in the simplicity and readability of codebases once state and state changes are managed in a central place and common idiom. This also helps with building \"offline first\" applications."
},
{
"name": "Three.js",
"description": "Despite the fervor surrounding the spate of new headsets, we believe there are many VR and AR scenarios that make sense in the browser, particularly on mobile. Given this trend, we have seen an uptick in usage of <a href=\"https://threejs.org/\"><strong>Three.js</strong></a>, a powerful JavaScript visualization and 3D rendering framework. The growth in support for WebGL, which it is based on, has helped adoption, as has the vibrant community supporting this open source project."
},
{
"name": "Vue.js",
"description": "In the ever-changing world of front-end JavaScript frameworks, <a href=\"https://vuejs.org/\"><strong>Vue.js</strong></a> has gained a lot of ground as a lightweight alternative to <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. It is designed to be a very flexible—and a less opinionated—library that offers a set of tools for building interactive web interfaces around concepts like modularity, components and reactive data flow. It has a low learning barrier, which makes it interesting for junior developers and beginners. Vue.js itself is not a full-blown framework; it is focused on the view layer only and therefore is easy to integrate with other libraries or existing projects."
},
{
"name": "WebRTC",
"description": "Widespread adoption of AR/VR as a collaboration and communication medium requires a modern and readily available video streaming platform. <strong><a href=\"https://webrtc.org/\">WebRTC</a></strong> is an emerging standard for real-time communication between browsers that enables video streaming within commonly available web technologies. The range of browsers that support this standard is increasing, but Microsoft and Apple have been slow to adopt WebRTC in their proprietary browsers. If momentum continues to build, WebRTC could form the future foundation for AR/VR collaboration on the web."
},
{
"name": "AngularJS",
"description": "<a href=\"https://angularjs.org/\"><strong>AngularJS</strong></a> helped revolutionize the world of single-page JavaScript applications, and we have delivered many projects successfully with it over the years. However, we are no longer recommending it (v1) for teams starting fresh projects. We prefer the ramp-up speed and more maintainable codebases we are seeing with <a href=\"/radar/languages-and-frameworks/ember-js\">Ember</a> and <a href=\"/radar/languages-and-frameworks/react-js\">React</a>, particularly in conjunction with <a href=\"/radar/languages-and-frameworks/redux\">Redux</a>."
},
{
"name": "JSPatch",
"description": "Many iOS developers are using <strong><a href=\"https://github.com/bang590/JSPatch\">JSPatch</a></strong> to dynamically patch their apps. When a JSPatch-enabled app runs, it loads a chunk of JavaScript (potentially via an insecure HTTP connection) and then bridges to the main Objective-C application code to change behavior, fix bugs, and so on. While convenient, we think monkey-patching live apps is a bad idea and should be avoided. When doing any amount of incremental patching, it's very important that your testing process matches what end users will experience, in order to properly validate functionality. An alternative approach is to use <a href=\"/radar/languages-and-frameworks/react-native\">React Native</a> for the app and <a href=\"https://apphub.io/\">AppHub</a> and <a href=\"https://microsoft.github.io/code-push/\">CodePush</a> to push small updates and new features."
}
]
},
{
"name": "assess",
"blips": [
{
"name": "Ember.js",
"description": "If you are faced with building a single-page application (SPA) and trying to choose a framework to build with, <strong><a href=\"http://emberjs.com/\">Ember.js</a></strong> has emerged as a leading choice. Our teams praise Ember for its highly productive developer experience, with far fewer surprises than other frameworks such as <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. The Ember CLI build tooling is a haven in the storm of JavaScript build tools, and the Ember core team and community are highly active and responsive."
},
{
"name": "React.js",
"description": "In the avalanche of front-end JavaScript frameworks, <a href=\"http://facebook.github.io/react/\"><strong>React.js</strong></a> stands out due to its design around a reactive data flow. Allowing only one-way data binding greatly simplifies the rendering logic and avoids many of the issues that commonly plague applications written with other frameworks. We're seeing the benefits of React.js on a growing number of projects, large and small, while at the same time we continue to be concerned about the state and the future of other popular frameworks like <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. This has led to React.js becoming our default choice for JavaScript frameworks."
},
{
"name": "Redux",
"description": "With the increasing complexity of single-page JavaScript applications, we have seen a more pressing need to make client-side state management predictable. <a href=\"http://redux.js.org/\"><strong>Redux</strong></a>, with its <a href=\"http://redux.js.org/docs/introduction/ThreePrinciples.html\">three principles</a> of restrictions for updating state, has proven to be invaluable in a number of projects we have implemented. <a href=\"https://egghead.io/courses/getting-started-with-redux\">Getting Started with Redux</a> and <a href=\"https://egghead.io/courses/building-react-applications-with-idiomatic-redux\">idiomatic Redux</a> tutorials are a good starting point for new and experienced users. Its minimal library design has spawned a rich set of tools, and we encourage you to check out the <a href=\"https://github.com/markerikson/redux-ecosystem-links\">redux-ecosystem-links</a> project for examples, middleware and utility libraries. We also particularly like the testability story: Dispatching actions, state transitions and rendering can be unit-tested separately from one another and with minimal amounts of mocking."
},
{
"name": "Spring Boot",
"description": "A lot of work has gone into <a href=\"http://projects.spring.io/spring-boot\"><strong>Spring Boot</strong></a> to reduce complexity and dependencies, which largely alleviates our previous reservations. If you live in a Spring ecosystem and are moving to microservices, Spring Boot is now the obvious choice. For those not in Springland, <a href=\"/radar/languages-and-frameworks/dropwizard\">Dropwizard</a> is also worthy of serious consideration."
},
{
"name": "Butterknife",
"description": "<strong><a href=\"https://github.com/JakeWharton/butterknife\">Butterknife</a></strong> is a field&nbsp;and method&nbsp;binding view-injection library. It allows the injection of arbitrary objects, views and listeners, thereby ensuring cleaner code with reduced glue code for Android development. With Butterknife, multiple views can be grouped into a list or array with common actions applied to the views simultaneously, without heavy reliance on XML configurations. Our project teams have used this library and benefited from its simplicity and ease of use."
},
{
"name": "Dagger",
"description": "With the increased need for Android-based applications, <strong><a href=\"http://google.github.io/dagger/\">Dagger</a> </strong>offers a fully static, compile-time dependency-injection framework. Dagger's strictly generated implementation and nonreliance on reflection-based solutions addresses many of the performance and development issues,&nbsp;thereby making it suitable for Android development. With Dagger, there is full traceability with easy debugging because the entire call stack for provision and creation is made available."
},
{
"name": "Dapper",
"description": "<a href=\"https://github.com/StackExchange/dapper-dot-net\"><strong>Dapper</strong></a> is a minimal, lightweight ORM of sorts for .NET. Rather than trying to write the SQL queries for you, Dapper maps SQL queries to dynamic objects. Though it's not brand new, Dapper has steadily gained acceptance from ThoughtWorks teams working in .NET. For the C# programmer, it removes some of the drudgery of mapping relational queries to objects while still allowing complete control over the SQL or stored procedures."
},
{
"name": "Elixir",
"description": "Interest in the <a href=\"http://elixir-lang.org/\"><strong>Elixir</strong></a> programming language continues to build. Increasingly, we see it used in serious projects and hear feedback from developers who find its Actor model to be robust and very fast. Elixir, which is built on top of the Erlang virtual machine, is showing promise for creating highly concurrent and fault-tolerant systems. Elixir has distinctive features such as the Pipe operator, which allows developers to build a pipeline of functions as you would in the UNIX command shell. The shared byte code allows Elixir to interoperate with Erlang and leverage existing libraries while supporting tools such as the Mix build tool, the IEx interactive shell and the <a href=\"http://elixir-lang.org/docs/stable/ex_unit/ExUnit.html\">ExUnit</a> unit-testing framework."
},
{
"name": "Enzyme",
"description": "We’ve been enjoying the rapid component-level UI testing that <a href=\"http://airbnb.io/enzyme/\"><strong>Enzyme</strong></a> provides for <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> applications. Unlike many other snapshot-based testing frameworks, Enzyme allows you to test without doing on-device rendering, which results in faster and more granular testing. This is a contributing factor in our ability to massively reduce the amount of functional testing we find we have to do in React applications."
},
{
"name": "Immutable.js",
"description": "Immutability is often emphasized in the functional programming paradigm, and most languages have the ability to create immutable objects—objects that can't be changed once created. <strong><a href=\"https://facebook.github.io/immutable-js/\">Immutable.js</a></strong> is a library for JavaScript that provides many persistent immutable data structures, which are highly efficient on modern JavaScript virtual machines. Immutable.js objects are, however, not normal JavaScript objects, so references to JavaScript objects from immutable objects should be avoided. More teams are using this library for tracking mutation and maintaining state in production. We recommend that developers investigate this library, especially when it's combined with the rest of the Facebook stack."
},
{
"name": "Phoenix",
"description": "Some of our ThoughtWorks teams have had very positive experiences with <strong>Phoenix</strong>, a server-side web MVC framework written in <a href=\"/radar/languages-and-frameworks/elixir\">Elixir</a>. In addition to being streamlined and easy to use, Phoenix takes advantage of Elixir to be extremely fast. For some developers, Phoenix evokes the joy they experienced when first discovering Ruby and Rails. Although the ecosystem of libraries for Phoenix is not as extensive as for some more mature frameworks, it should benefit from the continuing success and growth of support for Elixir."
},
{
"name": "Quick and Nimble",
"description": "Most of our iOS teams are now using the <strong><a href=\"https://github.com/Quick/Quick\">Quick</a> and <a href=\"https://github.com/Quick/Nimble\">Nimble</a></strong> pairing for their unit tests. In the <a href=\"http://rspec.info/\">RSpec</a> family of behavior-driven development (BDD) testing tools, it provides very readable tests (with describe blocks) across <a href=\"/radar/languages-and-frameworks/swift\">Swift</a> and Objective-C and has good support for asynchronous testing."
},
{
"name": "React Native",
"description": "We are seeing continued success with <a href=\"https://facebook.github.io/react-native/\"><strong>React Native</strong></a> for rapid cross-platform mobile development. Despite some churn as it undergoes continuing development, the advantages of trivial integration between native and nonnative code and views, the rapid development cycle (instant reload, chrome debugging, Flexbox layout) and general growth of the React style is winning us over. As with many frameworks, care needs to be taken to keep your code well structured, but diligent use of a tool like <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> really helps here."
},
{
"name": "Robolectric",
"description": "In the Android application-development world, <strong><a href=\"http://robolectric.org/\">Robolectric</a></strong> is a unit-testing framework that has been used by multiple teams within our technical community. It offers the best option among those available for writing real unit tests that extend or interact directly with Android components and support JUnit tests. We caution, though, that because it is an implementation of the Android SDK, there might be device-specific issues for some tests that pass in Robolectric. To manually mock all the Android dependencies, ensuring only test of the system-in-test will require a lot of complex code, and this framework addresses this effectively."
},
{
"name": "Aurelia",
"description": "<a href=\"http://aurelia.io/\"><strong>Aurelia</strong></a> is considered the next-generation JavaScript client framework and was written using a modern version of JavaScript: ECMAScript 2016. Aurelia was created by Rob Eisenberg, the creator of <a href=\"http://durandaljs.com/\">Durandal</a>. He left the <a href=\"https://angular.io/\">Angular 2.0</a> core team to dedicate his time to this project. The great thing about Aurelia is that it's highly modular, contains simple small libraries and is designed to be customized easily. Aurelia follows the pattern of convention over configuration, which enables easier production and consumption of modules, but there are no strong conventions that you have to adhere to. Aurelia has a large community, and in the project website you can learn more by using the tutorials."
},
{
"name": "ECMAScript 2017",
"description": ""
},
{
"name": "Elm",
"description": "We have been prompted to reconsider <a href=\"http://elm-lang.org/\"><strong>Elm</strong></a> because of the rapid adoption of <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> framework. Elm—the original inspiration for Redux—offers the view componentization and reactiveness of <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> along with the predictable state of Redux in a compiled, strongly typed functional language. Elm is written in Haskell and has a Haskell-like syntax but compiles down to HTML, CSS and JavaScript for the browser. JavaScript programmers rushing to embrace React.js and Redux might want to also consider Elm as a type-safe alternative for some applications."
},
{
"name": "GraphQL",
"description": "When we look at REST implementations in the wild, we frequently see REST misused to naively retrieve object graphs through chatty interactions between client and server. Facebook's <a href=\"https://github.com/facebook/graphql\"><strong>GraphQL</strong></a> is an interesting alternative to REST that might be a better approach for this very common use case. As a protocol for remotely retrieving object graphs, GraphQL has received enormous attention recently. One of GraphQL's most interesting features is its consumer-oriented nature: The structure of a response is driven entirely by the client, not the server. This decouples the consumer and forces the server to obey Postel's law. Client implementations are now available in many programming languages, but we have seen a flurry of interest of Facebook's <a href=\"https://facebook.github.io/relay/\">Relay</a>, a JavaScript framework that was designed to support the <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> stateless component model."
},
{
"name": "JuMP",
"description": "<strong><a href=\"https://github.com/JuliaOpt/JuMP.jl\">JuMP</a></strong> is a domain-specific language for <a href=\"https://en.wikipedia.org/wiki/Mathematical_optimization\">mathematical optimizations</a> in <a href=\"http://julialang.org/\">Julia</a>. JuMP defines a common API called <a href=\"https://github.com/JuliaOpt/MathProgBase.jl\">MathProgBase</a> and enables users to write solver-agnostic code in Julia. Currently supported solvers include <a href=\"http://artelys.com/en/optimization-tools/knitro\">Artelys Knitro</a>, <a href=\"https://projects.coin-or.org/Bonmin\">Bonmin</a>, <a href=\"https://projects.coin-or.org/Cbc\">Cbc</a>, <a href=\"https://projects.coin-or.org/Clp\">Clp</a>, <a href=\"https://projects.coin-or.org/Couenne\">Couenne</a>, <a href=\"http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/\">CPLEX</a>, <a href=\"https://github.com/ifa-ethz/ecos\">ECOS</a>, <a href=\"http://www.fico.com/en/products/fico-xpress-optimization-suite\">FICO Xpress</a>, <a href=\"http://www.gnu.org/software/glpk/\">GLPK</a>, <a href=\"http://www.gurobi.com\">Gurobi</a>, <a href=\"https://projects.coin-or.org/Ipopt\">Ipopt</a>, <a href=\"http://www.mosek.com/\">MOSEK</a>, <a href=\"http://ab-initio.mit.edu/wiki/index.php/NLopt\">NLopt</a> and <a href=\"https://github.com/cvxgrp/scs\">SCS</a>. One other benefit is the implementation of automatic differentiation technique in reverse mode to compute derivatives so users are not limited to the standard operators like sin, cos, log and sqrt but can also implement their own custom objective functions in Julia."
},
{
"name": "Physical Web",
"description": "We have been intrigued by the <strong><a href=\"https://google.github.io/physical-web/\">Physical Web</a></strong> standard created by Google. The idea of Physical Web is simple—beacons broadcast a URL—but the possibilities are broad. Basically, this is a way to annotate the physical world, tying objects and locations into the digital realm. The current transport mechanism is <a href=\"https://github.com/google/eddystone/tree/master/eddystone-url\">Eddystone URLs</a> over Bluetooth LE, and sample clients are available. Although there are obvious security concerns with following randomly discovered links, we are most interested in use cases with customized clients where you can filter or proxy the URLs as required."
},
{
"name": "Rapidoid",
"description": "<strong><a href=\"http://www.rapidoid.org/\">Rapidoid</a></strong> is a collection of web framework modules, including a fast low-level HTTP server implemented from scratch on top of Java NIO. Clever usage of off-heap input/output buffers, object pools and thread-local data structures provide Rapidoid an edge over other NIO-based servers like <a href=\"http://netty.io/\">Netty</a>. Being a fairly new project, Rapidoid has yet to implement a few features like built-in cache and SSL support; we suggest you check the <a href=\"https://github.com/rapidoid/rapidoid\">roadmap</a> for updates."
},
{
"name": "Recharts",
"description": "We've been enjoying how <strong><a href=\"http://recharts.org/\">Recharts</a></strong> integrates <a href=\"/radar/tools/d3\">D3</a> charts into <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> in a clean and declarative manner."
},
{
"name": "ReSwift",
"description": "We are excited that the <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> paradigm has made its way to Swift-land in the form of <a href=\"http://reswift.github.io/ReSwift\"><strong>ReSwift</strong></a>. We’ve found real benefits in the simplicity and readability of codebases once state and state changes are managed in a central place and common idiom. This also helps with building \"offline first\" applications."
},
{
"name": "Three.js",
"description": "Despite the fervor surrounding the spate of new headsets, we believe there are many VR and AR scenarios that make sense in the browser, particularly on mobile. Given this trend, we have seen an uptick in usage of <a href=\"https://threejs.org/\"><strong>Three.js</strong></a>, a powerful JavaScript visualization and 3D rendering framework. The growth in support for WebGL, which it is based on, has helped adoption, as has the vibrant community supporting this open source project."
},
{
"name": "Vue.js",
"description": "In the ever-changing world of front-end JavaScript frameworks, <a href=\"https://vuejs.org/\"><strong>Vue.js</strong></a> has gained a lot of ground as a lightweight alternative to <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. It is designed to be a very flexible—and a less opinionated—library that offers a set of tools for building interactive web interfaces around concepts like modularity, components and reactive data flow. It has a low learning barrier, which makes it interesting for junior developers and beginners. Vue.js itself is not a full-blown framework; it is focused on the view layer only and therefore is easy to integrate with other libraries or existing projects."
},
{
"name": "WebRTC",
"description": "Widespread adoption of AR/VR as a collaboration and communication medium requires a modern and readily available video streaming platform. <strong><a href=\"https://webrtc.org/\">WebRTC</a></strong> is an emerging standard for real-time communication between browsers that enables video streaming within commonly available web technologies. The range of browsers that support this standard is increasing, but Microsoft and Apple have been slow to adopt WebRTC in their proprietary browsers. If momentum continues to build, WebRTC could form the future foundation for AR/VR collaboration on the web."
},
{
"name": "AngularJS",
"description": "<a href=\"https://angularjs.org/\"><strong>AngularJS</strong></a> helped revolutionize the world of single-page JavaScript applications, and we have delivered many projects successfully with it over the years. However, we are no longer recommending it (v1) for teams starting fresh projects. We prefer the ramp-up speed and more maintainable codebases we are seeing with <a href=\"/radar/languages-and-frameworks/ember-js\">Ember</a> and <a href=\"/radar/languages-and-frameworks/react-js\">React</a>, particularly in conjunction with <a href=\"/radar/languages-and-frameworks/redux\">Redux</a>."
},
{
"name": "JSPatch",
"description": "Many iOS developers are using <strong><a href=\"https://github.com/bang590/JSPatch\">JSPatch</a></strong> to dynamically patch their apps. When a JSPatch-enabled app runs, it loads a chunk of JavaScript (potentially via an insecure HTTP connection) and then bridges to the main Objective-C application code to change behavior, fix bugs, and so on. While convenient, we think monkey-patching live apps is a bad idea and should be avoided. When doing any amount of incremental patching, it's very important that your testing process matches what end users will experience, in order to properly validate functionality. An alternative approach is to use <a href=\"/radar/languages-and-frameworks/react-native\">React Native</a> for the app and <a href=\"https://apphub.io/\">AppHub</a> and <a href=\"https://microsoft.github.io/code-push/\">CodePush</a> to push small updates and new features."
}
]
},
{
"name": "hold",
"blips": [
{
"name": "Ember.js",
"description": "If you are faced with building a single-page application (SPA) and trying to choose a framework to build with, <strong><a href=\"http://emberjs.com/\">Ember.js</a></strong> has emerged as a leading choice. Our teams praise Ember for its highly productive developer experience, with far fewer surprises than other frameworks such as <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. The Ember CLI build tooling is a haven in the storm of JavaScript build tools, and the Ember core team and community are highly active and responsive."
},
{
"name": "React.js",
"description": "In the avalanche of front-end JavaScript frameworks, <a href=\"http://facebook.github.io/react/\"><strong>React.js</strong></a> stands out due to its design around a reactive data flow. Allowing only one-way data binding greatly simplifies the rendering logic and avoids many of the issues that commonly plague applications written with other frameworks. We're seeing the benefits of React.js on a growing number of projects, large and small, while at the same time we continue to be concerned about the state and the future of other popular frameworks like <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. This has led to React.js becoming our default choice for JavaScript frameworks."
},
{
"name": "Redux",
"description": "With the increasing complexity of single-page JavaScript applications, we have seen a more pressing need to make client-side state management predictable. <a href=\"http://redux.js.org/\"><strong>Redux</strong></a>, with its <a href=\"http://redux.js.org/docs/introduction/ThreePrinciples.html\">three principles</a> of restrictions for updating state, has proven to be invaluable in a number of projects we have implemented. <a href=\"https://egghead.io/courses/getting-started-with-redux\">Getting Started with Redux</a> and <a href=\"https://egghead.io/courses/building-react-applications-with-idiomatic-redux\">idiomatic Redux</a> tutorials are a good starting point for new and experienced users. Its minimal library design has spawned a rich set of tools, and we encourage you to check out the <a href=\"https://github.com/markerikson/redux-ecosystem-links\">redux-ecosystem-links</a> project for examples, middleware and utility libraries. We also particularly like the testability story: Dispatching actions, state transitions and rendering can be unit-tested separately from one another and with minimal amounts of mocking."
},
{
"name": "Spring Boot",
"description": "A lot of work has gone into <a href=\"http://projects.spring.io/spring-boot\"><strong>Spring Boot</strong></a> to reduce complexity and dependencies, which largely alleviates our previous reservations. If you live in a Spring ecosystem and are moving to microservices, Spring Boot is now the obvious choice. For those not in Springland, <a href=\"/radar/languages-and-frameworks/dropwizard\">Dropwizard</a> is also worthy of serious consideration."
},
{
"name": "Butterknife",
"description": "<strong><a href=\"https://github.com/JakeWharton/butterknife\">Butterknife</a></strong> is a field&nbsp;and method&nbsp;binding view-injection library. It allows the injection of arbitrary objects, views and listeners, thereby ensuring cleaner code with reduced glue code for Android development. With Butterknife, multiple views can be grouped into a list or array with common actions applied to the views simultaneously, without heavy reliance on XML configurations. Our project teams have used this library and benefited from its simplicity and ease of use."
},
{
"name": "Dagger",
"description": "With the increased need for Android-based applications, <strong><a href=\"http://google.github.io/dagger/\">Dagger</a> </strong>offers a fully static, compile-time dependency-injection framework. Dagger's strictly generated implementation and nonreliance on reflection-based solutions addresses many of the performance and development issues,&nbsp;thereby making it suitable for Android development. With Dagger, there is full traceability with easy debugging because the entire call stack for provision and creation is made available."
},
{
"name": "Dapper",
"description": "<a href=\"https://github.com/StackExchange/dapper-dot-net\"><strong>Dapper</strong></a> is a minimal, lightweight ORM of sorts for .NET. Rather than trying to write the SQL queries for you, Dapper maps SQL queries to dynamic objects. Though it's not brand new, Dapper has steadily gained acceptance from ThoughtWorks teams working in .NET. For the C# programmer, it removes some of the drudgery of mapping relational queries to objects while still allowing complete control over the SQL or stored procedures."
},
{
"name": "Elixir",
"description": "Interest in the <a href=\"http://elixir-lang.org/\"><strong>Elixir</strong></a> programming language continues to build. Increasingly, we see it used in serious projects and hear feedback from developers who find its Actor model to be robust and very fast. Elixir, which is built on top of the Erlang virtual machine, is showing promise for creating highly concurrent and fault-tolerant systems. Elixir has distinctive features such as the Pipe operator, which allows developers to build a pipeline of functions as you would in the UNIX command shell. The shared byte code allows Elixir to interoperate with Erlang and leverage existing libraries while supporting tools such as the Mix build tool, the IEx interactive shell and the <a href=\"http://elixir-lang.org/docs/stable/ex_unit/ExUnit.html\">ExUnit</a> unit-testing framework."
},
{
"name": "Enzyme",
"description": "We’ve been enjoying the rapid component-level UI testing that <a href=\"http://airbnb.io/enzyme/\"><strong>Enzyme</strong></a> provides for <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> applications. Unlike many other snapshot-based testing frameworks, Enzyme allows you to test without doing on-device rendering, which results in faster and more granular testing. This is a contributing factor in our ability to massively reduce the amount of functional testing we find we have to do in React applications."
},
{
"name": "Immutable.js",
"description": "Immutability is often emphasized in the functional programming paradigm, and most languages have the ability to create immutable objects—objects that can't be changed once created. <strong><a href=\"https://facebook.github.io/immutable-js/\">Immutable.js</a></strong> is a library for JavaScript that provides many persistent immutable data structures, which are highly efficient on modern JavaScript virtual machines. Immutable.js objects are, however, not normal JavaScript objects, so references to JavaScript objects from immutable objects should be avoided. More teams are using this library for tracking mutation and maintaining state in production. We recommend that developers investigate this library, especially when it's combined with the rest of the Facebook stack."
},
{
"name": "Phoenix",
"description": "Some of our ThoughtWorks teams have had very positive experiences with <strong>Phoenix</strong>, a server-side web MVC framework written in <a href=\"/radar/languages-and-frameworks/elixir\">Elixir</a>. In addition to being streamlined and easy to use, Phoenix takes advantage of Elixir to be extremely fast. For some developers, Phoenix evokes the joy they experienced when first discovering Ruby and Rails. Although the ecosystem of libraries for Phoenix is not as extensive as for some more mature frameworks, it should benefit from the continuing success and growth of support for Elixir."
},
{
"name": "Quick and Nimble",
"description": "Most of our iOS teams are now using the <strong><a href=\"https://github.com/Quick/Quick\">Quick</a> and <a href=\"https://github.com/Quick/Nimble\">Nimble</a></strong> pairing for their unit tests. In the <a href=\"http://rspec.info/\">RSpec</a> family of behavior-driven development (BDD) testing tools, it provides very readable tests (with describe blocks) across <a href=\"/radar/languages-and-frameworks/swift\">Swift</a> and Objective-C and has good support for asynchronous testing."
},
{
"name": "React Native",
"description": "We are seeing continued success with <a href=\"https://facebook.github.io/react-native/\"><strong>React Native</strong></a> for rapid cross-platform mobile development. Despite some churn as it undergoes continuing development, the advantages of trivial integration between native and nonnative code and views, the rapid development cycle (instant reload, chrome debugging, Flexbox layout) and general growth of the React style is winning us over. As with many frameworks, care needs to be taken to keep your code well structured, but diligent use of a tool like <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> really helps here."
},
{
"name": "Robolectric",
"description": "In the Android application-development world, <strong><a href=\"http://robolectric.org/\">Robolectric</a></strong> is a unit-testing framework that has been used by multiple teams within our technical community. It offers the best option among those available for writing real unit tests that extend or interact directly with Android components and support JUnit tests. We caution, though, that because it is an implementation of the Android SDK, there might be device-specific issues for some tests that pass in Robolectric. To manually mock all the Android dependencies, ensuring only test of the system-in-test will require a lot of complex code, and this framework addresses this effectively."
},
{
"name": "Aurelia",
"description": "<a href=\"http://aurelia.io/\"><strong>Aurelia</strong></a> is considered the next-generation JavaScript client framework and was written using a modern version of JavaScript: ECMAScript 2016. Aurelia was created by Rob Eisenberg, the creator of <a href=\"http://durandaljs.com/\">Durandal</a>. He left the <a href=\"https://angular.io/\">Angular 2.0</a> core team to dedicate his time to this project. The great thing about Aurelia is that it's highly modular, contains simple small libraries and is designed to be customized easily. Aurelia follows the pattern of convention over configuration, which enables easier production and consumption of modules, but there are no strong conventions that you have to adhere to. Aurelia has a large community, and in the project website you can learn more by using the tutorials."
},
{
"name": "ECMAScript 2017",
"description": ""
},
{
"name": "Elm",
"description": "We have been prompted to reconsider <a href=\"http://elm-lang.org/\"><strong>Elm</strong></a> because of the rapid adoption of <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> framework. Elm—the original inspiration for Redux—offers the view componentization and reactiveness of <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> along with the predictable state of Redux in a compiled, strongly typed functional language. Elm is written in Haskell and has a Haskell-like syntax but compiles down to HTML, CSS and JavaScript for the browser. JavaScript programmers rushing to embrace React.js and Redux might want to also consider Elm as a type-safe alternative for some applications."
},
{
"name": "GraphQL",
"description": "When we look at REST implementations in the wild, we frequently see REST misused to naively retrieve object graphs through chatty interactions between client and server. Facebook's <a href=\"https://github.com/facebook/graphql\"><strong>GraphQL</strong></a> is an interesting alternative to REST that might be a better approach for this very common use case. As a protocol for remotely retrieving object graphs, GraphQL has received enormous attention recently. One of GraphQL's most interesting features is its consumer-oriented nature: The structure of a response is driven entirely by the client, not the server. This decouples the consumer and forces the server to obey Postel's law. Client implementations are now available in many programming languages, but we have seen a flurry of interest of Facebook's <a href=\"https://facebook.github.io/relay/\">Relay</a>, a JavaScript framework that was designed to support the <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> stateless component model."
},
{
"name": "JuMP",
"description": "<strong><a href=\"https://github.com/JuliaOpt/JuMP.jl\">JuMP</a></strong> is a domain-specific language for <a href=\"https://en.wikipedia.org/wiki/Mathematical_optimization\">mathematical optimizations</a> in <a href=\"http://julialang.org/\">Julia</a>. JuMP defines a common API called <a href=\"https://github.com/JuliaOpt/MathProgBase.jl\">MathProgBase</a> and enables users to write solver-agnostic code in Julia. Currently supported solvers include <a href=\"http://artelys.com/en/optimization-tools/knitro\">Artelys Knitro</a>, <a href=\"https://projects.coin-or.org/Bonmin\">Bonmin</a>, <a href=\"https://projects.coin-or.org/Cbc\">Cbc</a>, <a href=\"https://projects.coin-or.org/Clp\">Clp</a>, <a href=\"https://projects.coin-or.org/Couenne\">Couenne</a>, <a href=\"http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/\">CPLEX</a>, <a href=\"https://github.com/ifa-ethz/ecos\">ECOS</a>, <a href=\"http://www.fico.com/en/products/fico-xpress-optimization-suite\">FICO Xpress</a>, <a href=\"http://www.gnu.org/software/glpk/\">GLPK</a>, <a href=\"http://www.gurobi.com\">Gurobi</a>, <a href=\"https://projects.coin-or.org/Ipopt\">Ipopt</a>, <a href=\"http://www.mosek.com/\">MOSEK</a>, <a href=\"http://ab-initio.mit.edu/wiki/index.php/NLopt\">NLopt</a> and <a href=\"https://github.com/cvxgrp/scs\">SCS</a>. One other benefit is the implementation of automatic differentiation technique in reverse mode to compute derivatives so users are not limited to the standard operators like sin, cos, log and sqrt but can also implement their own custom objective functions in Julia."
},
{
"name": "Physical Web",
"description": "We have been intrigued by the <strong><a href=\"https://google.github.io/physical-web/\">Physical Web</a></strong> standard created by Google. The idea of Physical Web is simple—beacons broadcast a URL—but the possibilities are broad. Basically, this is a way to annotate the physical world, tying objects and locations into the digital realm. The current transport mechanism is <a href=\"https://github.com/google/eddystone/tree/master/eddystone-url\">Eddystone URLs</a> over Bluetooth LE, and sample clients are available. Although there are obvious security concerns with following randomly discovered links, we are most interested in use cases with customized clients where you can filter or proxy the URLs as required."
},
{
"name": "Rapidoid",
"description": "<strong><a href=\"http://www.rapidoid.org/\">Rapidoid</a></strong> is a collection of web framework modules, including a fast low-level HTTP server implemented from scratch on top of Java NIO. Clever usage of off-heap input/output buffers, object pools and thread-local data structures provide Rapidoid an edge over other NIO-based servers like <a href=\"http://netty.io/\">Netty</a>. Being a fairly new project, Rapidoid has yet to implement a few features like built-in cache and SSL support; we suggest you check the <a href=\"https://github.com/rapidoid/rapidoid\">roadmap</a> for updates."
},
{
"name": "Recharts",
"description": "We've been enjoying how <strong><a href=\"http://recharts.org/\">Recharts</a></strong> integrates <a href=\"/radar/tools/d3\">D3</a> charts into <a href=\"/radar/languages-and-frameworks/react-js\">React.js</a> in a clean and declarative manner."
},
{
"name": "ReSwift",
"description": "We are excited that the <a href=\"/radar/languages-and-frameworks/redux\">Redux</a> paradigm has made its way to Swift-land in the form of <a href=\"http://reswift.github.io/ReSwift\"><strong>ReSwift</strong></a>. We’ve found real benefits in the simplicity and readability of codebases once state and state changes are managed in a central place and common idiom. This also helps with building \"offline first\" applications."
},
{
"name": "Three.js",
"description": "Despite the fervor surrounding the spate of new headsets, we believe there are many VR and AR scenarios that make sense in the browser, particularly on mobile. Given this trend, we have seen an uptick in usage of <a href=\"https://threejs.org/\"><strong>Three.js</strong></a>, a powerful JavaScript visualization and 3D rendering framework. The growth in support for WebGL, which it is based on, has helped adoption, as has the vibrant community supporting this open source project."
},
{
"name": "Vue.js",
"description": "In the ever-changing world of front-end JavaScript frameworks, <a href=\"https://vuejs.org/\"><strong>Vue.js</strong></a> has gained a lot of ground as a lightweight alternative to <a href=\"/radar/languages-and-frameworks/angularjs\">AngularJS</a>. It is designed to be a very flexible—and a less opinionated—library that offers a set of tools for building interactive web interfaces around concepts like modularity, components and reactive data flow. It has a low learning barrier, which makes it interesting for junior developers and beginners. Vue.js itself is not a full-blown framework; it is focused on the view layer only and therefore is easy to integrate with other libraries or existing projects."
},
{
"name": "WebRTC",
"description": "Widespread adoption of AR/VR as a collaboration and communication medium requires a modern and readily available video streaming platform. <strong><a href=\"https://webrtc.org/\">WebRTC</a></strong> is an emerging standard for real-time communication between browsers that enables video streaming within commonly available web technologies. The range of browsers that support this standard is increasing, but Microsoft and Apple have been slow to adopt WebRTC in their proprietary browsers. If momentum continues to build, WebRTC could form the future foundation for AR/VR collaboration on the web."
},
{
"name": "AngularJS",
"description": "<a href=\"https://angularjs.org/\"><strong>AngularJS</strong></a> helped revolutionize the world of single-page JavaScript applications, and we have delivered many projects successfully with it over the years. However, we are no longer recommending it (v1) for teams starting fresh projects. We prefer the ramp-up speed and more maintainable codebases we are seeing with <a href=\"/radar/languages-and-frameworks/ember-js\">Ember</a> and <a href=\"/radar/languages-and-frameworks/react-js\">React</a>, particularly in conjunction with <a href=\"/radar/languages-and-frameworks/redux\">Redux</a>."
},
{
"name": "JSPatch",
"description": "Many iOS developers are using <strong><a href=\"https://github.com/bang590/JSPatch\">JSPatch</a></strong> to dynamically patch their apps. When a JSPatch-enabled app runs, it loads a chunk of JavaScript (potentially via an insecure HTTP connection) and then bridges to the main Objective-C application code to change behavior, fix bugs, and so on. While convenient, we think monkey-patching live apps is a bad idea and should be avoided. When doing any amount of incremental patching, it's very important that your testing process matches what end users will experience, in order to properly validate functionality. An alternative approach is to use <a href=\"/radar/languages-and-frameworks/react-native\">React Native</a> for the app and <a href=\"https://apphub.io/\">AppHub</a> and <a href=\"https://microsoft.github.io/code-push/\">CodePush</a> to push small updates and new features."
}
]
}
],
"Platforms" : [
{
"name": "adopt",
"blips": [
{
"name": "Docker",
"description": "We remain excited about <a href=\"https://www.docker.com/\"><strong>Docker</strong></a> as it evolves from a tool to a complex platform of technologies. Development teams love Docker, as the Docker image format makes it easier to achieve parity between development and production, making for reliable deployments. It is a natural fit in a microservices-style application as a packaging mechanism for self-contained services. On the operational front, Docker support in monitoring tools (<a href=\"/radar/tools/sensu\">Sensu</a>, <a href=\"/radar/tools/prometheus\">Prometheus</a>, <a href=\"https://github.com/google/cadvisor\">cAdvisor</a>, etc.), orchestration tools (<a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a>, etc.) and deployment-automation tools reflect the growing maturity of the platform and its readiness for production use. A word of caution, though: There is a prevalent view of Docker and Linux containers in general as being \"lightweight virtualization,\" but we would not recommend using Docker as a secure process-isolation mechanism, though we are paying attention to the introduction of user namespaces and seccomp profiles in version 1.10 in this regard."
},
{
"name": "HSTS",
"description": "<a href=\"https://www.owasp.org/index.php/HTTP_Strict_Transport_Security\">HTTP Strict Transport Security</a> (<strong>HSTS</strong>) is a now widely supported policy that allows websites to protect themselves from downgrade attacks. A downgrade attack in the context of HTTPS is one that can cause users of your site to fall back to HTTP rather than HTTPS, allowing for further attacks such as man-in-the-middle attacks. With HSTS, the server sends a header that informs the browser that it should only use HTTPS to access the website. Browser support is now widespread enough that this easy-to-implement feature should be added to any site using HTTPS. Mozilla’s <a href=\"https://observatory.mozilla.org/\">Observatory</a> can help identify this and other useful headers and configuration options that improve security and privacy. When implementing HSTS, it is critical to verify that all resources load properly over HTTPS, because once HSTS is turned on, there is (almost) no turning back until the expiry time. The directive to include subdomains should be added but, again, a thorough verification that all subdomains support secure transport is required."
},
{
"name": "Linux security modules",
"description": "<a href=\"http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-167.pdf\">Application whitelisting</a> has proven to be <a href=\"http://www.asd.gov.au/infosec/top-mitigations/top-4-strategies-explained.htm#mitigation1\">one of the most effective ways to mitigate cyber intrusion attacks</a>. A convenient way to implement this widely recommended practice is through <strong>Linux security modules</strong>. With SELinux or AppArmor included by default in most Linux distributions, and with more comprehensive tools such as Grsecurity readily available, we have moved this technology into the Adopt ring in this edition. These tools help teams assess questions about who has access to what resources on shared hosts, including contained services. This conservative approach to access management will help teams build security into their SDLC processes."
},
{
"name": "Apache Mesos",
"description": "We've continued to have positive experiences deploying the <strong><a href=\"http://mesos.apache.org/\">Apache Mesos</a> </strong>platform to manage cluster resources for highly distributed systems. Mesos abstracts out underlying computing resources such as CPU and storage, aiming to provide efficient utilization while maintaining isolation. Mesos includes <a href=\"https://mesos.github.io/chronos/\">Chronos</a> for distributed and fault-tolerant execution of scheduled jobs, and <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a> for orchestrating long-running processes in containers."
},
{
"name": "Auth0",
"description": "We have a growing belief that for most scenarios it is rarely worth rolling your own authentication code. Outsourced identity management speeds up delivery, reduces mistakes and tends to enable a faster response to newly discovered vulnerabilities. <strong><a href=\"https://auth0.com/\">Auth0</a></strong> has particularly impressed us in this field for its ease of integration, range of protocols and connectors supported, and rich management API."
},
{
"name": "AWS Lambda",
"description": "Our teams continue to enjoy using <strong><a href=\"https://aws.amazon.com/lambda/\">AWS Lambda</a></strong> and are beginning to use it to experiment with <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>, combining Lambda with the <a href=\"/radar/platforms/amazon-api-gateway\">API Gateway</a>. We do recommend that Lambda functions contain only a moderate amount of code. Ensuring the quality of a solution based on a tangle of many large Lambda functions is difficult, and such a solution may not be cost-effective. For more&nbsp;complex needs, deployments based on containers or VMs are still preferable. In addition, we have run into significant problems using Java for Lambda functions, with erratic latencies up to several seconds as the Lambda container is started. Of course, you can sidestep this issue by using JavaScript or Python, and if Lambda functions do not contain a lot of code, the choice of programming language should not matter too much."
},
{
"name": "Kubernetes",
"description": "<strong><a href=\"http://kubernetes.io/\">Kubernetes</a></strong> is Google's answer to the problem of deploying containers into a cluster of machines, which is becoming an increasingly common scenario. It is not the solution used by Google internally but an open source project that originated at Google and has seen a fair number of external contributions. Since we mentioned Kubernetes on the previous Radar, our initial positive impressions have been confirmed, and we are seeing successful use of Kubernetes in production at our clients."
},
{
"name": "Pivotal Cloud Foundry",
"description": "The PaaS space has seen a lot of movement since we last mentioned <a href=\"/radar/platforms/cloud-foundry\">Cloud Foundry</a> in 2012. While there are various distributions of the open source core, we have been impressed by the offering and ecosystem assembled as <a href=\"http://pivotal.io/platform\"><strong>Pivotal Cloud Foundry</strong></a>. While we expect continued convergence between the unstructured approach (<a href=\"/radar/platforms/docker\">Docker</a>, <a href=\"/radar/platforms/apache-mesos\">Mesos</a>, <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, etc.) and the more structured and opinionated buildpack style offered by Cloud Foundry and others, we see real benefit for organizations that are willing to accept the constraints and rate of evolution to adopt a PaaS. Of particular interest is the speed of development that comes from the simplification and standardization of the interaction between development teams and platform operations."
},
{
"name": "Rancher",
"description": "The emerging Containers as a Service (CaaS) space is seeing a lot of movement and provides a useful option between basic IaaS (Infrastructure as a Service) and more opinionated PaaS (Platform as a Service). While <a href=\"http://rancher.com/\"><strong>Rancher</strong></a> creates less noise than some other players, we have enjoyed the simplicity that it brings to running <a href=\"/radar/platforms/docker\">Docker</a> containers in production. It can run stand-alone as a full solution or in conjunction with tools like <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>."
},
{
"name": "Realm",
"description": "<strong><a href=\"https://realm.io/\">Realm</a></strong> is a database designed for use on mobile devices, with its own persistence engine to achieve high performance. Realm is marketed as a replacement for SQLite and Core Data. Note that migrations are not quite as straightforward as the Realm documentation would have you believe. However, more and more teams are choosing Realm as the persistence mechanism in production environments for mobile applications."
},
{
"name": "Unity beyond gaming",
"description": "After experiencing years of growth as a platform for game development, <strong><a href=\"https://unity3d.com/\">Unity</a></strong> has recently become the platform of choice for VR and AR application development. Whether you’re creating a fully immersive world for the Oculus or HTC Vive headsets, a holographic layer for your newly spatial enterprise application or an AR feature set for your mobile app, Unity likely provides what you need to both prototype it and get it ready for prime time. Many of us at ThoughtWorks believe that VR and AR represent the next significant shift in the computing platform, and for now, Unity is the single most important tool in the toolbox we use to develop for this change. We’ve used Unity to develop all our VR prototypes, as well as AR functionality for headsets and phone/tablet applications."
},
{
"name": ".NET Core",
"description": "<strong><a href=\"https://www.microsoft.com/net/core\">.NET Core</a></strong> is an open source modular product for creating applications that can be easily deployed in Windows, macOS and Linux. .NET Core makes it possible to build cross-platform web applications using <a href=\"http://www.asp.net/core\">ASP.NET Core</a> with a set of tools, libraries and frameworks—another choice for microservices architecture. The community around .NET Core and other related projects has been growing. New tools have appeared and evolved quickly, such as <a href=\"/radar/tools/visual-studio-code\">Visual Studio Code</a>. There are <a href=\"/radar/platforms/docker\">Docker</a> <a href=\"https://www.microsoft.com/net/core#docker\">images</a> based on both Linux and Windows (<a href=\"/radar/platforms/microsoft-nano-server\">Nano Server</a>) with .NET Core that simplify applying a microservice architecture. CoreCLR and CoreFX appeared in the Radar in the past. However, a few months ago Microsoft <a href=\"https://blogs.msdn.microsoft.com/dotnet/2016/06/27/announcing-net-core-1-0\">announced</a> the release of .NET Core 1.0, the first stable version. We see good new opportunities, changes and a vibrant community as reasons to keep assessing this product."
},
{
"name": "Amazon API Gateway",
"description": "<a href=\"https://aws.amazon.com/api-gateway/\"><strong>Amazon API Gateway</strong></a> is Amazon's offering enabling developers to expose API services to Internet clients. It offers the usual API gateway features like traffic management, monitoring, authentication and authorization. Our teams have been using this service to front other AWS capabilities like AWS Lambda as part of <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>. We continue to monitor for the challenges presented by <a href=\"/radar/platforms/overambitious-api-gateways\">overambitious API gateways</a>, but at this stage Amazon's offering appears to be lightweight enough to avoid those problems."
},
{
"name": "Apache Flink",
"description": "Interest continues to build for <strong><a href=\"https://flink.apache.org/\">Apache Flink</a></strong>, a new-generation platform for scalable distributed batch and stream processing. At the core of Apache Flink is a streaming data-flow engine, with support for tabular (SQL-like), graph-processing and machine&nbsp;learning operations. Apache Flink stands out with feature&nbsp;rich capabilities for stream processing: event time, rich streaming window operations, fault tolerance and exactly-once semantics. The project shows significant ongoing activity, with the latest release (1.1) introducing new datasource/sink integrations as well as improved streaming features."
},
{
"name": "AWS Application Load Balancer",
"description": "Amazon recently launched the <a href=\"https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/\"><strong>AWS Application Load Balancer</strong></a> (ALB), a direct replacement for Elastic Load Balancers introduced back in 2009. ALB supports Layer 7 traffic inspection and is built to support modern cloud architecture. If you’re building a microservices-based system using <a href=\"/radar/platforms/aws-ecs\">ECS</a>, the new load balancers will directly understand container hosting and scaling, with multiple containers and ports per EC2 instance. Content-based routing allows segmentation of requests onto groups of target servers, along with independent scaling of those groups. Health checks performed by the load balancers are much improved, with the ability to capture detailed metrics about application performance. We like everything that we see here, and teams have begun to report successful usage of ALB."
},
{
"name": "Cassandra carefully",
"description": "Apache’s <a href=\"http://cassandra.apache.org/\">Cassandra</a> database is a powerful, scalable Big Data solution for storing and processing large amounts of data, often using hundreds of nodes split over multiple worldwide locations. It’s a great tool and we like it, but too often we see teams run into trouble using it. We recommend using <strong>Cassandra carefully</strong>. Teams often misunderstand the use case for Cassandra, attempting to use it as a general-purpose data store when in fact it is optimized for fast reads on large data sets based on predefined keys or indexes. Its dependence on the storage schema can also make it difficult to evolve over time. Cassandra also has significant operational complexity and some rough edges, so unless you absolutely need the scaling it provides, a simpler solution is usually better. If you don’t need Cassandra’s specific use-case and scaling characteristics, you might just be choosing it out of <a href=\"/radar/techniques/big-data-envy\">Big Data envy</a>. Careful use of Cassandra will include extensive automated testing, and we’re happy to recommend <a href=\"https://github.com/jsevellec/cassandra-unit\">CassandraUnit</a> as part of your testing strategy."
},
{
"name": "Electron",
"description": "<strong><a href=\"http://electron.atom.io/\">Electron</a></strong> is a solid framework for building native desktop clients using web technologies such as HTML, CSS and JavaScript. Teams can leverage their web know-how to deliver polished cross-platform desktop clients without spending time learning another set of technologies."
},
{
"name": "Ethereum",
"description": "The hype seems to have peaked for blockchain and cryptocurrencies, as evidenced by the previous firehose-scale announcements in this area slowing to a trickle, and we expect some of the more speculative efforts to die out over time. One of the blockchains, <a href=\"https://www.ethereum.org/\"><strong>Ethereum</strong></a>, is making good progress and is worth watching. Ethereum is a public blockchain with a built-in programming language that allows \"smart contracts\" to be built into it. These are algorithmic movements of \"ether\" (the Ethereum cryptocurrency) in response to activity happening on the blockchain. R3Cev, the consortium building blockchain tech for banks, built its first proofs of concept on Ethereum. Ethereum has been used to build a Distributed Autonomous Organization (DAO)—one of the first \"algorithmic corporations\"—although a recent heist of <a href=\"http://www.coindesk.com/dao-attacked-code-issue-leads-60-million-ether-theft/\">$150m worth of Ether</a> demonstrates that the blockchain and cryptocurrencies are still the Wild West of the technology world."
},
{
"name": "HoloLens",
"description": "In the <strong><a href=\"https://www.microsoft.com/microsoft-hololens/en-us\">HoloLens</a></strong>, Microsoft has delivered the first truly usable AR headset. Not only is it a beautiful piece of industrial design and an eminently comfortable device to wear, but it also clearly demonstrates the promise of AR for the enterprise via its gorgeous optics and deep Windows 10 integration. We expect HoloLens to be the first AR platform on which we deliver substantial application functionality to our clients in the near term, and we look forward to its evolution as it gains broader traction."
},
{
"name": "IndiaStack",
"description": "<strong><a href=\"http://www.indiastack.org\">IndiaStack</a></strong> is a set of Open APIs designed with the goal of transforming India from a data-poor to a data-rich country. The stack emphasizes layered innovation by specifying a minimal set of APIs and encourages the rest of the ecosystem to build custom applications on top of these APIs. <a href=\"http://www.indiastack.org/Resource#Aadhaar\">Aadhaar</a> serves as one of the foundation layers, providing authentication services for more than a billion Indian citizens. In addition, there are services to provide paperless transactions through digital signatures (eSign), unified online payment (UPI) and an electronic consent layer (<a href=\"https://uidai.gov.in/beta/authentication/aadhaar-financial-inclusion/aadhaar-e-kyc.html\">e-KYC</a>) to securely provide Aadhaar details to service providers. We believe in the Open API–driven initiative to bring digital innovation, and the design principles behind IndiaStack could be used as a change agent for other regions/countries."
},
{
"name": "Nomad",
"description": "HashiCorp continues to turn out interesting software. The latest to catch our attention is <a href=\"https://www.nomadproject.io/\"><strong>Nomad</strong></a>, which is competing in the ever-more-populated scheduler arena. Major selling points include not just being limited to containerized workloads, and operating in multi–data center / multiregion deployments."
},
{
"name": "Nuance Mix",
"description": "<strong><a href=\"https://developer.nuance.com/public/index.php?task=mix\">Nuance Mix</a></strong> is a framework for natural language processing from the company that created the speech-to-text technology behind Dragon Speaking and the first roll-out of Siri. This framework supports the creation of grammars that allow for free-form user interaction via voice. The developer defines a domain-specific grammar that the framework can train itself to understand. The outcomes are responses to user input that identify the user's intents and interaction concepts. At first, it is limited to phrases close to the ones used to train it, but over time it can start to identify meaning from more divergent phrasing. Though it is still in beta, the accuracy from early exploration has been compelling, and the eventual product is one to watch for application forms that could benefit from hands-free user interaction—including mobile, IoT, AR, VR and interactive spaces."
},
{
"name": "OpenVR",
"description": "<strong><a href=\"https://github.com/ValveSoftware/openvr\">OpenVR</a></strong> is the underlying SDK in making many of the VR head-mounted displays (HMDs) work with Unity and will likely keep growing in importance. Much of the VR work at ThoughtWorks was built on top of OpenVR, because it will run on any HMD, unlike the other SDKs. Though it is not open source, it is free via the license. The Oculus SDK is more restrictive in its licensing and only works on Oculus devices. <a href=\"http://www.osvr.org/\">OSVR</a>, while truly open source, doesn't seem to have as much adoption yet. If you're going to develop a VR application and target as many devices as possible—and not use Unity or Unreal to develop them—OpenVR is the most concrete and pragmatic solution right now."
},
{
"name": "Tarantool",
"description": "<strong><a href=\"https://tarantool.org\">Tarantool</a></strong> is an open source <a href=\"/radar/tools/nosql\">NoSQL</a> solution that combines database and cache into one entity and provides APIs for writing application logic in <a href=\"/radar/languages-and-frameworks/lua\">Lua</a>. Both in-memory and disk-based engines are supported, and users can create multiple indexes (HASH, TREE, RTREE, BITSET) based on their use cases. The data itself is stored in <a href=\"http://msgpack.org\">MessagePack</a> format and uses the same protocol to communicate between clients and server. Tarantool supports write-ahead logs, transactions and asynchronous master-master replication. We are happy with the architectural decision of embracing single-writer policy and cooperative multitasking to handle concurrent connections."
},
{
"name": "TensorFlow",
"description": ""
},
{
"name": "wit.ai",
"description": "Hype surrounding machine intelligence has reached a crescendo, but as with Big Data, useful frameworks and tools are waiting to be discovered among all the hot air. One such tool is <a href=\"https://wit.ai/\"><strong>wit.ai</strong></a>, a SaaS platform that allows developers to create conversational interfaces using natural language processing (NLP). Wit works with either text or speech inputs, helps developers manage conversational intent and allows custom business logic to be implemented using JavaScript. The system is free for commercial and noncommercial use and encourages the creation of open applications. Be aware that you must agree to let Wit use your data in order to improve the service and for its own analysis, so read the <a href=\"https://wit.ai/terms\">terms and conditions</a> carefully. Another contender in this space is the <a href=\"https://dev.botframework.com/\">Microsoft Bot Framework</a>, but it’s available only in limited preview form as of this writing. As with most things Microsoft, we expect the Bot Framework to evolve quickly, so it’s worth keeping an eye on."
},
{
"name": "CMS as a platform",
"description": "We are seeing too many organizations run into trouble as they attempt to use their <strong>CMS as a platform</strong> for delivering large and complex digital applications. This is often driven by the vendor-fueled hope of bypassing unresponsive IT organizations and enabling the business to drag and drop changes directly to production. While we are very supportive of providing content producers with the right tools and workflows, for applications with complex business logic we tend to recommend treating your CMS as a component of your platform (often in a hybrid or headless mode) cooperating cleanly with other services, rather than attempting to implement all of your functionality in the CMS itself."
},
{
"name": "Overambitious API gateways",
"description": "One of our regular complaints is about business smarts implemented in middleware, resulting in transport software with ambitions to run critical application logic. Vendors in the highly competitive API gateway market continue to add features that differentiate their products. This results in <strong>overambitious API gateway</strong> products whose functionality—on top of what is essentially a reverse proxy—encourages designs that are difficult to test and deploy. API gateways can provide utility in dealing with some generic concerns—for example, authentication and rate-limiting—but any domain smarts such as data transformation or rule processing should live in applications or services where they can be controlled by product teams working closely with the domains they support."
},
{
"name": "Superficial private cloud",
"description": "We've seen the indisputable productivity gains that come from deployment of applications and services into mature cloud providers. Much of that gain comes from the ability of teams to deploy and operate their own services with a high degree of autonomy and responsibility. We are now regularly coming across <strong>Superficial Private Cloud</strong> offerings within organizations, where basic virtualization platforms are being given the “cloud” label. Often teams can self-provision a restricted set of fixed service types with limited access and little ability to customize the centrally governed “enterprise blueprints,” leading to kludge solutions. Deployment pace regularly remains constrained by manually provisioned infrastructure such as network, firewall and storage. We encourage organizations to more fully consider the costs of mandating the use of an inadequate private cloud offering."
}
]
},
{
"name": "trial",
"blips": [
{
"name": "Docker",
"description": "We remain excited about <a href=\"https://www.docker.com/\"><strong>Docker</strong></a> as it evolves from a tool to a complex platform of technologies. Development teams love Docker, as the Docker image format makes it easier to achieve parity between development and production, making for reliable deployments. It is a natural fit in a microservices-style application as a packaging mechanism for self-contained services. On the operational front, Docker support in monitoring tools (<a href=\"/radar/tools/sensu\">Sensu</a>, <a href=\"/radar/tools/prometheus\">Prometheus</a>, <a href=\"https://github.com/google/cadvisor\">cAdvisor</a>, etc.), orchestration tools (<a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a>, etc.) and deployment-automation tools reflect the growing maturity of the platform and its readiness for production use. A word of caution, though: There is a prevalent view of Docker and Linux containers in general as being \"lightweight virtualization,\" but we would not recommend using Docker as a secure process-isolation mechanism, though we are paying attention to the introduction of user namespaces and seccomp profiles in version 1.10 in this regard."
},
{
"name": "HSTS",
"description": "<a href=\"https://www.owasp.org/index.php/HTTP_Strict_Transport_Security\">HTTP Strict Transport Security</a> (<strong>HSTS</strong>) is a now widely supported policy that allows websites to protect themselves from downgrade attacks. A downgrade attack in the context of HTTPS is one that can cause users of your site to fall back to HTTP rather than HTTPS, allowing for further attacks such as man-in-the-middle attacks. With HSTS, the server sends a header that informs the browser that it should only use HTTPS to access the website. Browser support is now widespread enough that this easy-to-implement feature should be added to any site using HTTPS. Mozilla’s <a href=\"https://observatory.mozilla.org/\">Observatory</a> can help identify this and other useful headers and configuration options that improve security and privacy. When implementing HSTS, it is critical to verify that all resources load properly over HTTPS, because once HSTS is turned on, there is (almost) no turning back until the expiry time. The directive to include subdomains should be added but, again, a thorough verification that all subdomains support secure transport is required."
},
{
"name": "Linux security modules",
"description": "<a href=\"http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-167.pdf\">Application whitelisting</a> has proven to be <a href=\"http://www.asd.gov.au/infosec/top-mitigations/top-4-strategies-explained.htm#mitigation1\">one of the most effective ways to mitigate cyber intrusion attacks</a>. A convenient way to implement this widely recommended practice is through <strong>Linux security modules</strong>. With SELinux or AppArmor included by default in most Linux distributions, and with more comprehensive tools such as Grsecurity readily available, we have moved this technology into the Adopt ring in this edition. These tools help teams assess questions about who has access to what resources on shared hosts, including contained services. This conservative approach to access management will help teams build security into their SDLC processes."
},
{
"name": "Apache Mesos",
"description": "We've continued to have positive experiences deploying the <strong><a href=\"http://mesos.apache.org/\">Apache Mesos</a> </strong>platform to manage cluster resources for highly distributed systems. Mesos abstracts out underlying computing resources such as CPU and storage, aiming to provide efficient utilization while maintaining isolation. Mesos includes <a href=\"https://mesos.github.io/chronos/\">Chronos</a> for distributed and fault-tolerant execution of scheduled jobs, and <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a> for orchestrating long-running processes in containers."
},
{
"name": "Auth0",
"description": "We have a growing belief that for most scenarios it is rarely worth rolling your own authentication code. Outsourced identity management speeds up delivery, reduces mistakes and tends to enable a faster response to newly discovered vulnerabilities. <strong><a href=\"https://auth0.com/\">Auth0</a></strong> has particularly impressed us in this field for its ease of integration, range of protocols and connectors supported, and rich management API."
},
{
"name": "AWS Lambda",
"description": "Our teams continue to enjoy using <strong><a href=\"https://aws.amazon.com/lambda/\">AWS Lambda</a></strong> and are beginning to use it to experiment with <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>, combining Lambda with the <a href=\"/radar/platforms/amazon-api-gateway\">API Gateway</a>. We do recommend that Lambda functions contain only a moderate amount of code. Ensuring the quality of a solution based on a tangle of many large Lambda functions is difficult, and such a solution may not be cost-effective. For more&nbsp;complex needs, deployments based on containers or VMs are still preferable. In addition, we have run into significant problems using Java for Lambda functions, with erratic latencies up to several seconds as the Lambda container is started. Of course, you can sidestep this issue by using JavaScript or Python, and if Lambda functions do not contain a lot of code, the choice of programming language should not matter too much."
},
{
"name": "Kubernetes",
"description": "<strong><a href=\"http://kubernetes.io/\">Kubernetes</a></strong> is Google's answer to the problem of deploying containers into a cluster of machines, which is becoming an increasingly common scenario. It is not the solution used by Google internally but an open source project that originated at Google and has seen a fair number of external contributions. Since we mentioned Kubernetes on the previous Radar, our initial positive impressions have been confirmed, and we are seeing successful use of Kubernetes in production at our clients."
},
{
"name": "Pivotal Cloud Foundry",
"description": "The PaaS space has seen a lot of movement since we last mentioned <a href=\"/radar/platforms/cloud-foundry\">Cloud Foundry</a> in 2012. While there are various distributions of the open source core, we have been impressed by the offering and ecosystem assembled as <a href=\"http://pivotal.io/platform\"><strong>Pivotal Cloud Foundry</strong></a>. While we expect continued convergence between the unstructured approach (<a href=\"/radar/platforms/docker\">Docker</a>, <a href=\"/radar/platforms/apache-mesos\">Mesos</a>, <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, etc.) and the more structured and opinionated buildpack style offered by Cloud Foundry and others, we see real benefit for organizations that are willing to accept the constraints and rate of evolution to adopt a PaaS. Of particular interest is the speed of development that comes from the simplification and standardization of the interaction between development teams and platform operations."
},
{
"name": "Rancher",
"description": "The emerging Containers as a Service (CaaS) space is seeing a lot of movement and provides a useful option between basic IaaS (Infrastructure as a Service) and more opinionated PaaS (Platform as a Service). While <a href=\"http://rancher.com/\"><strong>Rancher</strong></a> creates less noise than some other players, we have enjoyed the simplicity that it brings to running <a href=\"/radar/platforms/docker\">Docker</a> containers in production. It can run stand-alone as a full solution or in conjunction with tools like <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>."
},
{
"name": "Realm",
"description": "<strong><a href=\"https://realm.io/\">Realm</a></strong> is a database designed for use on mobile devices, with its own persistence engine to achieve high performance. Realm is marketed as a replacement for SQLite and Core Data. Note that migrations are not quite as straightforward as the Realm documentation would have you believe. However, more and more teams are choosing Realm as the persistence mechanism in production environments for mobile applications."
},
{
"name": "Unity beyond gaming",
"description": "After experiencing years of growth as a platform for game development, <strong><a href=\"https://unity3d.com/\">Unity</a></strong> has recently become the platform of choice for VR and AR application development. Whether you’re creating a fully immersive world for the Oculus or HTC Vive headsets, a holographic layer for your newly spatial enterprise application or an AR feature set for your mobile app, Unity likely provides what you need to both prototype it and get it ready for prime time. Many of us at ThoughtWorks believe that VR and AR represent the next significant shift in the computing platform, and for now, Unity is the single most important tool in the toolbox we use to develop for this change. We’ve used Unity to develop all our VR prototypes, as well as AR functionality for headsets and phone/tablet applications."
},
{
"name": ".NET Core",
"description": "<strong><a href=\"https://www.microsoft.com/net/core\">.NET Core</a></strong> is an open source modular product for creating applications that can be easily deployed in Windows, macOS and Linux. .NET Core makes it possible to build cross-platform web applications using <a href=\"http://www.asp.net/core\">ASP.NET Core</a> with a set of tools, libraries and frameworks—another choice for microservices architecture. The community around .NET Core and other related projects has been growing. New tools have appeared and evolved quickly, such as <a href=\"/radar/tools/visual-studio-code\">Visual Studio Code</a>. There are <a href=\"/radar/platforms/docker\">Docker</a> <a href=\"https://www.microsoft.com/net/core#docker\">images</a> based on both Linux and Windows (<a href=\"/radar/platforms/microsoft-nano-server\">Nano Server</a>) with .NET Core that simplify applying a microservice architecture. CoreCLR and CoreFX appeared in the Radar in the past. However, a few months ago Microsoft <a href=\"https://blogs.msdn.microsoft.com/dotnet/2016/06/27/announcing-net-core-1-0\">announced</a> the release of .NET Core 1.0, the first stable version. We see good new opportunities, changes and a vibrant community as reasons to keep assessing this product."
},
{
"name": "Amazon API Gateway",
"description": "<a href=\"https://aws.amazon.com/api-gateway/\"><strong>Amazon API Gateway</strong></a> is Amazon's offering enabling developers to expose API services to Internet clients. It offers the usual API gateway features like traffic management, monitoring, authentication and authorization. Our teams have been using this service to front other AWS capabilities like AWS Lambda as part of <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>. We continue to monitor for the challenges presented by <a href=\"/radar/platforms/overambitious-api-gateways\">overambitious API gateways</a>, but at this stage Amazon's offering appears to be lightweight enough to avoid those problems."
},
{
"name": "Apache Flink",
"description": "Interest continues to build for <strong><a href=\"https://flink.apache.org/\">Apache Flink</a></strong>, a new-generation platform for scalable distributed batch and stream processing. At the core of Apache Flink is a streaming data-flow engine, with support for tabular (SQL-like), graph-processing and machine&nbsp;learning operations. Apache Flink stands out with feature&nbsp;rich capabilities for stream processing: event time, rich streaming window operations, fault tolerance and exactly-once semantics. The project shows significant ongoing activity, with the latest release (1.1) introducing new datasource/sink integrations as well as improved streaming features."
},
{
"name": "AWS Application Load Balancer",
"description": "Amazon recently launched the <a href=\"https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/\"><strong>AWS Application Load Balancer</strong></a> (ALB), a direct replacement for Elastic Load Balancers introduced back in 2009. ALB supports Layer 7 traffic inspection and is built to support modern cloud architecture. If you’re building a microservices-based system using <a href=\"/radar/platforms/aws-ecs\">ECS</a>, the new load balancers will directly understand container hosting and scaling, with multiple containers and ports per EC2 instance. Content-based routing allows segmentation of requests onto groups of target servers, along with independent scaling of those groups. Health checks performed by the load balancers are much improved, with the ability to capture detailed metrics about application performance. We like everything that we see here, and teams have begun to report successful usage of ALB."
},
{
"name": "Cassandra carefully",
"description": "Apache’s <a href=\"http://cassandra.apache.org/\">Cassandra</a> database is a powerful, scalable Big Data solution for storing and processing large amounts of data, often using hundreds of nodes split over multiple worldwide locations. It’s a great tool and we like it, but too often we see teams run into trouble using it. We recommend using <strong>Cassandra carefully</strong>. Teams often misunderstand the use case for Cassandra, attempting to use it as a general-purpose data store when in fact it is optimized for fast reads on large data sets based on predefined keys or indexes. Its dependence on the storage schema can also make it difficult to evolve over time. Cassandra also has significant operational complexity and some rough edges, so unless you absolutely need the scaling it provides, a simpler solution is usually better. If you don’t need Cassandra’s specific use-case and scaling characteristics, you might just be choosing it out of <a href=\"/radar/techniques/big-data-envy\">Big Data envy</a>. Careful use of Cassandra will include extensive automated testing, and we’re happy to recommend <a href=\"https://github.com/jsevellec/cassandra-unit\">CassandraUnit</a> as part of your testing strategy."
},
{
"name": "Electron",
"description": "<strong><a href=\"http://electron.atom.io/\">Electron</a></strong> is a solid framework for building native desktop clients using web technologies such as HTML, CSS and JavaScript. Teams can leverage their web know-how to deliver polished cross-platform desktop clients without spending time learning another set of technologies."
},
{
"name": "Ethereum",
"description": "The hype seems to have peaked for blockchain and cryptocurrencies, as evidenced by the previous firehose-scale announcements in this area slowing to a trickle, and we expect some of the more speculative efforts to die out over time. One of the blockchains, <a href=\"https://www.ethereum.org/\"><strong>Ethereum</strong></a>, is making good progress and is worth watching. Ethereum is a public blockchain with a built-in programming language that allows \"smart contracts\" to be built into it. These are algorithmic movements of \"ether\" (the Ethereum cryptocurrency) in response to activity happening on the blockchain. R3Cev, the consortium building blockchain tech for banks, built its first proofs of concept on Ethereum. Ethereum has been used to build a Distributed Autonomous Organization (DAO)—one of the first \"algorithmic corporations\"—although a recent heist of <a href=\"http://www.coindesk.com/dao-attacked-code-issue-leads-60-million-ether-theft/\">$150m worth of Ether</a> demonstrates that the blockchain and cryptocurrencies are still the Wild West of the technology world."
},
{
"name": "HoloLens",
"description": "In the <strong><a href=\"https://www.microsoft.com/microsoft-hololens/en-us\">HoloLens</a></strong>, Microsoft has delivered the first truly usable AR headset. Not only is it a beautiful piece of industrial design and an eminently comfortable device to wear, but it also clearly demonstrates the promise of AR for the enterprise via its gorgeous optics and deep Windows 10 integration. We expect HoloLens to be the first AR platform on which we deliver substantial application functionality to our clients in the near term, and we look forward to its evolution as it gains broader traction."
},
{
"name": "IndiaStack",
"description": "<strong><a href=\"http://www.indiastack.org\">IndiaStack</a></strong> is a set of Open APIs designed with the goal of transforming India from a data-poor to a data-rich country. The stack emphasizes layered innovation by specifying a minimal set of APIs and encourages the rest of the ecosystem to build custom applications on top of these APIs. <a href=\"http://www.indiastack.org/Resource#Aadhaar\">Aadhaar</a> serves as one of the foundation layers, providing authentication services for more than a billion Indian citizens. In addition, there are services to provide paperless transactions through digital signatures (eSign), unified online payment (UPI) and an electronic consent layer (<a href=\"https://uidai.gov.in/beta/authentication/aadhaar-financial-inclusion/aadhaar-e-kyc.html\">e-KYC</a>) to securely provide Aadhaar details to service providers. We believe in the Open API–driven initiative to bring digital innovation, and the design principles behind IndiaStack could be used as a change agent for other regions/countries."
},
{
"name": "Nomad",
"description": "HashiCorp continues to turn out interesting software. The latest to catch our attention is <a href=\"https://www.nomadproject.io/\"><strong>Nomad</strong></a>, which is competing in the ever-more-populated scheduler arena. Major selling points include not just being limited to containerized workloads, and operating in multi–data center / multiregion deployments."
},
{
"name": "Nuance Mix",
"description": "<strong><a href=\"https://developer.nuance.com/public/index.php?task=mix\">Nuance Mix</a></strong> is a framework for natural language processing from the company that created the speech-to-text technology behind Dragon Speaking and the first roll-out of Siri. This framework supports the creation of grammars that allow for free-form user interaction via voice. The developer defines a domain-specific grammar that the framework can train itself to understand. The outcomes are responses to user input that identify the user's intents and interaction concepts. At first, it is limited to phrases close to the ones used to train it, but over time it can start to identify meaning from more divergent phrasing. Though it is still in beta, the accuracy from early exploration has been compelling, and the eventual product is one to watch for application forms that could benefit from hands-free user interaction—including mobile, IoT, AR, VR and interactive spaces."
},
{
"name": "OpenVR",
"description": "<strong><a href=\"https://github.com/ValveSoftware/openvr\">OpenVR</a></strong> is the underlying SDK in making many of the VR head-mounted displays (HMDs) work with Unity and will likely keep growing in importance. Much of the VR work at ThoughtWorks was built on top of OpenVR, because it will run on any HMD, unlike the other SDKs. Though it is not open source, it is free via the license. The Oculus SDK is more restrictive in its licensing and only works on Oculus devices. <a href=\"http://www.osvr.org/\">OSVR</a>, while truly open source, doesn't seem to have as much adoption yet. If you're going to develop a VR application and target as many devices as possible—and not use Unity or Unreal to develop them—OpenVR is the most concrete and pragmatic solution right now."
},
{
"name": "Tarantool",
"description": "<strong><a href=\"https://tarantool.org\">Tarantool</a></strong> is an open source <a href=\"/radar/tools/nosql\">NoSQL</a> solution that combines database and cache into one entity and provides APIs for writing application logic in <a href=\"/radar/languages-and-frameworks/lua\">Lua</a>. Both in-memory and disk-based engines are supported, and users can create multiple indexes (HASH, TREE, RTREE, BITSET) based on their use cases. The data itself is stored in <a href=\"http://msgpack.org\">MessagePack</a> format and uses the same protocol to communicate between clients and server. Tarantool supports write-ahead logs, transactions and asynchronous master-master replication. We are happy with the architectural decision of embracing single-writer policy and cooperative multitasking to handle concurrent connections."
},
{
"name": "TensorFlow",
"description": ""
},
{
"name": "wit.ai",
"description": "Hype surrounding machine intelligence has reached a crescendo, but as with Big Data, useful frameworks and tools are waiting to be discovered among all the hot air. One such tool is <a href=\"https://wit.ai/\"><strong>wit.ai</strong></a>, a SaaS platform that allows developers to create conversational interfaces using natural language processing (NLP). Wit works with either text or speech inputs, helps developers manage conversational intent and allows custom business logic to be implemented using JavaScript. The system is free for commercial and noncommercial use and encourages the creation of open applications. Be aware that you must agree to let Wit use your data in order to improve the service and for its own analysis, so read the <a href=\"https://wit.ai/terms\">terms and conditions</a> carefully. Another contender in this space is the <a href=\"https://dev.botframework.com/\">Microsoft Bot Framework</a>, but it’s available only in limited preview form as of this writing. As with most things Microsoft, we expect the Bot Framework to evolve quickly, so it’s worth keeping an eye on."
},
{
"name": "CMS as a platform",
"description": "We are seeing too many organizations run into trouble as they attempt to use their <strong>CMS as a platform</strong> for delivering large and complex digital applications. This is often driven by the vendor-fueled hope of bypassing unresponsive IT organizations and enabling the business to drag and drop changes directly to production. While we are very supportive of providing content producers with the right tools and workflows, for applications with complex business logic we tend to recommend treating your CMS as a component of your platform (often in a hybrid or headless mode) cooperating cleanly with other services, rather than attempting to implement all of your functionality in the CMS itself."
},
{
"name": "Overambitious API gateways",
"description": "One of our regular complaints is about business smarts implemented in middleware, resulting in transport software with ambitions to run critical application logic. Vendors in the highly competitive API gateway market continue to add features that differentiate their products. This results in <strong>overambitious API gateway</strong> products whose functionality—on top of what is essentially a reverse proxy—encourages designs that are difficult to test and deploy. API gateways can provide utility in dealing with some generic concerns—for example, authentication and rate-limiting—but any domain smarts such as data transformation or rule processing should live in applications or services where they can be controlled by product teams working closely with the domains they support."
},
{
"name": "Superficial private cloud",
"description": "We've seen the indisputable productivity gains that come from deployment of applications and services into mature cloud providers. Much of that gain comes from the ability of teams to deploy and operate their own services with a high degree of autonomy and responsibility. We are now regularly coming across <strong>Superficial Private Cloud</strong> offerings within organizations, where basic virtualization platforms are being given the “cloud” label. Often teams can self-provision a restricted set of fixed service types with limited access and little ability to customize the centrally governed “enterprise blueprints,” leading to kludge solutions. Deployment pace regularly remains constrained by manually provisioned infrastructure such as network, firewall and storage. We encourage organizations to more fully consider the costs of mandating the use of an inadequate private cloud offering."
}
]
},
{
"name": "assess",
"blips": [
{
"name": "Docker",
"description": "We remain excited about <a href=\"https://www.docker.com/\"><strong>Docker</strong></a> as it evolves from a tool to a complex platform of technologies. Development teams love Docker, as the Docker image format makes it easier to achieve parity between development and production, making for reliable deployments. It is a natural fit in a microservices-style application as a packaging mechanism for self-contained services. On the operational front, Docker support in monitoring tools (<a href=\"/radar/tools/sensu\">Sensu</a>, <a href=\"/radar/tools/prometheus\">Prometheus</a>, <a href=\"https://github.com/google/cadvisor\">cAdvisor</a>, etc.), orchestration tools (<a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a>, etc.) and deployment-automation tools reflect the growing maturity of the platform and its readiness for production use. A word of caution, though: There is a prevalent view of Docker and Linux containers in general as being \"lightweight virtualization,\" but we would not recommend using Docker as a secure process-isolation mechanism, though we are paying attention to the introduction of user namespaces and seccomp profiles in version 1.10 in this regard."
},
{
"name": "HSTS",
"description": "<a href=\"https://www.owasp.org/index.php/HTTP_Strict_Transport_Security\">HTTP Strict Transport Security</a> (<strong>HSTS</strong>) is a now widely supported policy that allows websites to protect themselves from downgrade attacks. A downgrade attack in the context of HTTPS is one that can cause users of your site to fall back to HTTP rather than HTTPS, allowing for further attacks such as man-in-the-middle attacks. With HSTS, the server sends a header that informs the browser that it should only use HTTPS to access the website. Browser support is now widespread enough that this easy-to-implement feature should be added to any site using HTTPS. Mozilla’s <a href=\"https://observatory.mozilla.org/\">Observatory</a> can help identify this and other useful headers and configuration options that improve security and privacy. When implementing HSTS, it is critical to verify that all resources load properly over HTTPS, because once HSTS is turned on, there is (almost) no turning back until the expiry time. The directive to include subdomains should be added but, again, a thorough verification that all subdomains support secure transport is required."
},
{
"name": "Linux security modules",
"description": "<a href=\"http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-167.pdf\">Application whitelisting</a> has proven to be <a href=\"http://www.asd.gov.au/infosec/top-mitigations/top-4-strategies-explained.htm#mitigation1\">one of the most effective ways to mitigate cyber intrusion attacks</a>. A convenient way to implement this widely recommended practice is through <strong>Linux security modules</strong>. With SELinux or AppArmor included by default in most Linux distributions, and with more comprehensive tools such as Grsecurity readily available, we have moved this technology into the Adopt ring in this edition. These tools help teams assess questions about who has access to what resources on shared hosts, including contained services. This conservative approach to access management will help teams build security into their SDLC processes."
},
{
"name": "Apache Mesos",
"description": "We've continued to have positive experiences deploying the <strong><a href=\"http://mesos.apache.org/\">Apache Mesos</a> </strong>platform to manage cluster resources for highly distributed systems. Mesos abstracts out underlying computing resources such as CPU and storage, aiming to provide efficient utilization while maintaining isolation. Mesos includes <a href=\"https://mesos.github.io/chronos/\">Chronos</a> for distributed and fault-tolerant execution of scheduled jobs, and <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a> for orchestrating long-running processes in containers."
},
{
"name": "Auth0",
"description": "We have a growing belief that for most scenarios it is rarely worth rolling your own authentication code. Outsourced identity management speeds up delivery, reduces mistakes and tends to enable a faster response to newly discovered vulnerabilities. <strong><a href=\"https://auth0.com/\">Auth0</a></strong> has particularly impressed us in this field for its ease of integration, range of protocols and connectors supported, and rich management API."
},
{
"name": "AWS Lambda",
"description": "Our teams continue to enjoy using <strong><a href=\"https://aws.amazon.com/lambda/\">AWS Lambda</a></strong> and are beginning to use it to experiment with <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>, combining Lambda with the <a href=\"/radar/platforms/amazon-api-gateway\">API Gateway</a>. We do recommend that Lambda functions contain only a moderate amount of code. Ensuring the quality of a solution based on a tangle of many large Lambda functions is difficult, and such a solution may not be cost-effective. For more&nbsp;complex needs, deployments based on containers or VMs are still preferable. In addition, we have run into significant problems using Java for Lambda functions, with erratic latencies up to several seconds as the Lambda container is started. Of course, you can sidestep this issue by using JavaScript or Python, and if Lambda functions do not contain a lot of code, the choice of programming language should not matter too much."
},
{
"name": "Kubernetes",
"description": "<strong><a href=\"http://kubernetes.io/\">Kubernetes</a></strong> is Google's answer to the problem of deploying containers into a cluster of machines, which is becoming an increasingly common scenario. It is not the solution used by Google internally but an open source project that originated at Google and has seen a fair number of external contributions. Since we mentioned Kubernetes on the previous Radar, our initial positive impressions have been confirmed, and we are seeing successful use of Kubernetes in production at our clients."
},
{
"name": "Pivotal Cloud Foundry",
"description": "The PaaS space has seen a lot of movement since we last mentioned <a href=\"/radar/platforms/cloud-foundry\">Cloud Foundry</a> in 2012. While there are various distributions of the open source core, we have been impressed by the offering and ecosystem assembled as <a href=\"http://pivotal.io/platform\"><strong>Pivotal Cloud Foundry</strong></a>. While we expect continued convergence between the unstructured approach (<a href=\"/radar/platforms/docker\">Docker</a>, <a href=\"/radar/platforms/apache-mesos\">Mesos</a>, <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, etc.) and the more structured and opinionated buildpack style offered by Cloud Foundry and others, we see real benefit for organizations that are willing to accept the constraints and rate of evolution to adopt a PaaS. Of particular interest is the speed of development that comes from the simplification and standardization of the interaction between development teams and platform operations."
},
{
"name": "Rancher",
"description": "The emerging Containers as a Service (CaaS) space is seeing a lot of movement and provides a useful option between basic IaaS (Infrastructure as a Service) and more opinionated PaaS (Platform as a Service). While <a href=\"http://rancher.com/\"><strong>Rancher</strong></a> creates less noise than some other players, we have enjoyed the simplicity that it brings to running <a href=\"/radar/platforms/docker\">Docker</a> containers in production. It can run stand-alone as a full solution or in conjunction with tools like <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>."
},
{
"name": "Realm",
"description": "<strong><a href=\"https://realm.io/\">Realm</a></strong> is a database designed for use on mobile devices, with its own persistence engine to achieve high performance. Realm is marketed as a replacement for SQLite and Core Data. Note that migrations are not quite as straightforward as the Realm documentation would have you believe. However, more and more teams are choosing Realm as the persistence mechanism in production environments for mobile applications."
},
{
"name": "Unity beyond gaming",
"description": "After experiencing years of growth as a platform for game development, <strong><a href=\"https://unity3d.com/\">Unity</a></strong> has recently become the platform of choice for VR and AR application development. Whether you’re creating a fully immersive world for the Oculus or HTC Vive headsets, a holographic layer for your newly spatial enterprise application or an AR feature set for your mobile app, Unity likely provides what you need to both prototype it and get it ready for prime time. Many of us at ThoughtWorks believe that VR and AR represent the next significant shift in the computing platform, and for now, Unity is the single most important tool in the toolbox we use to develop for this change. We’ve used Unity to develop all our VR prototypes, as well as AR functionality for headsets and phone/tablet applications."
},
{
"name": ".NET Core",
"description": "<strong><a href=\"https://www.microsoft.com/net/core\">.NET Core</a></strong> is an open source modular product for creating applications that can be easily deployed in Windows, macOS and Linux. .NET Core makes it possible to build cross-platform web applications using <a href=\"http://www.asp.net/core\">ASP.NET Core</a> with a set of tools, libraries and frameworks—another choice for microservices architecture. The community around .NET Core and other related projects has been growing. New tools have appeared and evolved quickly, such as <a href=\"/radar/tools/visual-studio-code\">Visual Studio Code</a>. There are <a href=\"/radar/platforms/docker\">Docker</a> <a href=\"https://www.microsoft.com/net/core#docker\">images</a> based on both Linux and Windows (<a href=\"/radar/platforms/microsoft-nano-server\">Nano Server</a>) with .NET Core that simplify applying a microservice architecture. CoreCLR and CoreFX appeared in the Radar in the past. However, a few months ago Microsoft <a href=\"https://blogs.msdn.microsoft.com/dotnet/2016/06/27/announcing-net-core-1-0\">announced</a> the release of .NET Core 1.0, the first stable version. We see good new opportunities, changes and a vibrant community as reasons to keep assessing this product."
},
{
"name": "Amazon API Gateway",
"description": "<a href=\"https://aws.amazon.com/api-gateway/\"><strong>Amazon API Gateway</strong></a> is Amazon's offering enabling developers to expose API services to Internet clients. It offers the usual API gateway features like traffic management, monitoring, authentication and authorization. Our teams have been using this service to front other AWS capabilities like AWS Lambda as part of <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>. We continue to monitor for the challenges presented by <a href=\"/radar/platforms/overambitious-api-gateways\">overambitious API gateways</a>, but at this stage Amazon's offering appears to be lightweight enough to avoid those problems."
},
{
"name": "Apache Flink",
"description": "Interest continues to build for <strong><a href=\"https://flink.apache.org/\">Apache Flink</a></strong>, a new-generation platform for scalable distributed batch and stream processing. At the core of Apache Flink is a streaming data-flow engine, with support for tabular (SQL-like), graph-processing and machine&nbsp;learning operations. Apache Flink stands out with feature&nbsp;rich capabilities for stream processing: event time, rich streaming window operations, fault tolerance and exactly-once semantics. The project shows significant ongoing activity, with the latest release (1.1) introducing new datasource/sink integrations as well as improved streaming features."
},
{
"name": "AWS Application Load Balancer",
"description": "Amazon recently launched the <a href=\"https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/\"><strong>AWS Application Load Balancer</strong></a> (ALB), a direct replacement for Elastic Load Balancers introduced back in 2009. ALB supports Layer 7 traffic inspection and is built to support modern cloud architecture. If you’re building a microservices-based system using <a href=\"/radar/platforms/aws-ecs\">ECS</a>, the new load balancers will directly understand container hosting and scaling, with multiple containers and ports per EC2 instance. Content-based routing allows segmentation of requests onto groups of target servers, along with independent scaling of those groups. Health checks performed by the load balancers are much improved, with the ability to capture detailed metrics about application performance. We like everything that we see here, and teams have begun to report successful usage of ALB."
},
{
"name": "Cassandra carefully",
"description": "Apache’s <a href=\"http://cassandra.apache.org/\">Cassandra</a> database is a powerful, scalable Big Data solution for storing and processing large amounts of data, often using hundreds of nodes split over multiple worldwide locations. It’s a great tool and we like it, but too often we see teams run into trouble using it. We recommend using <strong>Cassandra carefully</strong>. Teams often misunderstand the use case for Cassandra, attempting to use it as a general-purpose data store when in fact it is optimized for fast reads on large data sets based on predefined keys or indexes. Its dependence on the storage schema can also make it difficult to evolve over time. Cassandra also has significant operational complexity and some rough edges, so unless you absolutely need the scaling it provides, a simpler solution is usually better. If you don’t need Cassandra’s specific use-case and scaling characteristics, you might just be choosing it out of <a href=\"/radar/techniques/big-data-envy\">Big Data envy</a>. Careful use of Cassandra will include extensive automated testing, and we’re happy to recommend <a href=\"https://github.com/jsevellec/cassandra-unit\">CassandraUnit</a> as part of your testing strategy."
},
{
"name": "Electron",
"description": "<strong><a href=\"http://electron.atom.io/\">Electron</a></strong> is a solid framework for building native desktop clients using web technologies such as HTML, CSS and JavaScript. Teams can leverage their web know-how to deliver polished cross-platform desktop clients without spending time learning another set of technologies."
},
{
"name": "Ethereum",
"description": "The hype seems to have peaked for blockchain and cryptocurrencies, as evidenced by the previous firehose-scale announcements in this area slowing to a trickle, and we expect some of the more speculative efforts to die out over time. One of the blockchains, <a href=\"https://www.ethereum.org/\"><strong>Ethereum</strong></a>, is making good progress and is worth watching. Ethereum is a public blockchain with a built-in programming language that allows \"smart contracts\" to be built into it. These are algorithmic movements of \"ether\" (the Ethereum cryptocurrency) in response to activity happening on the blockchain. R3Cev, the consortium building blockchain tech for banks, built its first proofs of concept on Ethereum. Ethereum has been used to build a Distributed Autonomous Organization (DAO)—one of the first \"algorithmic corporations\"—although a recent heist of <a href=\"http://www.coindesk.com/dao-attacked-code-issue-leads-60-million-ether-theft/\">$150m worth of Ether</a> demonstrates that the blockchain and cryptocurrencies are still the Wild West of the technology world."
},
{
"name": "HoloLens",
"description": "In the <strong><a href=\"https://www.microsoft.com/microsoft-hololens/en-us\">HoloLens</a></strong>, Microsoft has delivered the first truly usable AR headset. Not only is it a beautiful piece of industrial design and an eminently comfortable device to wear, but it also clearly demonstrates the promise of AR for the enterprise via its gorgeous optics and deep Windows 10 integration. We expect HoloLens to be the first AR platform on which we deliver substantial application functionality to our clients in the near term, and we look forward to its evolution as it gains broader traction."
},
{
"name": "IndiaStack",
"description": "<strong><a href=\"http://www.indiastack.org\">IndiaStack</a></strong> is a set of Open APIs designed with the goal of transforming India from a data-poor to a data-rich country. The stack emphasizes layered innovation by specifying a minimal set of APIs and encourages the rest of the ecosystem to build custom applications on top of these APIs. <a href=\"http://www.indiastack.org/Resource#Aadhaar\">Aadhaar</a> serves as one of the foundation layers, providing authentication services for more than a billion Indian citizens. In addition, there are services to provide paperless transactions through digital signatures (eSign), unified online payment (UPI) and an electronic consent layer (<a href=\"https://uidai.gov.in/beta/authentication/aadhaar-financial-inclusion/aadhaar-e-kyc.html\">e-KYC</a>) to securely provide Aadhaar details to service providers. We believe in the Open API–driven initiative to bring digital innovation, and the design principles behind IndiaStack could be used as a change agent for other regions/countries."
},
{
"name": "Nomad",
"description": "HashiCorp continues to turn out interesting software. The latest to catch our attention is <a href=\"https://www.nomadproject.io/\"><strong>Nomad</strong></a>, which is competing in the ever-more-populated scheduler arena. Major selling points include not just being limited to containerized workloads, and operating in multi–data center / multiregion deployments."
},
{
"name": "Nuance Mix",
"description": "<strong><a href=\"https://developer.nuance.com/public/index.php?task=mix\">Nuance Mix</a></strong> is a framework for natural language processing from the company that created the speech-to-text technology behind Dragon Speaking and the first roll-out of Siri. This framework supports the creation of grammars that allow for free-form user interaction via voice. The developer defines a domain-specific grammar that the framework can train itself to understand. The outcomes are responses to user input that identify the user's intents and interaction concepts. At first, it is limited to phrases close to the ones used to train it, but over time it can start to identify meaning from more divergent phrasing. Though it is still in beta, the accuracy from early exploration has been compelling, and the eventual product is one to watch for application forms that could benefit from hands-free user interaction—including mobile, IoT, AR, VR and interactive spaces."
},
{
"name": "OpenVR",
"description": "<strong><a href=\"https://github.com/ValveSoftware/openvr\">OpenVR</a></strong> is the underlying SDK in making many of the VR head-mounted displays (HMDs) work with Unity and will likely keep growing in importance. Much of the VR work at ThoughtWorks was built on top of OpenVR, because it will run on any HMD, unlike the other SDKs. Though it is not open source, it is free via the license. The Oculus SDK is more restrictive in its licensing and only works on Oculus devices. <a href=\"http://www.osvr.org/\">OSVR</a>, while truly open source, doesn't seem to have as much adoption yet. If you're going to develop a VR application and target as many devices as possible—and not use Unity or Unreal to develop them—OpenVR is the most concrete and pragmatic solution right now."
},
{
"name": "Tarantool",
"description": "<strong><a href=\"https://tarantool.org\">Tarantool</a></strong> is an open source <a href=\"/radar/tools/nosql\">NoSQL</a> solution that combines database and cache into one entity and provides APIs for writing application logic in <a href=\"/radar/languages-and-frameworks/lua\">Lua</a>. Both in-memory and disk-based engines are supported, and users can create multiple indexes (HASH, TREE, RTREE, BITSET) based on their use cases. The data itself is stored in <a href=\"http://msgpack.org\">MessagePack</a> format and uses the same protocol to communicate between clients and server. Tarantool supports write-ahead logs, transactions and asynchronous master-master replication. We are happy with the architectural decision of embracing single-writer policy and cooperative multitasking to handle concurrent connections."
},
{
"name": "TensorFlow",
"description": ""
},
{
"name": "wit.ai",
"description": "Hype surrounding machine intelligence has reached a crescendo, but as with Big Data, useful frameworks and tools are waiting to be discovered among all the hot air. One such tool is <a href=\"https://wit.ai/\"><strong>wit.ai</strong></a>, a SaaS platform that allows developers to create conversational interfaces using natural language processing (NLP). Wit works with either text or speech inputs, helps developers manage conversational intent and allows custom business logic to be implemented using JavaScript. The system is free for commercial and noncommercial use and encourages the creation of open applications. Be aware that you must agree to let Wit use your data in order to improve the service and for its own analysis, so read the <a href=\"https://wit.ai/terms\">terms and conditions</a> carefully. Another contender in this space is the <a href=\"https://dev.botframework.com/\">Microsoft Bot Framework</a>, but it’s available only in limited preview form as of this writing. As with most things Microsoft, we expect the Bot Framework to evolve quickly, so it’s worth keeping an eye on."
},
{
"name": "CMS as a platform",
"description": "We are seeing too many organizations run into trouble as they attempt to use their <strong>CMS as a platform</strong> for delivering large and complex digital applications. This is often driven by the vendor-fueled hope of bypassing unresponsive IT organizations and enabling the business to drag and drop changes directly to production. While we are very supportive of providing content producers with the right tools and workflows, for applications with complex business logic we tend to recommend treating your CMS as a component of your platform (often in a hybrid or headless mode) cooperating cleanly with other services, rather than attempting to implement all of your functionality in the CMS itself."
},
{
"name": "Overambitious API gateways",
"description": "One of our regular complaints is about business smarts implemented in middleware, resulting in transport software with ambitions to run critical application logic. Vendors in the highly competitive API gateway market continue to add features that differentiate their products. This results in <strong>overambitious API gateway</strong> products whose functionality—on top of what is essentially a reverse proxy—encourages designs that are difficult to test and deploy. API gateways can provide utility in dealing with some generic concerns—for example, authentication and rate-limiting—but any domain smarts such as data transformation or rule processing should live in applications or services where they can be controlled by product teams working closely with the domains they support."
},
{
"name": "Superficial private cloud",
"description": "We've seen the indisputable productivity gains that come from deployment of applications and services into mature cloud providers. Much of that gain comes from the ability of teams to deploy and operate their own services with a high degree of autonomy and responsibility. We are now regularly coming across <strong>Superficial Private Cloud</strong> offerings within organizations, where basic virtualization platforms are being given the “cloud” label. Often teams can self-provision a restricted set of fixed service types with limited access and little ability to customize the centrally governed “enterprise blueprints,” leading to kludge solutions. Deployment pace regularly remains constrained by manually provisioned infrastructure such as network, firewall and storage. We encourage organizations to more fully consider the costs of mandating the use of an inadequate private cloud offering."
}
]
},
{
"name": "hold",
"blips": [
{
"name": "Docker",
"description": "We remain excited about <a href=\"https://www.docker.com/\"><strong>Docker</strong></a> as it evolves from a tool to a complex platform of technologies. Development teams love Docker, as the Docker image format makes it easier to achieve parity between development and production, making for reliable deployments. It is a natural fit in a microservices-style application as a packaging mechanism for self-contained services. On the operational front, Docker support in monitoring tools (<a href=\"/radar/tools/sensu\">Sensu</a>, <a href=\"/radar/tools/prometheus\">Prometheus</a>, <a href=\"https://github.com/google/cadvisor\">cAdvisor</a>, etc.), orchestration tools (<a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a>, etc.) and deployment-automation tools reflect the growing maturity of the platform and its readiness for production use. A word of caution, though: There is a prevalent view of Docker and Linux containers in general as being \"lightweight virtualization,\" but we would not recommend using Docker as a secure process-isolation mechanism, though we are paying attention to the introduction of user namespaces and seccomp profiles in version 1.10 in this regard."
},
{
"name": "HSTS",
"description": "<a href=\"https://www.owasp.org/index.php/HTTP_Strict_Transport_Security\">HTTP Strict Transport Security</a> (<strong>HSTS</strong>) is a now widely supported policy that allows websites to protect themselves from downgrade attacks. A downgrade attack in the context of HTTPS is one that can cause users of your site to fall back to HTTP rather than HTTPS, allowing for further attacks such as man-in-the-middle attacks. With HSTS, the server sends a header that informs the browser that it should only use HTTPS to access the website. Browser support is now widespread enough that this easy-to-implement feature should be added to any site using HTTPS. Mozilla’s <a href=\"https://observatory.mozilla.org/\">Observatory</a> can help identify this and other useful headers and configuration options that improve security and privacy. When implementing HSTS, it is critical to verify that all resources load properly over HTTPS, because once HSTS is turned on, there is (almost) no turning back until the expiry time. The directive to include subdomains should be added but, again, a thorough verification that all subdomains support secure transport is required."
},
{
"name": "Linux security modules",
"description": "<a href=\"http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-167.pdf\">Application whitelisting</a> has proven to be <a href=\"http://www.asd.gov.au/infosec/top-mitigations/top-4-strategies-explained.htm#mitigation1\">one of the most effective ways to mitigate cyber intrusion attacks</a>. A convenient way to implement this widely recommended practice is through <strong>Linux security modules</strong>. With SELinux or AppArmor included by default in most Linux distributions, and with more comprehensive tools such as Grsecurity readily available, we have moved this technology into the Adopt ring in this edition. These tools help teams assess questions about who has access to what resources on shared hosts, including contained services. This conservative approach to access management will help teams build security into their SDLC processes."
},
{
"name": "Apache Mesos",
"description": "We've continued to have positive experiences deploying the <strong><a href=\"http://mesos.apache.org/\">Apache Mesos</a> </strong>platform to manage cluster resources for highly distributed systems. Mesos abstracts out underlying computing resources such as CPU and storage, aiming to provide efficient utilization while maintaining isolation. Mesos includes <a href=\"https://mesos.github.io/chronos/\">Chronos</a> for distributed and fault-tolerant execution of scheduled jobs, and <a href=\"https://mesosphere.github.io/marathon/\">Marathon</a> for orchestrating long-running processes in containers."
},
{
"name": "Auth0",
"description": "We have a growing belief that for most scenarios it is rarely worth rolling your own authentication code. Outsourced identity management speeds up delivery, reduces mistakes and tends to enable a faster response to newly discovered vulnerabilities. <strong><a href=\"https://auth0.com/\">Auth0</a></strong> has particularly impressed us in this field for its ease of integration, range of protocols and connectors supported, and rich management API."
},
{
"name": "AWS Lambda",
"description": "Our teams continue to enjoy using <strong><a href=\"https://aws.amazon.com/lambda/\">AWS Lambda</a></strong> and are beginning to use it to experiment with <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>, combining Lambda with the <a href=\"/radar/platforms/amazon-api-gateway\">API Gateway</a>. We do recommend that Lambda functions contain only a moderate amount of code. Ensuring the quality of a solution based on a tangle of many large Lambda functions is difficult, and such a solution may not be cost-effective. For more&nbsp;complex needs, deployments based on containers or VMs are still preferable. In addition, we have run into significant problems using Java for Lambda functions, with erratic latencies up to several seconds as the Lambda container is started. Of course, you can sidestep this issue by using JavaScript or Python, and if Lambda functions do not contain a lot of code, the choice of programming language should not matter too much."
},
{
"name": "Kubernetes",
"description": "<strong><a href=\"http://kubernetes.io/\">Kubernetes</a></strong> is Google's answer to the problem of deploying containers into a cluster of machines, which is becoming an increasingly common scenario. It is not the solution used by Google internally but an open source project that originated at Google and has seen a fair number of external contributions. Since we mentioned Kubernetes on the previous Radar, our initial positive impressions have been confirmed, and we are seeing successful use of Kubernetes in production at our clients."
},
{
"name": "Pivotal Cloud Foundry",
"description": "The PaaS space has seen a lot of movement since we last mentioned <a href=\"/radar/platforms/cloud-foundry\">Cloud Foundry</a> in 2012. While there are various distributions of the open source core, we have been impressed by the offering and ecosystem assembled as <a href=\"http://pivotal.io/platform\"><strong>Pivotal Cloud Foundry</strong></a>. While we expect continued convergence between the unstructured approach (<a href=\"/radar/platforms/docker\">Docker</a>, <a href=\"/radar/platforms/apache-mesos\">Mesos</a>, <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>, etc.) and the more structured and opinionated buildpack style offered by Cloud Foundry and others, we see real benefit for organizations that are willing to accept the constraints and rate of evolution to adopt a PaaS. Of particular interest is the speed of development that comes from the simplification and standardization of the interaction between development teams and platform operations."
},
{
"name": "Rancher",
"description": "The emerging Containers as a Service (CaaS) space is seeing a lot of movement and provides a useful option between basic IaaS (Infrastructure as a Service) and more opinionated PaaS (Platform as a Service). While <a href=\"http://rancher.com/\"><strong>Rancher</strong></a> creates less noise than some other players, we have enjoyed the simplicity that it brings to running <a href=\"/radar/platforms/docker\">Docker</a> containers in production. It can run stand-alone as a full solution or in conjunction with tools like <a href=\"/radar/platforms/kubernetes\">Kubernetes</a>."
},
{
"name": "Realm",
"description": "<strong><a href=\"https://realm.io/\">Realm</a></strong> is a database designed for use on mobile devices, with its own persistence engine to achieve high performance. Realm is marketed as a replacement for SQLite and Core Data. Note that migrations are not quite as straightforward as the Realm documentation would have you believe. However, more and more teams are choosing Realm as the persistence mechanism in production environments for mobile applications."
},
{
"name": "Unity beyond gaming",
"description": "After experiencing years of growth as a platform for game development, <strong><a href=\"https://unity3d.com/\">Unity</a></strong> has recently become the platform of choice for VR and AR application development. Whether you’re creating a fully immersive world for the Oculus or HTC Vive headsets, a holographic layer for your newly spatial enterprise application or an AR feature set for your mobile app, Unity likely provides what you need to both prototype it and get it ready for prime time. Many of us at ThoughtWorks believe that VR and AR represent the next significant shift in the computing platform, and for now, Unity is the single most important tool in the toolbox we use to develop for this change. We’ve used Unity to develop all our VR prototypes, as well as AR functionality for headsets and phone/tablet applications."
},
{
"name": ".NET Core",
"description": "<strong><a href=\"https://www.microsoft.com/net/core\">.NET Core</a></strong> is an open source modular product for creating applications that can be easily deployed in Windows, macOS and Linux. .NET Core makes it possible to build cross-platform web applications using <a href=\"http://www.asp.net/core\">ASP.NET Core</a> with a set of tools, libraries and frameworks—another choice for microservices architecture. The community around .NET Core and other related projects has been growing. New tools have appeared and evolved quickly, such as <a href=\"/radar/tools/visual-studio-code\">Visual Studio Code</a>. There are <a href=\"/radar/platforms/docker\">Docker</a> <a href=\"https://www.microsoft.com/net/core#docker\">images</a> based on both Linux and Windows (<a href=\"/radar/platforms/microsoft-nano-server\">Nano Server</a>) with .NET Core that simplify applying a microservice architecture. CoreCLR and CoreFX appeared in the Radar in the past. However, a few months ago Microsoft <a href=\"https://blogs.msdn.microsoft.com/dotnet/2016/06/27/announcing-net-core-1-0\">announced</a> the release of .NET Core 1.0, the first stable version. We see good new opportunities, changes and a vibrant community as reasons to keep assessing this product."
},
{
"name": "Amazon API Gateway",
"description": "<a href=\"https://aws.amazon.com/api-gateway/\"><strong>Amazon API Gateway</strong></a> is Amazon's offering enabling developers to expose API services to Internet clients. It offers the usual API gateway features like traffic management, monitoring, authentication and authorization. Our teams have been using this service to front other AWS capabilities like AWS Lambda as part of <a href=\"/radar/techniques/serverless-architecture\">serverless architectures</a>. We continue to monitor for the challenges presented by <a href=\"/radar/platforms/overambitious-api-gateways\">overambitious API gateways</a>, but at this stage Amazon's offering appears to be lightweight enough to avoid those problems."
},
{
"name": "Apache Flink",
"description": "Interest continues to build for <strong><a href=\"https://flink.apache.org/\">Apache Flink</a></strong>, a new-generation platform for scalable distributed batch and stream processing. At the core of Apache Flink is a streaming data-flow engine, with support for tabular (SQL-like), graph-processing and machine&nbsp;learning operations. Apache Flink stands out with feature&nbsp;rich capabilities for stream processing: event time, rich streaming window operations, fault tolerance and exactly-once semantics. The project shows significant ongoing activity, with the latest release (1.1) introducing new datasource/sink integrations as well as improved streaming features."
},
{
"name": "AWS Application Load Balancer",
"description": "Amazon recently launched the <a href=\"https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/\"><strong>AWS Application Load Balancer</strong></a> (ALB), a direct replacement for Elastic Load Balancers introduced back in 2009. ALB supports Layer 7 traffic inspection and is built to support modern cloud architecture. If you’re building a microservices-based system using <a href=\"/radar/platforms/aws-ecs\">ECS</a>, the new load balancers will directly understand container hosting and scaling, with multiple containers and ports per EC2 instance. Content-based routing allows segmentation of requests onto groups of target servers, along with independent scaling of those groups. Health checks performed by the load balancers are much improved, with the ability to capture detailed metrics about application performance. We like everything that we see here, and teams have begun to report successful usage of ALB."
},
{
"name": "Cassandra carefully",
"description": "Apache’s <a href=\"http://cassandra.apache.org/\">Cassandra</a> database is a powerful, scalable Big Data solution for storing and processing large amounts of data, often using hundreds of nodes split over multiple worldwide locations. It’s a great tool and we like it, but too often we see teams run into trouble using it. We recommend using <strong>Cassandra carefully</strong>. Teams often misunderstand the use case for Cassandra, attempting to use it as a general-purpose data store when in fact it is optimized for fast reads on large data sets based on predefined keys or indexes. Its dependence on the storage schema can also make it difficult to evolve over time. Cassandra also has significant operational complexity and some rough edges, so unless you absolutely need the scaling it provides, a simpler solution is usually better. If you don’t need Cassandra’s specific use-case and scaling characteristics, you might just be choosing it out of <a href=\"/radar/techniques/big-data-envy\">Big Data envy</a>. Careful use of Cassandra will include extensive automated testing, and we’re happy to recommend <a href=\"https://github.com/jsevellec/cassandra-unit\">CassandraUnit</a> as part of your testing strategy."
},
{
"name": "Electron",
"description": "<strong><a href=\"http://electron.atom.io/\">Electron</a></strong> is a solid framework for building native desktop clients using web technologies such as HTML, CSS and JavaScript. Teams can leverage their web know-how to deliver polished cross-platform desktop clients without spending time learning another set of technologies."
},
{
"name": "Ethereum",
"description": "The hype seems to have peaked for blockchain and cryptocurrencies, as evidenced by the previous firehose-scale announcements in this area slowing to a trickle, and we expect some of the more speculative efforts to die out over time. One of the blockchains, <a href=\"https://www.ethereum.org/\"><strong>Ethereum</strong></a>, is making good progress and is worth watching. Ethereum is a public blockchain with a built-in programming language that allows \"smart contracts\" to be built into it. These are algorithmic movements of \"ether\" (the Ethereum cryptocurrency) in response to activity happening on the blockchain. R3Cev, the consortium building blockchain tech for banks, built its first proofs of concept on Ethereum. Ethereum has been used to build a Distributed Autonomous Organization (DAO)—one of the first \"algorithmic corporations\"—although a recent heist of <a href=\"http://www.coindesk.com/dao-attacked-code-issue-leads-60-million-ether-theft/\">$150m worth of Ether</a> demonstrates that the blockchain and cryptocurrencies are still the Wild West of the technology world."
},
{
"name": "HoloLens",
"description": "In the <strong><a href=\"https://www.microsoft.com/microsoft-hololens/en-us\">HoloLens</a></strong>, Microsoft has delivered the first truly usable AR headset. Not only is it a beautiful piece of industrial design and an eminently comfortable device to wear, but it also clearly demonstrates the promise of AR for the enterprise via its gorgeous optics and deep Windows 10 integration. We expect HoloLens to be the first AR platform on which we deliver substantial application functionality to our clients in the near term, and we look forward to its evolution as it gains broader traction."
},
{
"name": "IndiaStack",
"description": "<strong><a href=\"http://www.indiastack.org\">IndiaStack</a></strong> is a set of Open APIs designed with the goal of transforming India from a data-poor to a data-rich country. The stack emphasizes layered innovation by specifying a minimal set of APIs and encourages the rest of the ecosystem to build custom applications on top of these APIs. <a href=\"http://www.indiastack.org/Resource#Aadhaar\">Aadhaar</a> serves as one of the foundation layers, providing authentication services for more than a billion Indian citizens. In addition, there are services to provide paperless transactions through digital signatures (eSign), unified online payment (UPI) and an electronic consent layer (<a href=\"https://uidai.gov.in/beta/authentication/aadhaar-financial-inclusion/aadhaar-e-kyc.html\">e-KYC</a>) to securely provide Aadhaar details to service providers. We believe in the Open API–driven initiative to bring digital innovation, and the design principles behind IndiaStack could be used as a change agent for other regions/countries."
},
{
"name": "Nomad",
"description": "HashiCorp continues to turn out interesting software. The latest to catch our attention is <a href=\"https://www.nomadproject.io/\"><strong>Nomad</strong></a>, which is competing in the ever-more-populated scheduler arena. Major selling points include not just being limited to containerized workloads, and operating in multi–data center / multiregion deployments."
},
{
"name": "Nuance Mix",
"description": "<strong><a href=\"https://developer.nuance.com/public/index.php?task=mix\">Nuance Mix</a></strong> is a framework for natural language processing from the company that created the speech-to-text technology behind Dragon Speaking and the first roll-out of Siri. This framework supports the creation of grammars that allow for free-form user interaction via voice. The developer defines a domain-specific grammar that the framework can train itself to understand. The outcomes are responses to user input that identify the user's intents and interaction concepts. At first, it is limited to phrases close to the ones used to train it, but over time it can start to identify meaning from more divergent phrasing. Though it is still in beta, the accuracy from early exploration has been compelling, and the eventual product is one to watch for application forms that could benefit from hands-free user interaction—including mobile, IoT, AR, VR and interactive spaces."
},
{
"name": "OpenVR",
"description": "<strong><a href=\"https://github.com/ValveSoftware/openvr\">OpenVR</a></strong> is the underlying SDK in making many of the VR head-mounted displays (HMDs) work with Unity and will likely keep growing in importance. Much of the VR work at ThoughtWorks was built on top of OpenVR, because it will run on any HMD, unlike the other SDKs. Though it is not open source, it is free via the license. The Oculus SDK is more restrictive in its licensing and only works on Oculus devices. <a href=\"http://www.osvr.org/\">OSVR</a>, while truly open source, doesn't seem to have as much adoption yet. If you're going to develop a VR application and target as many devices as possible—and not use Unity or Unreal to develop them—OpenVR is the most concrete and pragmatic solution right now."
},
{
"name": "Tarantool",
"description": "<strong><a href=\"https://tarantool.org\">Tarantool</a></strong> is an open source <a href=\"/radar/tools/nosql\">NoSQL</a> solution that combines database and cache into one entity and provides APIs for writing application logic in <a href=\"/radar/languages-and-frameworks/lua\">Lua</a>. Both in-memory and disk-based engines are supported, and users can create multiple indexes (HASH, TREE, RTREE, BITSET) based on their use cases. The data itself is stored in <a href=\"http://msgpack.org\">MessagePack</a> format and uses the same protocol to communicate between clients and server. Tarantool supports write-ahead logs, transactions and asynchronous master-master replication. We are happy with the architectural decision of embracing single-writer policy and cooperative multitasking to handle concurrent connections."
},
{
"name": "TensorFlow",
"description": ""
},
{
"name": "wit.ai",
"description": "Hype surrounding machine intelligence has reached a crescendo, but as with Big Data, useful frameworks and tools are waiting to be discovered among all the hot air. One such tool is <a href=\"https://wit.ai/\"><strong>wit.ai</strong></a>, a SaaS platform that allows developers to create conversational interfaces using natural language processing (NLP). Wit works with either text or speech inputs, helps developers manage conversational intent and allows custom business logic to be implemented using JavaScript. The system is free for commercial and noncommercial use and encourages the creation of open applications. Be aware that you must agree to let Wit use your data in order to improve the service and for its own analysis, so read the <a href=\"https://wit.ai/terms\">terms and conditions</a> carefully. Another contender in this space is the <a href=\"https://dev.botframework.com/\">Microsoft Bot Framework</a>, but it’s available only in limited preview form as of this writing. As with most things Microsoft, we expect the Bot Framework to evolve quickly, so it’s worth keeping an eye on."
},
{
"name": "CMS as a platform",
"description": "We are seeing too many organizations run into trouble as they attempt to use their <strong>CMS as a platform</strong> for delivering large and complex digital applications. This is often driven by the vendor-fueled hope of bypassing unresponsive IT organizations and enabling the business to drag and drop changes directly to production. While we are very supportive of providing content producers with the right tools and workflows, for applications with complex business logic we tend to recommend treating your CMS as a component of your platform (often in a hybrid or headless mode) cooperating cleanly with other services, rather than attempting to implement all of your functionality in the CMS itself."
},
{
"name": "Overambitious API gateways",
"description": "One of our regular complaints is about business smarts implemented in middleware, resulting in transport software with ambitions to run critical application logic. Vendors in the highly competitive API gateway market continue to add features that differentiate their products. This results in <strong>overambitious API gateway</strong> products whose functionality—on top of what is essentially a reverse proxy—encourages designs that are difficult to test and deploy. API gateways can provide utility in dealing with some generic concerns—for example, authentication and rate-limiting—but any domain smarts such as data transformation or rule processing should live in applications or services where they can be controlled by product teams working closely with the domains they support."
},
{
"name": "Superficial private cloud",
"description": "We've seen the indisputable productivity gains that come from deployment of applications and services into mature cloud providers. Much of that gain comes from the ability of teams to deploy and operate their own services with a high degree of autonomy and responsibility. We are now regularly coming across <strong>Superficial Private Cloud</strong> offerings within organizations, where basic virtualization platforms are being given the “cloud” label. Often teams can self-provision a restricted set of fixed service types with limited access and little ability to customize the centrally governed “enterprise blueprints,” leading to kludge solutions. Deployment pace regularly remains constrained by manually provisioned infrastructure such as network, firewall and storage. We encourage organizations to more fully consider the costs of mandating the use of an inadequate private cloud offering."
}
]
}
],
"Techniques" : [
{
"name": "adopt",
"blips": [
{
"name": "Consumer-driven contract testing",
"description": "We’ve decided to bring <strong>consumer-driven contract testing</strong> back from the archive for this edition even though we had allowed it to fade in the past. The concept isn’t new, but with the mainstream acceptance of microservices, we need to remind people that <a href=\"http://www.martinfowler.com/articles/consumerDrivenContracts.html\">consumer-driven contracts</a> are an essential part of a mature <a href=\"http://martinfowler.com/articles/microservice-testing/\">microservice testing</a> portfolio, enabling independent service deployments. But in addition, we want to point out that consumer-driven contract testing is a technique and an attitude that requires no special tool to implement. We love frameworks like <a href=\"https://github.com/realestate-com-au/pact\">Pact</a> because they make proper contract tests easier to implement in certain contexts. But we have noticed a tendency for teams to focus on the framework rather than on the general practice. Writing Pact tests is not a guarantee that you are creating consumer-driven contracts; likewise, in many situations you should be creating good consumer-driven contracts even where no pre-built testing tool exists."
},
{
"name": "Pipelines as code",
"description": "Teams are pushing for automation across their environments, including their development infrastructure. <strong>Pipelines as code</strong> is defining the deployment pipeline through code instead of configuring a running CI/CD tool. <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://readme.drone.io/usage/overview/\">Drone</a>, <a href=\"/radar/tools/gocd\">GoCD</a> and <a href=\"/radar/tools/concourse-ci\">Concourse</a> are examples that allow usage of this technique. Also, configuration automation tools for CI/CD systems like <a href=\"https://github.com/SpringerSBM/gomatic\">GoMatic</a> can be used to treat the deployment pipeline as code—versioned and tested."
},
{
"name": "Threat Modeling",
"description": "With the number of high-profile security breaches in the past months, software development teams no longer need convincing that they must place an emphasis on writing secure software and dealing with their users' data in a responsible way. The teams face a steep learning curve, though, and the vast number of potential threats—ranging from organized crime and government spying to teenagers who attack systems \"for the lulz\"—can be overwhelming. <a href=\"https://www.owasp.org/index.php/Category:Threat_Modeling\"><strong>Threat Modeling</strong></a> provides a set of techniques that help you identify and classify potential threats early in the development process. It is important to understand that it is only part of a strategy to stay ahead of threats. When used in conjunction with techniques such as establishing cross-functional security requirements to address common risks in the technologies a project uses and using automated security scanners, threat modeling can be a powerful asset."
},
{
"name": "APIs as a product",
"description": "Businesses have wholeheartedly embraced APIs as a way to expose business capabilities to both external and internal developers. APIs promise the ability to experiment quickly with new business ideas by recombining core capabilities. But what differentiates an API from an ordinary enterprise integration service? One difference lies in treating <strong>APIs as a product</strong>, even when the consumer is an internal system. Teams that build APIs should understand the needs of their customers and make the product compelling to them. Products are also improved, maintained and supported over the long term. They should have an owner who advocates for the customer and strives for continual improvement. Products are actively maintained and supported, easy to find and easy to use. In our experience, a product orientation is the missing ingredient that makes the difference between ordinary enterprise integration and an agile business built on a platform of APIs."
},
{
"name": "Bug bounties",
"description": "The use of <strong>bug bounties</strong> continues to grow in popularity for many organizations, including enterprises and notable government bodies. A bug-bounty program encourages participants to identify potentially damaging vulnerabilities in return for reward or recognition. Companies like <a href=\"https://hackerone.com/\">HackerOne</a> and <a href=\"https://bugcrowd.com/\">Bugcrowd</a> offer services to help organizations manage this process more easily, and we're seeing these services gather adoption."
},
{
"name": "Data Lake",
"description": "A <strong><a href=\"http://martinfowler.com/bliki/DataLake.html\">Data Lake</a></strong> is an immutable data store of largely unprocessed \"raw\" data, acting as a source for data analytics. While the technique can clearly be misused, we have used it successfully at clients, hence motivating its move to trial. We continue to recommend other approaches for operational collaborations, limiting the use of the data lake to reporting, analytics and feeding data into data marts."
},
{
"name": "Hosting PII data in the EU",
"description": "In a number of countries, we see government agencies seeking broad access to private, personally identifiable information (PII). The increased use of public cloud solutions makes it more difficult for organizations to protect the data entrusted to them by their users while also respecting all relevant laws. The European Union has some of the most progressive privacy laws, and all the major cloud providers—Amazon, Google and Microsoft—offer multiple data centers and regions within the European Union. Therefore, we recommend that companies, especially those with a global user base, assess the feasibility of a safe haven for their users' data by <strong>hosting PII data in the EU</strong>. Since we wrote about this technique in the last Radar, we have rolled out a new internal system that handles sensitive information relating to all our employees, and we have chosen to host it in a data center located in the European Union."
},
{
"name": "Lightweight Architecture Decision Records",
"description": "Although much documentation can be replaced with highly readable code and tests, in a world of <a href=\"/radar/techniques/evolutionary-architecture\">evolutionary architecture</a> it's important to record certain design decisions for the benefit of future team members and for external oversight. <strong>Lightweight Architecture Decision Records</strong> is <a href=\"http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions\">a technique</a> for capturing important architectural decisions along with their context and consequences. Although these items are often stored in a wiki or collaboration tool, we generally prefer <a href=\"https://github.com/npryce/adr-tools\">storing them in source control</a> with simple markup."
},
{
"name": "Reactive architectures",
"description": "We see continued adoption and success of <strong>reactive architectures</strong>, with reactive language extensions and reactive frameworks being very popular (we added several such blips in this edition of the Radar). User interfaces, in particular, benefit greatly from a reactive style of programming. Our caveats last time still hold true: Architectures based on asynchronous message passing introduce complexity and make the overall system harder to understand—it's no longer possible to simply read the program code and understand what the system does. We recommend assessing the performance and scalability needs of your system before committing to this architectural style."
},
{
"name": "Serverless architecture",
"description": "<a href=\"http://www.martinfowler.com/articles/serverless.html\"><strong>Serverless architecture</strong></a> is an approach that replaces long-running virtual machines with ephemeral compute power that comes into existence on request and disappears immediately after use. Since the last Radar, we have had several teams put applications into production using a \"serverless\" style. Our teams like the approach, it’s working well for them and we consider it a valid architectural choice. Note that serverless doesn’t have to be an all-or-nothing approach: some of our teams have deployed a new chunk of their systems using serverless while sticking to a traditional architectural approach for other pieces."
},
{
"name": "Client-directed query",
"description": "Although many problems that people encounter with RESTful approaches to APIs can be attributed to the <a href=\"/radar/techniques/anemic-rest\">anemic REST</a> antipattern, some use cases warrant exploration of other approaches. In particular, organizations that have to support a long tail of client applications (and thus a likely proliferation of API versions even if they employ <a href=\"/radar/techniques/consumer-driven-contract-testing\">consumer-driven contracts</a>)—and have a large portion of their APIs supporting the endless-list style of activity feeds—may hit some limits in RESTful architectures. These can sometimes be mitigated by employing the <strong>client-directed query</strong> approach to client-server interaction. We see this approach being successfully used in both <a href=\"/radar/languages-and-frameworks/graphql\">GraphQL</a> and <a href=\"https://github.com/Netflix/falcor\">Falcor</a>, where clients have more control over both the contents and the granularity of the data returned to them. This does put more responsibility onto the service layer and can still lead to tight coupling to the underlying data model, but the benefits may be worth exploring if well-modeled RESTful APIs aren’t working for you."
},
{
"name": "Container security scanning",
"description": "The container revolution instigated by <a href=\"/radar/platforms/docker\">Docker</a> has massively reduced the friction in moving applications between environments but at the same time has blown a rather large hole in the traditional controls over what can go to production. The technique of <strong>container security scanning</strong> is a necessary response to this threat vector. Docker now provides its own <a href=\"https://blog.docker.com/2016/05/docker-security-scanning/\">security scanning tools</a>, as does <a href=\"https://coreos.com/blog/vulnerability-analysis-for-containers/\">CoreOS</a>, and we’ve also had success with the <a href=\"https://benchmarks.cisecurity.org/\">CIS Security Benchmarks</a>. Whichever approach you take, we believe the topic of automated container security validation is of high value and a necessary part of PaaS thinking. "
},
{
"name": "Content Security Policies",
"description": "We are finding <a href=\"https://en.wikipedia.org/wiki/Content_Security_Policy\"><strong>Content Security Policies</strong></a> to be a helpful addition to our security toolkit when dealing with websites that pull assets from mixed contexts. The policy defines a set of rules about where assets can come from (and whether to allow inline script tags). The browser then refuses to load or execute JavaScript, CSS or images that violate those rules. When used in conjunction with good practices, such as output encoding, it provides good mitigation for XSS attacks. Interestingly, the optional endpoint for posting JSON reports of violations is how Twitter discovered that ISPs were injecting HTML or JavaScript into their pages."
},
{
"name": "Differential privacy",
"description": "It has long been known that \"anonymized\" bulk data sets can reveal information about individuals, especially when multiple data sets are cross-referenced together. With <a href=\"https://www.washingtonpost.com/news/the-switch/wp/2016/05/13/new-government-data-shows-a-staggering-number-of-americans-have-stopped-basic-online-activities/\">increasing concern over personal privacy</a>, some companies—including <a href=\"https://www.wired.com/2016/06/apples-differential-privacy-collecting-data/\">Apple</a> and <a href=\"http://research.google.com/pubs/pub42852.html\">Google</a>—are turning to <strong>differential privacy</strong> techniques in order to improve individual privacy while retaining the ability to perform useful analytics on large numbers of users. Differential privacy is a cryptographic technique that attempts to maximize the accuracy of statistical queries from a database while minimizing the chances of identifying its records. These results can be achieved by introducing a low amount of \"noise\" to the data, but it’s important to note that this is an ongoing research area. Apple has announced plans to incorporate differential privacy into its products—and we wholeheartedly applaud its commitment to customers' privacy—but the usual Apple secrecy has left some security experts <a href=\"https://blog.cryptographyengineering.com/2016/06/15/what-is-differential-privacy/\">scratching their heads</a>. We continue to recommend <a href=\"http://martinfowler.com/bliki/Datensparsamkeit.html\">Datensparsamkeit</a> as an alternative approach: simply storing the minimum data you actually need will achieve better privacy results in most cases."
},
{
"name": "Micro frontends",
"description": "We've seen significant benefit from introducing <a href=\"/radar/techniques/microservices\">microservice architectures</a>, which have allowed teams to scale delivery of independently deployed and maintained services. However, teams have often struggled to avoid the creation of front-end monoliths—large and sprawling browser applications that are as difficult to maintain and evolve as the monolithic server-side applications we've abandoned. We're seeing an approach emerge that our teams call <strong>micro frontends</strong>. In this approach, a web application is broken up by its pages and features, with each feature being owned end-to-end by a single team. Multiple techniques exist to bring the application features—some old and some new—together as a cohesive user experience, but the goal remains to allow each feature to be developed, tested and deployed independently from others. The <a href=\"/radar/techniques/bff-backend-for-frontends\">BFF - backend for frontends</a> approach works well here, with each team developing a BFF to support its set of application features."
},
{
"name": "OWASP ASVS",
"description": "As more development teams incorporate security earlier in the development life cycle, figuring out requirements to limit security risks can seem like a daunting task. Few people have the extensive technical knowledge needed to identify all the risks that an application might face, and teams might struggle just trying to decide where to begin. Relying on frameworks such as OWASP's <a href=\"https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project\"><strong>ASVS</strong></a> (Application Security Verification Standard) can help make this easier. Although somewhat lengthy, it contains a thorough list of requirements categorized by functions such as authentication, access control, and error handling and logging, which can be reviewed as needed. It is also helpful as a resource for testers when it comes time to verify software."
},
{
"name": "Unikernels",
"description": "With the continued rise to domination of the container model led by Docker adoption, we think it's worth calling attention to the continued rapid development in the <strong>Unikernel</strong> space. Unikernels are single-purpose library operating systems that can be compiled down from high-level languages to run directly on the hypervisors used by commodity cloud platforms. They promise a number of advantages over containers, not least their superfast startup time and very small attack surface area. Many are still at the research-project phase—<a href=\"http://research.microsoft.com/en-us/projects/drawbridge/\">Drawbridge</a> from Microsoft Research, <a href=\"https://mirage.io/\">MirageOS</a> and <a href=\"http://galois.com/project/halvm/\">HaLVM</a> amongst others—but we think the ideas are very interesting and combine nicely with the technique of <a href=\"/radar/techniques/serverless-architecture\">serverless architecture</a>. "
},
{
"name": "VR beyond gaming",
"description": "The idea of virtual reality has been around for more than 50 years, and with successive improvements of computing technology many ideas have been hyped and explored. We believe that we're reaching a tipping point now. Modern graphics cards provide sufficient compute power to render detailed, realistic scenes in high resolutions, and at the same time at least two consumer-oriented VR headsets (the <a href=\"http://www.htcvive.com/uk/\">HTC Vive</a> and Facebook's <a href=\"https://www.oculus.com/en-us/\">Oculus Rift</a>) are coming to market. These headsets are affordable, they have high-resolution displays, and they eliminate perceivable motion-tracking lag, which was causing issues such as headaches and nausea before. The headsets are mainly targeted at enthusiast video gaming, but we are convinced that they will open many possibilities for <strong>VR beyond gaming</strong>, particularly as the low-fi approaches, such as <a href=\"https://www.google.co.uk/get/cardboard/get-cardboard/\">Google Cardboard</a>, are driving greater awareness."
},
{
"name": "A single CI instance for all teams",
"description": "There might be the impression that it's easier to manage a <strong>single CI (Continuous Integration) instance for all teams</strong> because it gives them a single configuration and monitoring point. But a bloated instance that is shared by every team in an organization can cause a lot of damage. We have found that problems like build timeouts, configuration conflicts and gigantic build queues appear more frequently. Having this single point of failure can interrupt the work of many teams. Carefully consider the trade-off between these pitfalls and having a single point of configuration. In organizations with multiple teams, we recommend having CI instances distributed by teams, with enterprise decisions based not on the single CI installation but on defining guidelines about the instances' selection and configuration."
},
{
"name": "Anemic REST",
"description": ""
},
{
"name": "Big Data envy",
"description": "We continue to see organizations chasing \"cool\" technologies, taking on unnecessary complexity and risk when a simpler choice would be better. One particular theme is using distributed, Big Data systems for relatively small data sets. This behavior prompts us to put <strong>Big Data envy</strong> on hold once more, with some additional data points from our recent experience. The <a href=\"http://cassandra.apache.org/\">Apache Cassandra</a> database promises massive scalability on commodity hardware, but we have seen teams overwhelmed by its architectural and operational complexity. Unless you have data volumes that require a 100+ node cluster, we recommend against using Cassandra. The operational team you’ll need to keep the thing running just isn’t worth it. While creating this edition of the Radar, we discussed several new database technologies, many offering \"10x\" performance improvements over existing systems. We’re always skeptical until new technology—especially something as critical as a database—has been properly proven. <a href=\"/radar/tools/jepsen\">Jepsen</a> provides <a href=\"http://jepsen.io/analyses.html\">analysis</a> of database performance under difficult conditions and has found <a href=\"https://aphyr.com/posts/283-call-me-maybe-redis\">numerous</a> <a href=\"https://aphyr.com/posts/284-call-me-maybe-mongodb\">bugs</a> in various NoSQL databases. We recommend maintaining a healthy dose of skepticism and keeping an eye on sites such as Jepsen when you evaluate database tech."
},
{
"name": "Cloud lift and shift",
"description": "As more organizations are choosing to deploy applications in the cloud, we're regularly finding IT groups that are wastefully trying to replicate their existing data center management and security approaches in&nbsp;the cloud. This often comes in the form of firewalls, load balancers, network proxies, access control, security appliances and services that are extended into the cloud with minimal rethinking. We've seen organizations build their own orchestration APIs in front of the cloud providers to constrain the services that can be utilized by teams. In most cases these layers serve only to cripple the capability, taking away most of the intended benefits of moving to the cloud. In this edition of the Radar, we've chosen to rehighlight <strong>cloud lift and shift</strong> as a technique to avoid. Organizations should instead look more deeply at the intent of their existing security and operational controls, and look for alternative controls that work in the cloud without creating unnecessary constraints. Many of those controls will already exist for mature cloud providers, and teams that adopt the cloud can use native APIs for self-serve provisioning and operations."
}
]
},
{
"name": "trial",
"blips": [
{
"name": "Consumer-driven contract testing",
"description": "We’ve decided to bring <strong>consumer-driven contract testing</strong> back from the archive for this edition even though we had allowed it to fade in the past. The concept isn’t new, but with the mainstream acceptance of microservices, we need to remind people that <a href=\"http://www.martinfowler.com/articles/consumerDrivenContracts.html\">consumer-driven contracts</a> are an essential part of a mature <a href=\"http://martinfowler.com/articles/microservice-testing/\">microservice testing</a> portfolio, enabling independent service deployments. But in addition, we want to point out that consumer-driven contract testing is a technique and an attitude that requires no special tool to implement. We love frameworks like <a href=\"https://github.com/realestate-com-au/pact\">Pact</a> because they make proper contract tests easier to implement in certain contexts. But we have noticed a tendency for teams to focus on the framework rather than on the general practice. Writing Pact tests is not a guarantee that you are creating consumer-driven contracts; likewise, in many situations you should be creating good consumer-driven contracts even where no pre-built testing tool exists."
},
{
"name": "Pipelines as code",
"description": "Teams are pushing for automation across their environments, including their development infrastructure. <strong>Pipelines as code</strong> is defining the deployment pipeline through code instead of configuring a running CI/CD tool. <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://readme.drone.io/usage/overview/\">Drone</a>, <a href=\"/radar/tools/gocd\">GoCD</a> and <a href=\"/radar/tools/concourse-ci\">Concourse</a> are examples that allow usage of this technique. Also, configuration automation tools for CI/CD systems like <a href=\"https://github.com/SpringerSBM/gomatic\">GoMatic</a> can be used to treat the deployment pipeline as code—versioned and tested."
},
{
"name": "Threat Modeling",
"description": "With the number of high-profile security breaches in the past months, software development teams no longer need convincing that they must place an emphasis on writing secure software and dealing with their users' data in a responsible way. The teams face a steep learning curve, though, and the vast number of potential threats—ranging from organized crime and government spying to teenagers who attack systems \"for the lulz\"—can be overwhelming. <a href=\"https://www.owasp.org/index.php/Category:Threat_Modeling\"><strong>Threat Modeling</strong></a> provides a set of techniques that help you identify and classify potential threats early in the development process. It is important to understand that it is only part of a strategy to stay ahead of threats. When used in conjunction with techniques such as establishing cross-functional security requirements to address common risks in the technologies a project uses and using automated security scanners, threat modeling can be a powerful asset."
},
{
"name": "APIs as a product",
"description": "Businesses have wholeheartedly embraced APIs as a way to expose business capabilities to both external and internal developers. APIs promise the ability to experiment quickly with new business ideas by recombining core capabilities. But what differentiates an API from an ordinary enterprise integration service? One difference lies in treating <strong>APIs as a product</strong>, even when the consumer is an internal system. Teams that build APIs should understand the needs of their customers and make the product compelling to them. Products are also improved, maintained and supported over the long term. They should have an owner who advocates for the customer and strives for continual improvement. Products are actively maintained and supported, easy to find and easy to use. In our experience, a product orientation is the missing ingredient that makes the difference between ordinary enterprise integration and an agile business built on a platform of APIs."
},
{
"name": "Bug bounties",
"description": "The use of <strong>bug bounties</strong> continues to grow in popularity for many organizations, including enterprises and notable government bodies. A bug-bounty program encourages participants to identify potentially damaging vulnerabilities in return for reward or recognition. Companies like <a href=\"https://hackerone.com/\">HackerOne</a> and <a href=\"https://bugcrowd.com/\">Bugcrowd</a> offer services to help organizations manage this process more easily, and we're seeing these services gather adoption."
},
{
"name": "Data Lake",
"description": "A <strong><a href=\"http://martinfowler.com/bliki/DataLake.html\">Data Lake</a></strong> is an immutable data store of largely unprocessed \"raw\" data, acting as a source for data analytics. While the technique can clearly be misused, we have used it successfully at clients, hence motivating its move to trial. We continue to recommend other approaches for operational collaborations, limiting the use of the data lake to reporting, analytics and feeding data into data marts."
},
{
"name": "Hosting PII data in the EU",
"description": "In a number of countries, we see government agencies seeking broad access to private, personally identifiable information (PII). The increased use of public cloud solutions makes it more difficult for organizations to protect the data entrusted to them by their users while also respecting all relevant laws. The European Union has some of the most progressive privacy laws, and all the major cloud providers—Amazon, Google and Microsoft—offer multiple data centers and regions within the European Union. Therefore, we recommend that companies, especially those with a global user base, assess the feasibility of a safe haven for their users' data by <strong>hosting PII data in the EU</strong>. Since we wrote about this technique in the last Radar, we have rolled out a new internal system that handles sensitive information relating to all our employees, and we have chosen to host it in a data center located in the European Union."
},
{
"name": "Lightweight Architecture Decision Records",
"description": "Although much documentation can be replaced with highly readable code and tests, in a world of <a href=\"/radar/techniques/evolutionary-architecture\">evolutionary architecture</a> it's important to record certain design decisions for the benefit of future team members and for external oversight. <strong>Lightweight Architecture Decision Records</strong> is <a href=\"http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions\">a technique</a> for capturing important architectural decisions along with their context and consequences. Although these items are often stored in a wiki or collaboration tool, we generally prefer <a href=\"https://github.com/npryce/adr-tools\">storing them in source control</a> with simple markup."
},
{
"name": "Reactive architectures",
"description": "We see continued adoption and success of <strong>reactive architectures</strong>, with reactive language extensions and reactive frameworks being very popular (we added several such blips in this edition of the Radar). User interfaces, in particular, benefit greatly from a reactive style of programming. Our caveats last time still hold true: Architectures based on asynchronous message passing introduce complexity and make the overall system harder to understand—it's no longer possible to simply read the program code and understand what the system does. We recommend assessing the performance and scalability needs of your system before committing to this architectural style."
},
{
"name": "Serverless architecture",
"description": "<a href=\"http://www.martinfowler.com/articles/serverless.html\"><strong>Serverless architecture</strong></a> is an approach that replaces long-running virtual machines with ephemeral compute power that comes into existence on request and disappears immediately after use. Since the last Radar, we have had several teams put applications into production using a \"serverless\" style. Our teams like the approach, it’s working well for them and we consider it a valid architectural choice. Note that serverless doesn’t have to be an all-or-nothing approach: some of our teams have deployed a new chunk of their systems using serverless while sticking to a traditional architectural approach for other pieces."
},
{
"name": "Client-directed query",
"description": "Although many problems that people encounter with RESTful approaches to APIs can be attributed to the <a href=\"/radar/techniques/anemic-rest\">anemic REST</a> antipattern, some use cases warrant exploration of other approaches. In particular, organizations that have to support a long tail of client applications (and thus a likely proliferation of API versions even if they employ <a href=\"/radar/techniques/consumer-driven-contract-testing\">consumer-driven contracts</a>)—and have a large portion of their APIs supporting the endless-list style of activity feeds—may hit some limits in RESTful architectures. These can sometimes be mitigated by employing the <strong>client-directed query</strong> approach to client-server interaction. We see this approach being successfully used in both <a href=\"/radar/languages-and-frameworks/graphql\">GraphQL</a> and <a href=\"https://github.com/Netflix/falcor\">Falcor</a>, where clients have more control over both the contents and the granularity of the data returned to them. This does put more responsibility onto the service layer and can still lead to tight coupling to the underlying data model, but the benefits may be worth exploring if well-modeled RESTful APIs aren’t working for you."
},
{
"name": "Container security scanning",
"description": "The container revolution instigated by <a href=\"/radar/platforms/docker\">Docker</a> has massively reduced the friction in moving applications between environments but at the same time has blown a rather large hole in the traditional controls over what can go to production. The technique of <strong>container security scanning</strong> is a necessary response to this threat vector. Docker now provides its own <a href=\"https://blog.docker.com/2016/05/docker-security-scanning/\">security scanning tools</a>, as does <a href=\"https://coreos.com/blog/vulnerability-analysis-for-containers/\">CoreOS</a>, and we’ve also had success with the <a href=\"https://benchmarks.cisecurity.org/\">CIS Security Benchmarks</a>. Whichever approach you take, we believe the topic of automated container security validation is of high value and a necessary part of PaaS thinking. "
},
{
"name": "Content Security Policies",
"description": "We are finding <a href=\"https://en.wikipedia.org/wiki/Content_Security_Policy\"><strong>Content Security Policies</strong></a> to be a helpful addition to our security toolkit when dealing with websites that pull assets from mixed contexts. The policy defines a set of rules about where assets can come from (and whether to allow inline script tags). The browser then refuses to load or execute JavaScript, CSS or images that violate those rules. When used in conjunction with good practices, such as output encoding, it provides good mitigation for XSS attacks. Interestingly, the optional endpoint for posting JSON reports of violations is how Twitter discovered that ISPs were injecting HTML or JavaScript into their pages."
},
{
"name": "Differential privacy",
"description": "It has long been known that \"anonymized\" bulk data sets can reveal information about individuals, especially when multiple data sets are cross-referenced together. With <a href=\"https://www.washingtonpost.com/news/the-switch/wp/2016/05/13/new-government-data-shows-a-staggering-number-of-americans-have-stopped-basic-online-activities/\">increasing concern over personal privacy</a>, some companies—including <a href=\"https://www.wired.com/2016/06/apples-differential-privacy-collecting-data/\">Apple</a> and <a href=\"http://research.google.com/pubs/pub42852.html\">Google</a>—are turning to <strong>differential privacy</strong> techniques in order to improve individual privacy while retaining the ability to perform useful analytics on large numbers of users. Differential privacy is a cryptographic technique that attempts to maximize the accuracy of statistical queries from a database while minimizing the chances of identifying its records. These results can be achieved by introducing a low amount of \"noise\" to the data, but it’s important to note that this is an ongoing research area. Apple has announced plans to incorporate differential privacy into its products—and we wholeheartedly applaud its commitment to customers' privacy—but the usual Apple secrecy has left some security experts <a href=\"https://blog.cryptographyengineering.com/2016/06/15/what-is-differential-privacy/\">scratching their heads</a>. We continue to recommend <a href=\"http://martinfowler.com/bliki/Datensparsamkeit.html\">Datensparsamkeit</a> as an alternative approach: simply storing the minimum data you actually need will achieve better privacy results in most cases."
},
{
"name": "Micro frontends",
"description": "We've seen significant benefit from introducing <a href=\"/radar/techniques/microservices\">microservice architectures</a>, which have allowed teams to scale delivery of independently deployed and maintained services. However, teams have often struggled to avoid the creation of front-end monoliths—large and sprawling browser applications that are as difficult to maintain and evolve as the monolithic server-side applications we've abandoned. We're seeing an approach emerge that our teams call <strong>micro frontends</strong>. In this approach, a web application is broken up by its pages and features, with each feature being owned end-to-end by a single team. Multiple techniques exist to bring the application features—some old and some new—together as a cohesive user experience, but the goal remains to allow each feature to be developed, tested and deployed independently from others. The <a href=\"/radar/techniques/bff-backend-for-frontends\">BFF - backend for frontends</a> approach works well here, with each team developing a BFF to support its set of application features."
},
{
"name": "OWASP ASVS",
"description": "As more development teams incorporate security earlier in the development life cycle, figuring out requirements to limit security risks can seem like a daunting task. Few people have the extensive technical knowledge needed to identify all the risks that an application might face, and teams might struggle just trying to decide where to begin. Relying on frameworks such as OWASP's <a href=\"https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project\"><strong>ASVS</strong></a> (Application Security Verification Standard) can help make this easier. Although somewhat lengthy, it contains a thorough list of requirements categorized by functions such as authentication, access control, and error handling and logging, which can be reviewed as needed. It is also helpful as a resource for testers when it comes time to verify software."
},
{
"name": "Unikernels",
"description": "With the continued rise to domination of the container model led by Docker adoption, we think it's worth calling attention to the continued rapid development in the <strong>Unikernel</strong> space. Unikernels are single-purpose library operating systems that can be compiled down from high-level languages to run directly on the hypervisors used by commodity cloud platforms. They promise a number of advantages over containers, not least their superfast startup time and very small attack surface area. Many are still at the research-project phase—<a href=\"http://research.microsoft.com/en-us/projects/drawbridge/\">Drawbridge</a> from Microsoft Research, <a href=\"https://mirage.io/\">MirageOS</a> and <a href=\"http://galois.com/project/halvm/\">HaLVM</a> amongst others—but we think the ideas are very interesting and combine nicely with the technique of <a href=\"/radar/techniques/serverless-architecture\">serverless architecture</a>. "
},
{
"name": "VR beyond gaming",
"description": "The idea of virtual reality has been around for more than 50 years, and with successive improvements of computing technology many ideas have been hyped and explored. We believe that we're reaching a tipping point now. Modern graphics cards provide sufficient compute power to render detailed, realistic scenes in high resolutions, and at the same time at least two consumer-oriented VR headsets (the <a href=\"http://www.htcvive.com/uk/\">HTC Vive</a> and Facebook's <a href=\"https://www.oculus.com/en-us/\">Oculus Rift</a>) are coming to market. These headsets are affordable, they have high-resolution displays, and they eliminate perceivable motion-tracking lag, which was causing issues such as headaches and nausea before. The headsets are mainly targeted at enthusiast video gaming, but we are convinced that they will open many possibilities for <strong>VR beyond gaming</strong>, particularly as the low-fi approaches, such as <a href=\"https://www.google.co.uk/get/cardboard/get-cardboard/\">Google Cardboard</a>, are driving greater awareness."
},
{
"name": "A single CI instance for all teams",
"description": "There might be the impression that it's easier to manage a <strong>single CI (Continuous Integration) instance for all teams</strong> because it gives them a single configuration and monitoring point. But a bloated instance that is shared by every team in an organization can cause a lot of damage. We have found that problems like build timeouts, configuration conflicts and gigantic build queues appear more frequently. Having this single point of failure can interrupt the work of many teams. Carefully consider the trade-off between these pitfalls and having a single point of configuration. In organizations with multiple teams, we recommend having CI instances distributed by teams, with enterprise decisions based not on the single CI installation but on defining guidelines about the instances' selection and configuration."
},
{
"name": "Anemic REST",
"description": ""
},
{
"name": "Big Data envy",
"description": "We continue to see organizations chasing \"cool\" technologies, taking on unnecessary complexity and risk when a simpler choice would be better. One particular theme is using distributed, Big Data systems for relatively small data sets. This behavior prompts us to put <strong>Big Data envy</strong> on hold once more, with some additional data points from our recent experience. The <a href=\"http://cassandra.apache.org/\">Apache Cassandra</a> database promises massive scalability on commodity hardware, but we have seen teams overwhelmed by its architectural and operational complexity. Unless you have data volumes that require a 100+ node cluster, we recommend against using Cassandra. The operational team you’ll need to keep the thing running just isn’t worth it. While creating this edition of the Radar, we discussed several new database technologies, many offering \"10x\" performance improvements over existing systems. We’re always skeptical until new technology—especially something as critical as a database—has been properly proven. <a href=\"/radar/tools/jepsen\">Jepsen</a> provides <a href=\"http://jepsen.io/analyses.html\">analysis</a> of database performance under difficult conditions and has found <a href=\"https://aphyr.com/posts/283-call-me-maybe-redis\">numerous</a> <a href=\"https://aphyr.com/posts/284-call-me-maybe-mongodb\">bugs</a> in various NoSQL databases. We recommend maintaining a healthy dose of skepticism and keeping an eye on sites such as Jepsen when you evaluate database tech."
},
{
"name": "Cloud lift and shift",
"description": "As more organizations are choosing to deploy applications in the cloud, we're regularly finding IT groups that are wastefully trying to replicate their existing data center management and security approaches in&nbsp;the cloud. This often comes in the form of firewalls, load balancers, network proxies, access control, security appliances and services that are extended into the cloud with minimal rethinking. We've seen organizations build their own orchestration APIs in front of the cloud providers to constrain the services that can be utilized by teams. In most cases these layers serve only to cripple the capability, taking away most of the intended benefits of moving to the cloud. In this edition of the Radar, we've chosen to rehighlight <strong>cloud lift and shift</strong> as a technique to avoid. Organizations should instead look more deeply at the intent of their existing security and operational controls, and look for alternative controls that work in the cloud without creating unnecessary constraints. Many of those controls will already exist for mature cloud providers, and teams that adopt the cloud can use native APIs for self-serve provisioning and operations."
}
]
},
{
"name": "assess",
"blips": [
{
"name": "Consumer-driven contract testing",
"description": "We’ve decided to bring <strong>consumer-driven contract testing</strong> back from the archive for this edition even though we had allowed it to fade in the past. The concept isn’t new, but with the mainstream acceptance of microservices, we need to remind people that <a href=\"http://www.martinfowler.com/articles/consumerDrivenContracts.html\">consumer-driven contracts</a> are an essential part of a mature <a href=\"http://martinfowler.com/articles/microservice-testing/\">microservice testing</a> portfolio, enabling independent service deployments. But in addition, we want to point out that consumer-driven contract testing is a technique and an attitude that requires no special tool to implement. We love frameworks like <a href=\"https://github.com/realestate-com-au/pact\">Pact</a> because they make proper contract tests easier to implement in certain contexts. But we have noticed a tendency for teams to focus on the framework rather than on the general practice. Writing Pact tests is not a guarantee that you are creating consumer-driven contracts; likewise, in many situations you should be creating good consumer-driven contracts even where no pre-built testing tool exists."
},
{
"name": "Pipelines as code",
"description": "Teams are pushing for automation across their environments, including their development infrastructure. <strong>Pipelines as code</strong> is defining the deployment pipeline through code instead of configuring a running CI/CD tool. <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://readme.drone.io/usage/overview/\">Drone</a>, <a href=\"/radar/tools/gocd\">GoCD</a> and <a href=\"/radar/tools/concourse-ci\">Concourse</a> are examples that allow usage of this technique. Also, configuration automation tools for CI/CD systems like <a href=\"https://github.com/SpringerSBM/gomatic\">GoMatic</a> can be used to treat the deployment pipeline as code—versioned and tested."
},
{
"name": "Threat Modeling",
"description": "With the number of high-profile security breaches in the past months, software development teams no longer need convincing that they must place an emphasis on writing secure software and dealing with their users' data in a responsible way. The teams face a steep learning curve, though, and the vast number of potential threats—ranging from organized crime and government spying to teenagers who attack systems \"for the lulz\"—can be overwhelming. <a href=\"https://www.owasp.org/index.php/Category:Threat_Modeling\"><strong>Threat Modeling</strong></a> provides a set of techniques that help you identify and classify potential threats early in the development process. It is important to understand that it is only part of a strategy to stay ahead of threats. When used in conjunction with techniques such as establishing cross-functional security requirements to address common risks in the technologies a project uses and using automated security scanners, threat modeling can be a powerful asset."
},
{
"name": "APIs as a product",
"description": "Businesses have wholeheartedly embraced APIs as a way to expose business capabilities to both external and internal developers. APIs promise the ability to experiment quickly with new business ideas by recombining core capabilities. But what differentiates an API from an ordinary enterprise integration service? One difference lies in treating <strong>APIs as a product</strong>, even when the consumer is an internal system. Teams that build APIs should understand the needs of their customers and make the product compelling to them. Products are also improved, maintained and supported over the long term. They should have an owner who advocates for the customer and strives for continual improvement. Products are actively maintained and supported, easy to find and easy to use. In our experience, a product orientation is the missing ingredient that makes the difference between ordinary enterprise integration and an agile business built on a platform of APIs."
},
{
"name": "Bug bounties",
"description": "The use of <strong>bug bounties</strong> continues to grow in popularity for many organizations, including enterprises and notable government bodies. A bug-bounty program encourages participants to identify potentially damaging vulnerabilities in return for reward or recognition. Companies like <a href=\"https://hackerone.com/\">HackerOne</a> and <a href=\"https://bugcrowd.com/\">Bugcrowd</a> offer services to help organizations manage this process more easily, and we're seeing these services gather adoption."
},
{
"name": "Data Lake",
"description": "A <strong><a href=\"http://martinfowler.com/bliki/DataLake.html\">Data Lake</a></strong> is an immutable data store of largely unprocessed \"raw\" data, acting as a source for data analytics. While the technique can clearly be misused, we have used it successfully at clients, hence motivating its move to trial. We continue to recommend other approaches for operational collaborations, limiting the use of the data lake to reporting, analytics and feeding data into data marts."
},
{
"name": "Hosting PII data in the EU",
"description": "In a number of countries, we see government agencies seeking broad access to private, personally identifiable information (PII). The increased use of public cloud solutions makes it more difficult for organizations to protect the data entrusted to them by their users while also respecting all relevant laws. The European Union has some of the most progressive privacy laws, and all the major cloud providers—Amazon, Google and Microsoft—offer multiple data centers and regions within the European Union. Therefore, we recommend that companies, especially those with a global user base, assess the feasibility of a safe haven for their users' data by <strong>hosting PII data in the EU</strong>. Since we wrote about this technique in the last Radar, we have rolled out a new internal system that handles sensitive information relating to all our employees, and we have chosen to host it in a data center located in the European Union."
},
{
"name": "Lightweight Architecture Decision Records",
"description": "Although much documentation can be replaced with highly readable code and tests, in a world of <a href=\"/radar/techniques/evolutionary-architecture\">evolutionary architecture</a> it's important to record certain design decisions for the benefit of future team members and for external oversight. <strong>Lightweight Architecture Decision Records</strong> is <a href=\"http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions\">a technique</a> for capturing important architectural decisions along with their context and consequences. Although these items are often stored in a wiki or collaboration tool, we generally prefer <a href=\"https://github.com/npryce/adr-tools\">storing them in source control</a> with simple markup."
},
{
"name": "Reactive architectures",
"description": "We see continued adoption and success of <strong>reactive architectures</strong>, with reactive language extensions and reactive frameworks being very popular (we added several such blips in this edition of the Radar). User interfaces, in particular, benefit greatly from a reactive style of programming. Our caveats last time still hold true: Architectures based on asynchronous message passing introduce complexity and make the overall system harder to understand—it's no longer possible to simply read the program code and understand what the system does. We recommend assessing the performance and scalability needs of your system before committing to this architectural style."
},
{
"name": "Serverless architecture",
"description": "<a href=\"http://www.martinfowler.com/articles/serverless.html\"><strong>Serverless architecture</strong></a> is an approach that replaces long-running virtual machines with ephemeral compute power that comes into existence on request and disappears immediately after use. Since the last Radar, we have had several teams put applications into production using a \"serverless\" style. Our teams like the approach, it’s working well for them and we consider it a valid architectural choice. Note that serverless doesn’t have to be an all-or-nothing approach: some of our teams have deployed a new chunk of their systems using serverless while sticking to a traditional architectural approach for other pieces."
},
{
"name": "Client-directed query",
"description": "Although many problems that people encounter with RESTful approaches to APIs can be attributed to the <a href=\"/radar/techniques/anemic-rest\">anemic REST</a> antipattern, some use cases warrant exploration of other approaches. In particular, organizations that have to support a long tail of client applications (and thus a likely proliferation of API versions even if they employ <a href=\"/radar/techniques/consumer-driven-contract-testing\">consumer-driven contracts</a>)—and have a large portion of their APIs supporting the endless-list style of activity feeds—may hit some limits in RESTful architectures. These can sometimes be mitigated by employing the <strong>client-directed query</strong> approach to client-server interaction. We see this approach being successfully used in both <a href=\"/radar/languages-and-frameworks/graphql\">GraphQL</a> and <a href=\"https://github.com/Netflix/falcor\">Falcor</a>, where clients have more control over both the contents and the granularity of the data returned to them. This does put more responsibility onto the service layer and can still lead to tight coupling to the underlying data model, but the benefits may be worth exploring if well-modeled RESTful APIs aren’t working for you."
},
{
"name": "Container security scanning",
"description": "The container revolution instigated by <a href=\"/radar/platforms/docker\">Docker</a> has massively reduced the friction in moving applications between environments but at the same time has blown a rather large hole in the traditional controls over what can go to production. The technique of <strong>container security scanning</strong> is a necessary response to this threat vector. Docker now provides its own <a href=\"https://blog.docker.com/2016/05/docker-security-scanning/\">security scanning tools</a>, as does <a href=\"https://coreos.com/blog/vulnerability-analysis-for-containers/\">CoreOS</a>, and we’ve also had success with the <a href=\"https://benchmarks.cisecurity.org/\">CIS Security Benchmarks</a>. Whichever approach you take, we believe the topic of automated container security validation is of high value and a necessary part of PaaS thinking. "
},
{
"name": "Content Security Policies",
"description": "We are finding <a href=\"https://en.wikipedia.org/wiki/Content_Security_Policy\"><strong>Content Security Policies</strong></a> to be a helpful addition to our security toolkit when dealing with websites that pull assets from mixed contexts. The policy defines a set of rules about where assets can come from (and whether to allow inline script tags). The browser then refuses to load or execute JavaScript, CSS or images that violate those rules. When used in conjunction with good practices, such as output encoding, it provides good mitigation for XSS attacks. Interestingly, the optional endpoint for posting JSON reports of violations is how Twitter discovered that ISPs were injecting HTML or JavaScript into their pages."
},
{
"name": "Differential privacy",
"description": "It has long been known that \"anonymized\" bulk data sets can reveal information about individuals, especially when multiple data sets are cross-referenced together. With <a href=\"https://www.washingtonpost.com/news/the-switch/wp/2016/05/13/new-government-data-shows-a-staggering-number-of-americans-have-stopped-basic-online-activities/\">increasing concern over personal privacy</a>, some companies—including <a href=\"https://www.wired.com/2016/06/apples-differential-privacy-collecting-data/\">Apple</a> and <a href=\"http://research.google.com/pubs/pub42852.html\">Google</a>—are turning to <strong>differential privacy</strong> techniques in order to improve individual privacy while retaining the ability to perform useful analytics on large numbers of users. Differential privacy is a cryptographic technique that attempts to maximize the accuracy of statistical queries from a database while minimizing the chances of identifying its records. These results can be achieved by introducing a low amount of \"noise\" to the data, but it’s important to note that this is an ongoing research area. Apple has announced plans to incorporate differential privacy into its products—and we wholeheartedly applaud its commitment to customers' privacy—but the usual Apple secrecy has left some security experts <a href=\"https://blog.cryptographyengineering.com/2016/06/15/what-is-differential-privacy/\">scratching their heads</a>. We continue to recommend <a href=\"http://martinfowler.com/bliki/Datensparsamkeit.html\">Datensparsamkeit</a> as an alternative approach: simply storing the minimum data you actually need will achieve better privacy results in most cases."
},
{
"name": "Micro frontends",
"description": "We've seen significant benefit from introducing <a href=\"/radar/techniques/microservices\">microservice architectures</a>, which have allowed teams to scale delivery of independently deployed and maintained services. However, teams have often struggled to avoid the creation of front-end monoliths—large and sprawling browser applications that are as difficult to maintain and evolve as the monolithic server-side applications we've abandoned. We're seeing an approach emerge that our teams call <strong>micro frontends</strong>. In this approach, a web application is broken up by its pages and features, with each feature being owned end-to-end by a single team. Multiple techniques exist to bring the application features—some old and some new—together as a cohesive user experience, but the goal remains to allow each feature to be developed, tested and deployed independently from others. The <a href=\"/radar/techniques/bff-backend-for-frontends\">BFF - backend for frontends</a> approach works well here, with each team developing a BFF to support its set of application features."
},
{
"name": "OWASP ASVS",
"description": "As more development teams incorporate security earlier in the development life cycle, figuring out requirements to limit security risks can seem like a daunting task. Few people have the extensive technical knowledge needed to identify all the risks that an application might face, and teams might struggle just trying to decide where to begin. Relying on frameworks such as OWASP's <a href=\"https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project\"><strong>ASVS</strong></a> (Application Security Verification Standard) can help make this easier. Although somewhat lengthy, it contains a thorough list of requirements categorized by functions such as authentication, access control, and error handling and logging, which can be reviewed as needed. It is also helpful as a resource for testers when it comes time to verify software."
},
{
"name": "Unikernels",
"description": "With the continued rise to domination of the container model led by Docker adoption, we think it's worth calling attention to the continued rapid development in the <strong>Unikernel</strong> space. Unikernels are single-purpose library operating systems that can be compiled down from high-level languages to run directly on the hypervisors used by commodity cloud platforms. They promise a number of advantages over containers, not least their superfast startup time and very small attack surface area. Many are still at the research-project phase—<a href=\"http://research.microsoft.com/en-us/projects/drawbridge/\">Drawbridge</a> from Microsoft Research, <a href=\"https://mirage.io/\">MirageOS</a> and <a href=\"http://galois.com/project/halvm/\">HaLVM</a> amongst others—but we think the ideas are very interesting and combine nicely with the technique of <a href=\"/radar/techniques/serverless-architecture\">serverless architecture</a>. "
},
{
"name": "VR beyond gaming",
"description": "The idea of virtual reality has been around for more than 50 years, and with successive improvements of computing technology many ideas have been hyped and explored. We believe that we're reaching a tipping point now. Modern graphics cards provide sufficient compute power to render detailed, realistic scenes in high resolutions, and at the same time at least two consumer-oriented VR headsets (the <a href=\"http://www.htcvive.com/uk/\">HTC Vive</a> and Facebook's <a href=\"https://www.oculus.com/en-us/\">Oculus Rift</a>) are coming to market. These headsets are affordable, they have high-resolution displays, and they eliminate perceivable motion-tracking lag, which was causing issues such as headaches and nausea before. The headsets are mainly targeted at enthusiast video gaming, but we are convinced that they will open many possibilities for <strong>VR beyond gaming</strong>, particularly as the low-fi approaches, such as <a href=\"https://www.google.co.uk/get/cardboard/get-cardboard/\">Google Cardboard</a>, are driving greater awareness."
},
{
"name": "A single CI instance for all teams",
"description": "There might be the impression that it's easier to manage a <strong>single CI (Continuous Integration) instance for all teams</strong> because it gives them a single configuration and monitoring point. But a bloated instance that is shared by every team in an organization can cause a lot of damage. We have found that problems like build timeouts, configuration conflicts and gigantic build queues appear more frequently. Having this single point of failure can interrupt the work of many teams. Carefully consider the trade-off between these pitfalls and having a single point of configuration. In organizations with multiple teams, we recommend having CI instances distributed by teams, with enterprise decisions based not on the single CI installation but on defining guidelines about the instances' selection and configuration."
},
{
"name": "Anemic REST",
"description": ""
},
{
"name": "Big Data envy",
"description": "We continue to see organizations chasing \"cool\" technologies, taking on unnecessary complexity and risk when a simpler choice would be better. One particular theme is using distributed, Big Data systems for relatively small data sets. This behavior prompts us to put <strong>Big Data envy</strong> on hold once more, with some additional data points from our recent experience. The <a href=\"http://cassandra.apache.org/\">Apache Cassandra</a> database promises massive scalability on commodity hardware, but we have seen teams overwhelmed by its architectural and operational complexity. Unless you have data volumes that require a 100+ node cluster, we recommend against using Cassandra. The operational team you’ll need to keep the thing running just isn’t worth it. While creating this edition of the Radar, we discussed several new database technologies, many offering \"10x\" performance improvements over existing systems. We’re always skeptical until new technology—especially something as critical as a database—has been properly proven. <a href=\"/radar/tools/jepsen\">Jepsen</a> provides <a href=\"http://jepsen.io/analyses.html\">analysis</a> of database performance under difficult conditions and has found <a href=\"https://aphyr.com/posts/283-call-me-maybe-redis\">numerous</a> <a href=\"https://aphyr.com/posts/284-call-me-maybe-mongodb\">bugs</a> in various NoSQL databases. We recommend maintaining a healthy dose of skepticism and keeping an eye on sites such as Jepsen when you evaluate database tech."
},
{
"name": "Cloud lift and shift",
"description": "As more organizations are choosing to deploy applications in the cloud, we're regularly finding IT groups that are wastefully trying to replicate their existing data center management and security approaches in&nbsp;the cloud. This often comes in the form of firewalls, load balancers, network proxies, access control, security appliances and services that are extended into the cloud with minimal rethinking. We've seen organizations build their own orchestration APIs in front of the cloud providers to constrain the services that can be utilized by teams. In most cases these layers serve only to cripple the capability, taking away most of the intended benefits of moving to the cloud. In this edition of the Radar, we've chosen to rehighlight <strong>cloud lift and shift</strong> as a technique to avoid. Organizations should instead look more deeply at the intent of their existing security and operational controls, and look for alternative controls that work in the cloud without creating unnecessary constraints. Many of those controls will already exist for mature cloud providers, and teams that adopt the cloud can use native APIs for self-serve provisioning and operations."
}
]
},
{
"name": "hold",
"blips": [
{
"name": "Consumer-driven contract testing",
"description": "We’ve decided to bring <strong>consumer-driven contract testing</strong> back from the archive for this edition even though we had allowed it to fade in the past. The concept isn’t new, but with the mainstream acceptance of microservices, we need to remind people that <a href=\"http://www.martinfowler.com/articles/consumerDrivenContracts.html\">consumer-driven contracts</a> are an essential part of a mature <a href=\"http://martinfowler.com/articles/microservice-testing/\">microservice testing</a> portfolio, enabling independent service deployments. But in addition, we want to point out that consumer-driven contract testing is a technique and an attitude that requires no special tool to implement. We love frameworks like <a href=\"https://github.com/realestate-com-au/pact\">Pact</a> because they make proper contract tests easier to implement in certain contexts. But we have noticed a tendency for teams to focus on the framework rather than on the general practice. Writing Pact tests is not a guarantee that you are creating consumer-driven contracts; likewise, in many situations you should be creating good consumer-driven contracts even where no pre-built testing tool exists."
},
{
"name": "Pipelines as code",
"description": "Teams are pushing for automation across their environments, including their development infrastructure. <strong>Pipelines as code</strong> is defining the deployment pipeline through code instead of configuring a running CI/CD tool. <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://readme.drone.io/usage/overview/\">Drone</a>, <a href=\"/radar/tools/gocd\">GoCD</a> and <a href=\"/radar/tools/concourse-ci\">Concourse</a> are examples that allow usage of this technique. Also, configuration automation tools for CI/CD systems like <a href=\"https://github.com/SpringerSBM/gomatic\">GoMatic</a> can be used to treat the deployment pipeline as code—versioned and tested."
},
{
"name": "Threat Modeling",
"description": "With the number of high-profile security breaches in the past months, software development teams no longer need convincing that they must place an emphasis on writing secure software and dealing with their users' data in a responsible way. The teams face a steep learning curve, though, and the vast number of potential threats—ranging from organized crime and government spying to teenagers who attack systems \"for the lulz\"—can be overwhelming. <a href=\"https://www.owasp.org/index.php/Category:Threat_Modeling\"><strong>Threat Modeling</strong></a> provides a set of techniques that help you identify and classify potential threats early in the development process. It is important to understand that it is only part of a strategy to stay ahead of threats. When used in conjunction with techniques such as establishing cross-functional security requirements to address common risks in the technologies a project uses and using automated security scanners, threat modeling can be a powerful asset."
},
{
"name": "APIs as a product",
"description": "Businesses have wholeheartedly embraced APIs as a way to expose business capabilities to both external and internal developers. APIs promise the ability to experiment quickly with new business ideas by recombining core capabilities. But what differentiates an API from an ordinary enterprise integration service? One difference lies in treating <strong>APIs as a product</strong>, even when the consumer is an internal system. Teams that build APIs should understand the needs of their customers and make the product compelling to them. Products are also improved, maintained and supported over the long term. They should have an owner who advocates for the customer and strives for continual improvement. Products are actively maintained and supported, easy to find and easy to use. In our experience, a product orientation is the missing ingredient that makes the difference between ordinary enterprise integration and an agile business built on a platform of APIs."
},
{
"name": "Bug bounties",
"description": "The use of <strong>bug bounties</strong> continues to grow in popularity for many organizations, including enterprises and notable government bodies. A bug-bounty program encourages participants to identify potentially damaging vulnerabilities in return for reward or recognition. Companies like <a href=\"https://hackerone.com/\">HackerOne</a> and <a href=\"https://bugcrowd.com/\">Bugcrowd</a> offer services to help organizations manage this process more easily, and we're seeing these services gather adoption."
},
{
"name": "Data Lake",
"description": "A <strong><a href=\"http://martinfowler.com/bliki/DataLake.html\">Data Lake</a></strong> is an immutable data store of largely unprocessed \"raw\" data, acting as a source for data analytics. While the technique can clearly be misused, we have used it successfully at clients, hence motivating its move to trial. We continue to recommend other approaches for operational collaborations, limiting the use of the data lake to reporting, analytics and feeding data into data marts."
},
{
"name": "Hosting PII data in the EU",
"description": "In a number of countries, we see government agencies seeking broad access to private, personally identifiable information (PII). The increased use of public cloud solutions makes it more difficult for organizations to protect the data entrusted to them by their users while also respecting all relevant laws. The European Union has some of the most progressive privacy laws, and all the major cloud providers—Amazon, Google and Microsoft—offer multiple data centers and regions within the European Union. Therefore, we recommend that companies, especially those with a global user base, assess the feasibility of a safe haven for their users' data by <strong>hosting PII data in the EU</strong>. Since we wrote about this technique in the last Radar, we have rolled out a new internal system that handles sensitive information relating to all our employees, and we have chosen to host it in a data center located in the European Union."
},
{
"name": "Lightweight Architecture Decision Records",
"description": "Although much documentation can be replaced with highly readable code and tests, in a world of <a href=\"/radar/techniques/evolutionary-architecture\">evolutionary architecture</a> it's important to record certain design decisions for the benefit of future team members and for external oversight. <strong>Lightweight Architecture Decision Records</strong> is <a href=\"http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions\">a technique</a> for capturing important architectural decisions along with their context and consequences. Although these items are often stored in a wiki or collaboration tool, we generally prefer <a href=\"https://github.com/npryce/adr-tools\">storing them in source control</a> with simple markup."
},
{
"name": "Reactive architectures",
"description": "We see continued adoption and success of <strong>reactive architectures</strong>, with reactive language extensions and reactive frameworks being very popular (we added several such blips in this edition of the Radar). User interfaces, in particular, benefit greatly from a reactive style of programming. Our caveats last time still hold true: Architectures based on asynchronous message passing introduce complexity and make the overall system harder to understand—it's no longer possible to simply read the program code and understand what the system does. We recommend assessing the performance and scalability needs of your system before committing to this architectural style."
},
{
"name": "Serverless architecture",
"description": "<a href=\"http://www.martinfowler.com/articles/serverless.html\"><strong>Serverless architecture</strong></a> is an approach that replaces long-running virtual machines with ephemeral compute power that comes into existence on request and disappears immediately after use. Since the last Radar, we have had several teams put applications into production using a \"serverless\" style. Our teams like the approach, it’s working well for them and we consider it a valid architectural choice. Note that serverless doesn’t have to be an all-or-nothing approach: some of our teams have deployed a new chunk of their systems using serverless while sticking to a traditional architectural approach for other pieces."
},
{
"name": "Client-directed query",
"description": "Although many problems that people encounter with RESTful approaches to APIs can be attributed to the <a href=\"/radar/techniques/anemic-rest\">anemic REST</a> antipattern, some use cases warrant exploration of other approaches. In particular, organizations that have to support a long tail of client applications (and thus a likely proliferation of API versions even if they employ <a href=\"/radar/techniques/consumer-driven-contract-testing\">consumer-driven contracts</a>)—and have a large portion of their APIs supporting the endless-list style of activity feeds—may hit some limits in RESTful architectures. These can sometimes be mitigated by employing the <strong>client-directed query</strong> approach to client-server interaction. We see this approach being successfully used in both <a href=\"/radar/languages-and-frameworks/graphql\">GraphQL</a> and <a href=\"https://github.com/Netflix/falcor\">Falcor</a>, where clients have more control over both the contents and the granularity of the data returned to them. This does put more responsibility onto the service layer and can still lead to tight coupling to the underlying data model, but the benefits may be worth exploring if well-modeled RESTful APIs aren’t working for you."
},
{
"name": "Container security scanning",
"description": "The container revolution instigated by <a href=\"/radar/platforms/docker\">Docker</a> has massively reduced the friction in moving applications between environments but at the same time has blown a rather large hole in the traditional controls over what can go to production. The technique of <strong>container security scanning</strong> is a necessary response to this threat vector. Docker now provides its own <a href=\"https://blog.docker.com/2016/05/docker-security-scanning/\">security scanning tools</a>, as does <a href=\"https://coreos.com/blog/vulnerability-analysis-for-containers/\">CoreOS</a>, and we’ve also had success with the <a href=\"https://benchmarks.cisecurity.org/\">CIS Security Benchmarks</a>. Whichever approach you take, we believe the topic of automated container security validation is of high value and a necessary part of PaaS thinking. "
},
{
"name": "Content Security Policies",
"description": "We are finding <a href=\"https://en.wikipedia.org/wiki/Content_Security_Policy\"><strong>Content Security Policies</strong></a> to be a helpful addition to our security toolkit when dealing with websites that pull assets from mixed contexts. The policy defines a set of rules about where assets can come from (and whether to allow inline script tags). The browser then refuses to load or execute JavaScript, CSS or images that violate those rules. When used in conjunction with good practices, such as output encoding, it provides good mitigation for XSS attacks. Interestingly, the optional endpoint for posting JSON reports of violations is how Twitter discovered that ISPs were injecting HTML or JavaScript into their pages."
},
{
"name": "Differential privacy",
"description": "It has long been known that \"anonymized\" bulk data sets can reveal information about individuals, especially when multiple data sets are cross-referenced together. With <a href=\"https://www.washingtonpost.com/news/the-switch/wp/2016/05/13/new-government-data-shows-a-staggering-number-of-americans-have-stopped-basic-online-activities/\">increasing concern over personal privacy</a>, some companies—including <a href=\"https://www.wired.com/2016/06/apples-differential-privacy-collecting-data/\">Apple</a> and <a href=\"http://research.google.com/pubs/pub42852.html\">Google</a>—are turning to <strong>differential privacy</strong> techniques in order to improve individual privacy while retaining the ability to perform useful analytics on large numbers of users. Differential privacy is a cryptographic technique that attempts to maximize the accuracy of statistical queries from a database while minimizing the chances of identifying its records. These results can be achieved by introducing a low amount of \"noise\" to the data, but it’s important to note that this is an ongoing research area. Apple has announced plans to incorporate differential privacy into its products—and we wholeheartedly applaud its commitment to customers' privacy—but the usual Apple secrecy has left some security experts <a href=\"https://blog.cryptographyengineering.com/2016/06/15/what-is-differential-privacy/\">scratching their heads</a>. We continue to recommend <a href=\"http://martinfowler.com/bliki/Datensparsamkeit.html\">Datensparsamkeit</a> as an alternative approach: simply storing the minimum data you actually need will achieve better privacy results in most cases."
},
{
"name": "Micro frontends",
"description": "We've seen significant benefit from introducing <a href=\"/radar/techniques/microservices\">microservice architectures</a>, which have allowed teams to scale delivery of independently deployed and maintained services. However, teams have often struggled to avoid the creation of front-end monoliths—large and sprawling browser applications that are as difficult to maintain and evolve as the monolithic server-side applications we've abandoned. We're seeing an approach emerge that our teams call <strong>micro frontends</strong>. In this approach, a web application is broken up by its pages and features, with each feature being owned end-to-end by a single team. Multiple techniques exist to bring the application features—some old and some new—together as a cohesive user experience, but the goal remains to allow each feature to be developed, tested and deployed independently from others. The <a href=\"/radar/techniques/bff-backend-for-frontends\">BFF - backend for frontends</a> approach works well here, with each team developing a BFF to support its set of application features."
},
{
"name": "OWASP ASVS",
"description": "As more development teams incorporate security earlier in the development life cycle, figuring out requirements to limit security risks can seem like a daunting task. Few people have the extensive technical knowledge needed to identify all the risks that an application might face, and teams might struggle just trying to decide where to begin. Relying on frameworks such as OWASP's <a href=\"https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project\"><strong>ASVS</strong></a> (Application Security Verification Standard) can help make this easier. Although somewhat lengthy, it contains a thorough list of requirements categorized by functions such as authentication, access control, and error handling and logging, which can be reviewed as needed. It is also helpful as a resource for testers when it comes time to verify software."
},
{
"name": "Unikernels",
"description": "With the continued rise to domination of the container model led by Docker adoption, we think it's worth calling attention to the continued rapid development in the <strong>Unikernel</strong> space. Unikernels are single-purpose library operating systems that can be compiled down from high-level languages to run directly on the hypervisors used by commodity cloud platforms. They promise a number of advantages over containers, not least their superfast startup time and very small attack surface area. Many are still at the research-project phase—<a href=\"http://research.microsoft.com/en-us/projects/drawbridge/\">Drawbridge</a> from Microsoft Research, <a href=\"https://mirage.io/\">MirageOS</a> and <a href=\"http://galois.com/project/halvm/\">HaLVM</a> amongst others—but we think the ideas are very interesting and combine nicely with the technique of <a href=\"/radar/techniques/serverless-architecture\">serverless architecture</a>. "
},
{
"name": "VR beyond gaming",
"description": "The idea of virtual reality has been around for more than 50 years, and with successive improvements of computing technology many ideas have been hyped and explored. We believe that we're reaching a tipping point now. Modern graphics cards provide sufficient compute power to render detailed, realistic scenes in high resolutions, and at the same time at least two consumer-oriented VR headsets (the <a href=\"http://www.htcvive.com/uk/\">HTC Vive</a> and Facebook's <a href=\"https://www.oculus.com/en-us/\">Oculus Rift</a>) are coming to market. These headsets are affordable, they have high-resolution displays, and they eliminate perceivable motion-tracking lag, which was causing issues such as headaches and nausea before. The headsets are mainly targeted at enthusiast video gaming, but we are convinced that they will open many possibilities for <strong>VR beyond gaming</strong>, particularly as the low-fi approaches, such as <a href=\"https://www.google.co.uk/get/cardboard/get-cardboard/\">Google Cardboard</a>, are driving greater awareness."
},
{
"name": "A single CI instance for all teams",
"description": "There might be the impression that it's easier to manage a <strong>single CI (Continuous Integration) instance for all teams</strong> because it gives them a single configuration and monitoring point. But a bloated instance that is shared by every team in an organization can cause a lot of damage. We have found that problems like build timeouts, configuration conflicts and gigantic build queues appear more frequently. Having this single point of failure can interrupt the work of many teams. Carefully consider the trade-off between these pitfalls and having a single point of configuration. In organizations with multiple teams, we recommend having CI instances distributed by teams, with enterprise decisions based not on the single CI installation but on defining guidelines about the instances' selection and configuration."
},
{
"name": "Anemic REST",
"description": ""
},
{
"name": "Big Data envy",
"description": "We continue to see organizations chasing \"cool\" technologies, taking on unnecessary complexity and risk when a simpler choice would be better. One particular theme is using distributed, Big Data systems for relatively small data sets. This behavior prompts us to put <strong>Big Data envy</strong> on hold once more, with some additional data points from our recent experience. The <a href=\"http://cassandra.apache.org/\">Apache Cassandra</a> database promises massive scalability on commodity hardware, but we have seen teams overwhelmed by its architectural and operational complexity. Unless you have data volumes that require a 100+ node cluster, we recommend against using Cassandra. The operational team you’ll need to keep the thing running just isn’t worth it. While creating this edition of the Radar, we discussed several new database technologies, many offering \"10x\" performance improvements over existing systems. We’re always skeptical until new technology—especially something as critical as a database—has been properly proven. <a href=\"/radar/tools/jepsen\">Jepsen</a> provides <a href=\"http://jepsen.io/analyses.html\">analysis</a> of database performance under difficult conditions and has found <a href=\"https://aphyr.com/posts/283-call-me-maybe-redis\">numerous</a> <a href=\"https://aphyr.com/posts/284-call-me-maybe-mongodb\">bugs</a> in various NoSQL databases. We recommend maintaining a healthy dose of skepticism and keeping an eye on sites such as Jepsen when you evaluate database tech."
},
{
"name": "Cloud lift and shift",
"description": "As more organizations are choosing to deploy applications in the cloud, we're regularly finding IT groups that are wastefully trying to replicate their existing data center management and security approaches in&nbsp;the cloud. This often comes in the form of firewalls, load balancers, network proxies, access control, security appliances and services that are extended into the cloud with minimal rethinking. We've seen organizations build their own orchestration APIs in front of the cloud providers to constrain the services that can be utilized by teams. In most cases these layers serve only to cripple the capability, taking away most of the intended benefits of moving to the cloud. In this edition of the Radar, we've chosen to rehighlight <strong>cloud lift and shift</strong> as a technique to avoid. Organizations should instead look more deeply at the intent of their existing security and operational controls, and look for alternative controls that work in the cloud without creating unnecessary constraints. Many of those controls will already exist for mature cloud providers, and teams that adopt the cloud can use native APIs for self-serve provisioning and operations."
}
]
}
],
"Tools" : [
{
"name": "adopt",
"blips": [
{
"name": "Babel",
"description": "<a href=\"http://babeljs.io/\"><strong>Babel.js</strong></a> has become the default compiler for writing next-generation JavaScript. Its ecosystem is really taking off, thanks to its restructured <a href=\"http://babeljs.io/docs/plugins/#presets\">plugin system</a>. It allows developers to write <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> (and even ES7) code that runs in the browser or in the server without sacrificing backward compatibility for older browsers, and with very little configuration. It has first-class support for different build-and-test systems, which makes integration with any current workflow simple. It is a great piece of software that has become the main driver of ES6 (and ES7) adoption and innovation."
},
{
"name": "Consul",
"description": "We have moved <strong><a href=\"http://consul.io\">Consul</a></strong>, the service-discovery tool supporting both DNS- and HTTP-based discovery mechanisms, into Adopt. It goes beyond other discovery tools by providing customizable health checks for registered services, ensuring that unhealthy instances are marked accordingly. More tools have emerged to work with Consul to make it even more powerful. <a href=\"https://github.com/hashicorp/consul-template\">Consul Template</a> enables configuration files to be populated with information from Consul, making things like client-side load balancing using mod_proxy much easier. In the world of Docker, <a href=\"https://github.com/gliderlabs/registrator\">registrator</a> can automatically register Docker containers as they appear with Consul with extremely little effort, making it much easier to manage container-based setups. You should still think long and hard about whether you need a tool like this or whether something simpler will do, but if you decide you need service discovery, you won't go far wrong with Consul."
},
{
"name": "Grafana",
"description": "When combining modern techniques and architecture styles, such as <a href=\"/radar/techniques/microservices\">microservices</a>, <a href=\"/radar/techniques/devops\">DevOps</a> and <a href=\"/radar/techniques/qa-in-production\">QA in production</a>, development teams need increasingly sophisticated monitoring. Simply looking a graphs of disk usage and CPU utilization is not sufficient anymore, and many teams collect application and business-specific metrics using tools such a Graphite and Kibana. <strong><a href=\"http://grafana.org/\">Grafana</a></strong> makes it easy to create useful and elegant dashboards for data from a number of sources. A particularly useful feature allows timescales of different graphs to be synchronized, which helps with spotting correlations in the underlying data. The templating system that is being added shows a lot promise and will likely make managing sets of similar services even easier. Based on its strengths, Grafana has become our default choice in this category."
},
{
"name": "Packer",
"description": "Machine images have become a staple of modern deployment pipelines, and there are a number of tools and techniques to create the images. Because of its comprehensive feature set and the positive experiences we've had with it, we recommend <a href=\"http://packer.io\"><strong>Packer</strong></a> over the alternatives. We also recommend against trying to write custom scripts to do what Packer does out of the box."
},
{
"name": "Apache Kafka",
"description": "Many organizations are now looking closely at new data architectures that capture information as immutable sequences of events at scale. <a href=\"http://kafka.apache.org/\"><strong>Apache Kafka</strong></a> continues to build momentum as an open source messaging framework that provides a solution for publishing ordered event feeds to large numbers of independent, lightweight consumers. Configuring Kafka is nontrivial, but our teams are reporting positive experiences with the framework."
},
{
"name": "Espresso",
"description": "At the top of the testing pyramid for Android application development, our teams are increasingly using <strong><a href=\"https://google.github.io/android-testing-support-library/docs/espresso/index.html\">Espresso</a></strong> as the functional-testing tool. Its small-core API hides the messy implementation details and helps in writing concise tests, with faster and reliable test execution. With Espresso, you can run automated UI tests simulating user interactions within a single target app on both emulators and real devices across different Android versions."
},
{
"name": "fastlane",
"description": "<strong><a href=\"https://fastlane.tools/\">fastlane</a></strong> is our go-to tool for automating most of the boring activities involved in getting iOS and Android mobile apps built, tested, documented and provisioned. Simple configuration, a range of tooling and multiple pipelines make this a key ingredient in doing <a href=\"/radar/techniques/continuous-delivery-cd\">continuous delivery</a> for mobile."
},
{
"name": "Galen",
"description": "Testing that layout and styling of responsive websites is working as expected across various form factors can be a slow and often manual process. <strong><a href=\"http://galenframework.com/\">Galen</a></strong> helps ease this problem by providing a simple language, running on top of <a href=\"http://www.seleniumhq.org/\">Selenium</a>, that allows you to specify expectations for the appearance of your website in various screen sizes. Although Galen suffers from the typical brittleness and speed issues of any end-to-end testing approach, we have found benefit in the early feedback on design issues."
},
{
"name": "HashiCorp Vault",
"description": "Having a way to securely manage secrets is increasingly becoming a huge project issue. The old practice of keeping secrets in a file or in environment variables is becoming hard to manage, especially in environments with multiple applications and large numbers of <a href=\"/radar/techniques/microservices\">microservices</a>. <a href=\"https://github.com/hashicorp/vault\"><strong>HashiCorp Vault</strong></a> addresses the problem by providing mechanisms for securely accessing secrets through a unified interface. It has served us well on a number of projects, and our teams liked how easy it was to integrate Vault with their services. Storing and updating secrets is a bit cumbersome, because it relies on a command-line tool and a fair amount of discipline from the team."
},
{
"name": "JSONassert",
"description": ""
},
{
"name": "Let's Encrypt",
"description": "<a href=\"https://letsencrypt.org/\"><strong>Let's Encrypt</strong></a> first appeared on the Radar last edition, and since December 2015 this project has moved its beta status from private to public, meaning users will no longer be required to have an invitation in order to try it. Let's Encrypt grants access to a simpler mechanism to obtain and manage certificates for a larger set of users who are seeking a way to secure their websites. It also promotes a big step forward in terms of security and privacy. This trend has already begun within ThoughtWorks, and many of our projects now have certificates verified by Let's Encrypt."
},
{
"name": "Load Impact",
"description": "<strong><a href=\"https://loadimpact.com/\">Load Impact</a> </strong>is a SaaS load-testing tool that can generate highly realistic loads of up to 1.2 million concurrent users. Record and playback web interactions using a Chrome plugin simulate network connections for mobile or desktop users and generate load from up to 10 different locations around the world. While not the only on-demand load-testing tool we've used—we also like <a href=\"https://blazemeter.com/\">BlazeMeter</a>—our teams were very enthusiastic about Load Impact."
},
{
"name": "OWASP Dependency-Check",
"description": "In a world full of libraries and tools that simplify the life of many software developers, deficiencies in their security have become visible and have increased the vulnerability surface in the applications that use them. <a href=\"https://www.owasp.org/index.php/OWASP_Dependency_Check\"><strong>OWASP Dependency-Check</strong></a> automatically identifies potential security problems in the code, checking if there are any known publicly disclosed vulnerabilities, then using methods to constantly update the database of public vulnerabilities. Dependency-Check has some interfaces and plugins to automate this verification in Java and .NET (which we have used successfully) as well as Ruby, Node.js and Python."
},
{
"name": "Pa11y",
"description": "<a href=\"http://pa11y.org/\"><strong>Pa11y</strong></a> is an automatic accessibility tester that can run from the command line and be embedded into a build pipeline. Our teams have had success using Pa11y on a highly dynamic site by first creating a static HTML version, then running the accessibility tests against that. For many systems—especially government websites—accessibility testing is a requirement, and Pa11y makes it all a lot easier."
},
{
"name": "Serverspec",
"description": "In the past we have included automated <a href=\"/radar/techniques/provisioning-testing\">Provisioning Testing</a> as a recommended technique, and in this issue we highlight <a href=\"http://serverspec.org/\"><strong>Serverspec</strong></a> as a popular tool for implementing those tests. Although this tool is not new, we are seeing its use become more common as more cross-functional delivery teams take on responsibility for infrastructure provisioning. Serverspec is built on the Ruby library RSpec and comes with a comprehensive set of helpers for asserting that server configuration is correct."
},
{
"name": "Talisman",
"description": "With the maturity of tools such as <a href=\"/radar/tools/hashicorp-vault\">Vault</a>, there is no longer an excuse for storing secrets in code repositories, particularly since this often ends up being the soft underbelly of important systems. We’ve previously mentioned repository-scanning tools such as <a href=\"/radar/tools/gitrob\">Gitrob</a>, but we are now pushing proactive tools such as (the ThoughtWorks-created) <strong><a href=\"https://github.com/thoughtworks/talisman\">Talisman</a></strong>, which is a prepush hook for Git that scans commits for secrets matching predefined patterns."
},
{
"name": "Terraform",
"description": "With <strong><a href=\"https://www.terraform.io/\">Terraform</a></strong>, you can manage cloud infrastructure by writing declarative definitions. The configuration of the servers instantiated by Terraform is usually left to tools like Puppet, Chef or Ansible. We like Terraform because the syntax of its files is quite readable and because it supports a number of cloud providers while making no attempt to provide an artificial abstraction across those providers. Following our first, more cautious, mention of Terraform almost two years ago, it has seen continued development and has evolved into a stable product that has proven its value in our projects. The issue with state file management can now be sidestepped by using what Terraform calls a \"remote state backend.\" We’ve successfully used <a href=\"/radar/tools/consul\">Consul</a> for that purpose."
},
{
"name": "tmate",
"description": "Pair programming is an essential technique for us, and—given that we’re seeing more and more teams whose members are distributed across multiple locations—we have experimented with a number of tools to support remote pairing. We certainly liked <a href=\"https://screenhero.com/\">ScreenHero</a> but are concerned about its future. For teams that don’t rely on a graphical IDE, using <strong> <a href=\"https://tmate.io/\">tmate</a></strong> for pairing has turned out to be a great solution. tmate is a fork of the popular tmux tool, and compared to <a href=\"http://hamvocke.com/blog/remote-pair-programming-with-tmux/\">tmux for remote pairing</a>, the setup is much easier. Compared to graphical screen-sharing solutions, the bandwidth and resource requirements are modest, and it obviously never suffers from blurry screens. Teams can also set up their own server, thus retaining full control of the privacy and integrity of the solution."
},
{
"name": "Webpack",
"description": "<a href=\"http://webpack.github.io/\"><strong>Webpack</strong></a> has solidified itself as our go-to JavaScript module bundler. With its ever-growing <a href=\"https://github.com/webpack/docs/wiki/list-of-loaders\">list of loaders</a>, it provides a single dependency tree for all your static assets, allowing flexible manipulation of JavaScript, CSS, etc. and minimizing what needs to be sent to the browser and when. Of particular relevance is the smooth integration among AMD, CommonJS and <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> modules and how it has enabled teams to work in ES6 and seamlessly transpile (using <a href=\"http://babeljs.io/\">Babel</a>) to earlier versions for browser compatibility. Many of our teams also value <a href=\"http://browserify.org/\">Browserify</a>, which covers a similar space but is more focused on making Node.js modules available for client-side use."
},
{
"name": "Zipkin",
"description": "Development on <a href=\"https://github.com/openzipkin/zipkin\"><strong>Zipkin</strong></a> has continued apace, and since the middle of 2015 it has moved to the <em>openzipkin/zipkin</em> organization at GitHub. There are now bindings for Python, Go, Java, Ruby, Scala and C#; and there are Docker images available for those wanting to get started quickly. We still like this tool. There is an active and growing community around usage of it, and implementation is getting easier. If you need a way of measuring the end-to-end latency of many logical requests,&nbsp;Zipkin continues to be a strong choice."
},
{
"name": "Android-x86",
"description": "<strong><a href=\"http://www.android-x86.org/\">Android-x86</a></strong> is a port of the <a href=\"http://source.android.com/\">Android open source</a> project to x86 platforms. The project started by hosting various patches from the community for x86 support but then later created its own codebase to provide support for different x86 platforms. We have seen significant time savings by utilizing Android-x86 in our CI servers instead of emulators for hermetic UI testing. However, for UI-specific tests targeting a particular device resolution—simulating low memory, bandwidth and battery—it is better to stick with emulators."
},
{
"name": "axios",
"description": "Our teams have had success with <a href=\"https://github.com/mzabriskie/axios\"><strong>axios</strong></a>, a promises-based HTTP client in JavaScript that they describe as \"better than <a href=\"/radar/languages-and-frameworks/fetch\">Fetch</a>.\" The project has lots of endorsements and activity on GitHub, and it gets a thumbs-up from us."
},
{
"name": "Bottled Water",
"description": "With the growth of interest in streaming data architectures and the downstream data lakes they feed, we have seen an increased reliance on \"change data capture\" tooling to connect transactional data stores to stream-processing systems. <strong><a href=\"https://github.com/confluentinc/bottledwater-pg\">Bottled Water</a></strong> is a welcome addition to this field, converting changes in <a href=\"/radar/platforms/postgresql-for-nosql\">PostgreSQL</a>’s write-ahead log into <a href=\"/radar/tools/apache-kafka\">Kafka</a> events. One downside of this approach, however, is that you are tied to low-level database events rather than the higher-level <a href=\"/radar/techniques/capture-domain-events-explicitly\">business events</a> we recommend as the foundation for an event-oriented architecture."
},
{
"name": "Clojure.spec",
"description": "One of those perpetual developer debates involves language typing: How much is just right? <a href=\"/radar/languages-and-frameworks/clojure\">Clojure</a>, the dynamically typed functional Lisp on the JVM, added a new entry into this discussion that blurs the lines. <strong><a href=\"https://clojure.org/about/spec\">Clojure.spec</a></strong> is a new facility built into Clojure that allows developers to wrap type and other verification criteria around data structures, such as allowable value ranges. Once they are established, Clojure uses these specifications to provide a slew of benefits: generated tests, validation, destructuring of data structures and others. Clojure.spec is a promising way to have the benefits of types and ranges where developers need them but not everywhere."
},
{
"name": "FBSnapshotTestcase",
"description": "Testing the visual portion of iOS applications can be painful, slow and flakey, which is why we’re happy to include <strong><a href=\"https://github.com/facebook/ios-snapshot-test-case\">FBSnapshotTestcase</a></strong> in our toolkit. It automates taking, storing and diff-ing snapshots of UI components so you can keep your interfaces pixel-perfect. Since it runs as a unit test (in the simulator), it is faster and more reliable than functional-testing approaches."
},
{
"name": "Grasp",
"description": "We had our collective minds blown by a little JavaScript command-line refactoring tool called <strong><a href=\"http://www.graspjs.com/\">Grasp</a></strong>. Providing a rich set of selectors and operating against the abstract syntax tree, it is leagues ahead of fiddling with sed and grep. A useful addition to the toolkit in our ongoing quest to treat <a href=\"/radar/languages-and-frameworks/javascript-as-a-first-class-language\">JavaScript as a first-class language</a>."
},
{
"name": "LambdaCD",
"description": "<a href=\"http://www.lambda.cd\"><strong>LambdaCD</strong></a> provides teams with a way to define Continuous Delivery pipelines in Clojure. This brings the benefits of <a href=\"/radar/tools/infrastructure-as-code\">Infrastructure as code</a> to the configuration of CD servers: source-control management, unit testing, refactoring and code reuse. In the \"pipelines as code\" space, LambdaCD stands out for being lightweight, self-contained and fully programmable, allowing teams to work with their pipelines in the same way that they do with their code."
},
{
"name": "Pinpoint",
"description": "Teams using the Phoenix Server or <a href=\"/radar/techniques/phoenix-environments\">Phoenix Environment </a> techniques have found little in the way of support from Application Performance Management (APM) tools. Their licensing models, based on long-running, limited amounts of tin, and their difficulty in dealing with ephemeral hardware, have meant that they are often more trouble than they are worth. However, distributed systems need monitoring, and at some point many teams recognize the need for an APM tool. We think <strong><a href=\"https://github.com/naver/pinpoint\">Pinpoint</a></strong>, an open source tool in this space, is worth investigating as an alternative to AppDynamics and Dynatrace. Pinpoint is written in Java, with plugins available for many servers, databases and frameworks. While we think you can go a long way using a combination of other lightweight open source tools—<a href=\"/radar/tools/zipkin\">Zipkin</a>, for example—if you are in the market for an APM, Pinpoint is worth considering."
},
{
"name": "Pitest",
"description": "<a href=\"http://pitest.org\"><strong>Pitest</strong></a> is a test coverage analysis tool for Java that uses a mutation-testing technique. Traditional test coverage analysis tends to measure the number of lines that are executed by your tests. It is therefore only able to identify code that is definitely not tested. Mutation testing, on the other hand, tries to test the quality of those lines that are executed by your test code and yet might contain general errors. Several problems can be spotted this way, helping the team to measure and grow a healthy test suite. Most of such tools tend to be slow and difficult to use, but Pitest has proven to have better performance, is easy to set up, and is actively supported."
},
{
"name": "Repsheet",
"description": "Attacks on web properties using bots are becoming more sophisticated. Identifying these bad actors and their behaviors is the goal of the <a href=\"http://getrepsheet.com/\"><strong>Repsheet</strong></a> project. It's a plugin for either Apache or NGINX that records user activity, fingerprints actors using predefined and user-defined rules, and then allows action to be taken, including the ability to block offensive actors. It includes a utility that visualizes current actors; this puts the ability to manage bot-based threats in the hands of team members, increasing security awareness for teams. We like this since it's a good example of a simple tool solving a very real but often invisible problem—bot-based attacks."
},
{
"name": "Scikit-learn",
"description": "<a href=\"http://scikit-learn.org/stable/\"><strong>Scikit-learn</strong></a> is an increasingly popular machine-learning library written in Python. It provides a robust set of machine-learning models such as clustering, classification, regression and dimensionality reduction, and a rich set of functionality for companion tasks like model selection, model evaluation and data preparation. Since it is designed to be simple, reusable in various contexts and well documented, we see this tool accessible even to nonexperts to explore the machine-learning space."
},
{
"name": "Jenkins as a deployment pipeline",
"description": "We know we're in perilous territory here, since we build a competing tool, but we feel we have to address a persistent problem. Continuous Integration tools like CruiseControl and Jenkins are valuable for software development, but as your build process gets more complex it requires something beyond just Continuous Integration: It requires a <a href=\"http://martinfowler.com/bliki/DeploymentPipeline.html\">deployment pipeline</a>. We frequently see people trying to use <strong>Jenkins as a Deployment Pipeline</strong> with the aid of plugins, but our experience is that these quickly become a tangle. Jenkins 2.0 introduces \"Pipeline as Code\" but continues to model pipelines using plugins and fails to change the core Jenkins product to model pipelines directly. In our experience, tools that are built around a first-class representation of deployment pipelines are much more suitable, and this is what drove us to replace CruiseControl with <a href=\"https://www.go.cd/\">GoCD</a>. Today we see several products that embrace deployment pipelines, including <a href=\"/radar/tools/concourse-ci\">ConcourseCI</a>, <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://spinnaker.io/\">Spinnaker,</a> <a href=\"https://github.com/drone\">Drone</a> and <a href=\"/radar/tools/gocd\">GoCD</a>."
}
]
},
{
"name": "trial",
"blips": [
{
"name": "Babel",
"description": "<a href=\"http://babeljs.io/\"><strong>Babel.js</strong></a> has become the default compiler for writing next-generation JavaScript. Its ecosystem is really taking off, thanks to its restructured <a href=\"http://babeljs.io/docs/plugins/#presets\">plugin system</a>. It allows developers to write <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> (and even ES7) code that runs in the browser or in the server without sacrificing backward compatibility for older browsers, and with very little configuration. It has first-class support for different build-and-test systems, which makes integration with any current workflow simple. It is a great piece of software that has become the main driver of ES6 (and ES7) adoption and innovation."
},
{
"name": "Consul",
"description": "We have moved <strong><a href=\"http://consul.io\">Consul</a></strong>, the service-discovery tool supporting both DNS- and HTTP-based discovery mechanisms, into Adopt. It goes beyond other discovery tools by providing customizable health checks for registered services, ensuring that unhealthy instances are marked accordingly. More tools have emerged to work with Consul to make it even more powerful. <a href=\"https://github.com/hashicorp/consul-template\">Consul Template</a> enables configuration files to be populated with information from Consul, making things like client-side load balancing using mod_proxy much easier. In the world of Docker, <a href=\"https://github.com/gliderlabs/registrator\">registrator</a> can automatically register Docker containers as they appear with Consul with extremely little effort, making it much easier to manage container-based setups. You should still think long and hard about whether you need a tool like this or whether something simpler will do, but if you decide you need service discovery, you won't go far wrong with Consul."
},
{
"name": "Grafana",
"description": "When combining modern techniques and architecture styles, such as <a href=\"/radar/techniques/microservices\">microservices</a>, <a href=\"/radar/techniques/devops\">DevOps</a> and <a href=\"/radar/techniques/qa-in-production\">QA in production</a>, development teams need increasingly sophisticated monitoring. Simply looking a graphs of disk usage and CPU utilization is not sufficient anymore, and many teams collect application and business-specific metrics using tools such a Graphite and Kibana. <strong><a href=\"http://grafana.org/\">Grafana</a></strong> makes it easy to create useful and elegant dashboards for data from a number of sources. A particularly useful feature allows timescales of different graphs to be synchronized, which helps with spotting correlations in the underlying data. The templating system that is being added shows a lot promise and will likely make managing sets of similar services even easier. Based on its strengths, Grafana has become our default choice in this category."
},
{
"name": "Packer",
"description": "Machine images have become a staple of modern deployment pipelines, and there are a number of tools and techniques to create the images. Because of its comprehensive feature set and the positive experiences we've had with it, we recommend <a href=\"http://packer.io\"><strong>Packer</strong></a> over the alternatives. We also recommend against trying to write custom scripts to do what Packer does out of the box."
},
{
"name": "Apache Kafka",
"description": "Many organizations are now looking closely at new data architectures that capture information as immutable sequences of events at scale. <a href=\"http://kafka.apache.org/\"><strong>Apache Kafka</strong></a> continues to build momentum as an open source messaging framework that provides a solution for publishing ordered event feeds to large numbers of independent, lightweight consumers. Configuring Kafka is nontrivial, but our teams are reporting positive experiences with the framework."
},
{
"name": "Espresso",
"description": "At the top of the testing pyramid for Android application development, our teams are increasingly using <strong><a href=\"https://google.github.io/android-testing-support-library/docs/espresso/index.html\">Espresso</a></strong> as the functional-testing tool. Its small-core API hides the messy implementation details and helps in writing concise tests, with faster and reliable test execution. With Espresso, you can run automated UI tests simulating user interactions within a single target app on both emulators and real devices across different Android versions."
},
{
"name": "fastlane",
"description": "<strong><a href=\"https://fastlane.tools/\">fastlane</a></strong> is our go-to tool for automating most of the boring activities involved in getting iOS and Android mobile apps built, tested, documented and provisioned. Simple configuration, a range of tooling and multiple pipelines make this a key ingredient in doing <a href=\"/radar/techniques/continuous-delivery-cd\">continuous delivery</a> for mobile."
},
{
"name": "Galen",
"description": "Testing that layout and styling of responsive websites is working as expected across various form factors can be a slow and often manual process. <strong><a href=\"http://galenframework.com/\">Galen</a></strong> helps ease this problem by providing a simple language, running on top of <a href=\"http://www.seleniumhq.org/\">Selenium</a>, that allows you to specify expectations for the appearance of your website in various screen sizes. Although Galen suffers from the typical brittleness and speed issues of any end-to-end testing approach, we have found benefit in the early feedback on design issues."
},
{
"name": "HashiCorp Vault",
"description": "Having a way to securely manage secrets is increasingly becoming a huge project issue. The old practice of keeping secrets in a file or in environment variables is becoming hard to manage, especially in environments with multiple applications and large numbers of <a href=\"/radar/techniques/microservices\">microservices</a>. <a href=\"https://github.com/hashicorp/vault\"><strong>HashiCorp Vault</strong></a> addresses the problem by providing mechanisms for securely accessing secrets through a unified interface. It has served us well on a number of projects, and our teams liked how easy it was to integrate Vault with their services. Storing and updating secrets is a bit cumbersome, because it relies on a command-line tool and a fair amount of discipline from the team."
},
{
"name": "JSONassert",
"description": ""
},
{
"name": "Let's Encrypt",
"description": "<a href=\"https://letsencrypt.org/\"><strong>Let's Encrypt</strong></a> first appeared on the Radar last edition, and since December 2015 this project has moved its beta status from private to public, meaning users will no longer be required to have an invitation in order to try it. Let's Encrypt grants access to a simpler mechanism to obtain and manage certificates for a larger set of users who are seeking a way to secure their websites. It also promotes a big step forward in terms of security and privacy. This trend has already begun within ThoughtWorks, and many of our projects now have certificates verified by Let's Encrypt."
},
{
"name": "Load Impact",
"description": "<strong><a href=\"https://loadimpact.com/\">Load Impact</a> </strong>is a SaaS load-testing tool that can generate highly realistic loads of up to 1.2 million concurrent users. Record and playback web interactions using a Chrome plugin simulate network connections for mobile or desktop users and generate load from up to 10 different locations around the world. While not the only on-demand load-testing tool we've used—we also like <a href=\"https://blazemeter.com/\">BlazeMeter</a>—our teams were very enthusiastic about Load Impact."
},
{
"name": "OWASP Dependency-Check",
"description": "In a world full of libraries and tools that simplify the life of many software developers, deficiencies in their security have become visible and have increased the vulnerability surface in the applications that use them. <a href=\"https://www.owasp.org/index.php/OWASP_Dependency_Check\"><strong>OWASP Dependency-Check</strong></a> automatically identifies potential security problems in the code, checking if there are any known publicly disclosed vulnerabilities, then using methods to constantly update the database of public vulnerabilities. Dependency-Check has some interfaces and plugins to automate this verification in Java and .NET (which we have used successfully) as well as Ruby, Node.js and Python."
},
{
"name": "Pa11y",
"description": "<a href=\"http://pa11y.org/\"><strong>Pa11y</strong></a> is an automatic accessibility tester that can run from the command line and be embedded into a build pipeline. Our teams have had success using Pa11y on a highly dynamic site by first creating a static HTML version, then running the accessibility tests against that. For many systems—especially government websites—accessibility testing is a requirement, and Pa11y makes it all a lot easier."
},
{
"name": "Serverspec",
"description": "In the past we have included automated <a href=\"/radar/techniques/provisioning-testing\">Provisioning Testing</a> as a recommended technique, and in this issue we highlight <a href=\"http://serverspec.org/\"><strong>Serverspec</strong></a> as a popular tool for implementing those tests. Although this tool is not new, we are seeing its use become more common as more cross-functional delivery teams take on responsibility for infrastructure provisioning. Serverspec is built on the Ruby library RSpec and comes with a comprehensive set of helpers for asserting that server configuration is correct."
},
{
"name": "Talisman",
"description": "With the maturity of tools such as <a href=\"/radar/tools/hashicorp-vault\">Vault</a>, there is no longer an excuse for storing secrets in code repositories, particularly since this often ends up being the soft underbelly of important systems. We’ve previously mentioned repository-scanning tools such as <a href=\"/radar/tools/gitrob\">Gitrob</a>, but we are now pushing proactive tools such as (the ThoughtWorks-created) <strong><a href=\"https://github.com/thoughtworks/talisman\">Talisman</a></strong>, which is a prepush hook for Git that scans commits for secrets matching predefined patterns."
},
{
"name": "Terraform",
"description": "With <strong><a href=\"https://www.terraform.io/\">Terraform</a></strong>, you can manage cloud infrastructure by writing declarative definitions. The configuration of the servers instantiated by Terraform is usually left to tools like Puppet, Chef or Ansible. We like Terraform because the syntax of its files is quite readable and because it supports a number of cloud providers while making no attempt to provide an artificial abstraction across those providers. Following our first, more cautious, mention of Terraform almost two years ago, it has seen continued development and has evolved into a stable product that has proven its value in our projects. The issue with state file management can now be sidestepped by using what Terraform calls a \"remote state backend.\" We’ve successfully used <a href=\"/radar/tools/consul\">Consul</a> for that purpose."
},
{
"name": "tmate",
"description": "Pair programming is an essential technique for us, and—given that we’re seeing more and more teams whose members are distributed across multiple locations—we have experimented with a number of tools to support remote pairing. We certainly liked <a href=\"https://screenhero.com/\">ScreenHero</a> but are concerned about its future. For teams that don’t rely on a graphical IDE, using <strong> <a href=\"https://tmate.io/\">tmate</a></strong> for pairing has turned out to be a great solution. tmate is a fork of the popular tmux tool, and compared to <a href=\"http://hamvocke.com/blog/remote-pair-programming-with-tmux/\">tmux for remote pairing</a>, the setup is much easier. Compared to graphical screen-sharing solutions, the bandwidth and resource requirements are modest, and it obviously never suffers from blurry screens. Teams can also set up their own server, thus retaining full control of the privacy and integrity of the solution."
},
{
"name": "Webpack",
"description": "<a href=\"http://webpack.github.io/\"><strong>Webpack</strong></a> has solidified itself as our go-to JavaScript module bundler. With its ever-growing <a href=\"https://github.com/webpack/docs/wiki/list-of-loaders\">list of loaders</a>, it provides a single dependency tree for all your static assets, allowing flexible manipulation of JavaScript, CSS, etc. and minimizing what needs to be sent to the browser and when. Of particular relevance is the smooth integration among AMD, CommonJS and <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> modules and how it has enabled teams to work in ES6 and seamlessly transpile (using <a href=\"http://babeljs.io/\">Babel</a>) to earlier versions for browser compatibility. Many of our teams also value <a href=\"http://browserify.org/\">Browserify</a>, which covers a similar space but is more focused on making Node.js modules available for client-side use."
},
{
"name": "Zipkin",
"description": "Development on <a href=\"https://github.com/openzipkin/zipkin\"><strong>Zipkin</strong></a> has continued apace, and since the middle of 2015 it has moved to the <em>openzipkin/zipkin</em> organization at GitHub. There are now bindings for Python, Go, Java, Ruby, Scala and C#; and there are Docker images available for those wanting to get started quickly. We still like this tool. There is an active and growing community around usage of it, and implementation is getting easier. If you need a way of measuring the end-to-end latency of many logical requests,&nbsp;Zipkin continues to be a strong choice."
},
{
"name": "Android-x86",
"description": "<strong><a href=\"http://www.android-x86.org/\">Android-x86</a></strong> is a port of the <a href=\"http://source.android.com/\">Android open source</a> project to x86 platforms. The project started by hosting various patches from the community for x86 support but then later created its own codebase to provide support for different x86 platforms. We have seen significant time savings by utilizing Android-x86 in our CI servers instead of emulators for hermetic UI testing. However, for UI-specific tests targeting a particular device resolution—simulating low memory, bandwidth and battery—it is better to stick with emulators."
},
{
"name": "axios",
"description": "Our teams have had success with <a href=\"https://github.com/mzabriskie/axios\"><strong>axios</strong></a>, a promises-based HTTP client in JavaScript that they describe as \"better than <a href=\"/radar/languages-and-frameworks/fetch\">Fetch</a>.\" The project has lots of endorsements and activity on GitHub, and it gets a thumbs-up from us."
},
{
"name": "Bottled Water",
"description": "With the growth of interest in streaming data architectures and the downstream data lakes they feed, we have seen an increased reliance on \"change data capture\" tooling to connect transactional data stores to stream-processing systems. <strong><a href=\"https://github.com/confluentinc/bottledwater-pg\">Bottled Water</a></strong> is a welcome addition to this field, converting changes in <a href=\"/radar/platforms/postgresql-for-nosql\">PostgreSQL</a>’s write-ahead log into <a href=\"/radar/tools/apache-kafka\">Kafka</a> events. One downside of this approach, however, is that you are tied to low-level database events rather than the higher-level <a href=\"/radar/techniques/capture-domain-events-explicitly\">business events</a> we recommend as the foundation for an event-oriented architecture."
},
{
"name": "Clojure.spec",
"description": "One of those perpetual developer debates involves language typing: How much is just right? <a href=\"/radar/languages-and-frameworks/clojure\">Clojure</a>, the dynamically typed functional Lisp on the JVM, added a new entry into this discussion that blurs the lines. <strong><a href=\"https://clojure.org/about/spec\">Clojure.spec</a></strong> is a new facility built into Clojure that allows developers to wrap type and other verification criteria around data structures, such as allowable value ranges. Once they are established, Clojure uses these specifications to provide a slew of benefits: generated tests, validation, destructuring of data structures and others. Clojure.spec is a promising way to have the benefits of types and ranges where developers need them but not everywhere."
},
{
"name": "FBSnapshotTestcase",
"description": "Testing the visual portion of iOS applications can be painful, slow and flakey, which is why we’re happy to include <strong><a href=\"https://github.com/facebook/ios-snapshot-test-case\">FBSnapshotTestcase</a></strong> in our toolkit. It automates taking, storing and diff-ing snapshots of UI components so you can keep your interfaces pixel-perfect. Since it runs as a unit test (in the simulator), it is faster and more reliable than functional-testing approaches."
},
{
"name": "Grasp",
"description": "We had our collective minds blown by a little JavaScript command-line refactoring tool called <strong><a href=\"http://www.graspjs.com/\">Grasp</a></strong>. Providing a rich set of selectors and operating against the abstract syntax tree, it is leagues ahead of fiddling with sed and grep. A useful addition to the toolkit in our ongoing quest to treat <a href=\"/radar/languages-and-frameworks/javascript-as-a-first-class-language\">JavaScript as a first-class language</a>."
},
{
"name": "LambdaCD",
"description": "<a href=\"http://www.lambda.cd\"><strong>LambdaCD</strong></a> provides teams with a way to define Continuous Delivery pipelines in Clojure. This brings the benefits of <a href=\"/radar/tools/infrastructure-as-code\">Infrastructure as code</a> to the configuration of CD servers: source-control management, unit testing, refactoring and code reuse. In the \"pipelines as code\" space, LambdaCD stands out for being lightweight, self-contained and fully programmable, allowing teams to work with their pipelines in the same way that they do with their code."
},
{
"name": "Pinpoint",
"description": "Teams using the Phoenix Server or <a href=\"/radar/techniques/phoenix-environments\">Phoenix Environment </a> techniques have found little in the way of support from Application Performance Management (APM) tools. Their licensing models, based on long-running, limited amounts of tin, and their difficulty in dealing with ephemeral hardware, have meant that they are often more trouble than they are worth. However, distributed systems need monitoring, and at some point many teams recognize the need for an APM tool. We think <strong><a href=\"https://github.com/naver/pinpoint\">Pinpoint</a></strong>, an open source tool in this space, is worth investigating as an alternative to AppDynamics and Dynatrace. Pinpoint is written in Java, with plugins available for many servers, databases and frameworks. While we think you can go a long way using a combination of other lightweight open source tools—<a href=\"/radar/tools/zipkin\">Zipkin</a>, for example—if you are in the market for an APM, Pinpoint is worth considering."
},
{
"name": "Pitest",
"description": "<a href=\"http://pitest.org\"><strong>Pitest</strong></a> is a test coverage analysis tool for Java that uses a mutation-testing technique. Traditional test coverage analysis tends to measure the number of lines that are executed by your tests. It is therefore only able to identify code that is definitely not tested. Mutation testing, on the other hand, tries to test the quality of those lines that are executed by your test code and yet might contain general errors. Several problems can be spotted this way, helping the team to measure and grow a healthy test suite. Most of such tools tend to be slow and difficult to use, but Pitest has proven to have better performance, is easy to set up, and is actively supported."
},
{
"name": "Repsheet",
"description": "Attacks on web properties using bots are becoming more sophisticated. Identifying these bad actors and their behaviors is the goal of the <a href=\"http://getrepsheet.com/\"><strong>Repsheet</strong></a> project. It's a plugin for either Apache or NGINX that records user activity, fingerprints actors using predefined and user-defined rules, and then allows action to be taken, including the ability to block offensive actors. It includes a utility that visualizes current actors; this puts the ability to manage bot-based threats in the hands of team members, increasing security awareness for teams. We like this since it's a good example of a simple tool solving a very real but often invisible problem—bot-based attacks."
},
{
"name": "Scikit-learn",
"description": "<a href=\"http://scikit-learn.org/stable/\"><strong>Scikit-learn</strong></a> is an increasingly popular machine-learning library written in Python. It provides a robust set of machine-learning models such as clustering, classification, regression and dimensionality reduction, and a rich set of functionality for companion tasks like model selection, model evaluation and data preparation. Since it is designed to be simple, reusable in various contexts and well documented, we see this tool accessible even to nonexperts to explore the machine-learning space."
},
{
"name": "Jenkins as a deployment pipeline",
"description": "We know we're in perilous territory here, since we build a competing tool, but we feel we have to address a persistent problem. Continuous Integration tools like CruiseControl and Jenkins are valuable for software development, but as your build process gets more complex it requires something beyond just Continuous Integration: It requires a <a href=\"http://martinfowler.com/bliki/DeploymentPipeline.html\">deployment pipeline</a>. We frequently see people trying to use <strong>Jenkins as a Deployment Pipeline</strong> with the aid of plugins, but our experience is that these quickly become a tangle. Jenkins 2.0 introduces \"Pipeline as Code\" but continues to model pipelines using plugins and fails to change the core Jenkins product to model pipelines directly. In our experience, tools that are built around a first-class representation of deployment pipelines are much more suitable, and this is what drove us to replace CruiseControl with <a href=\"https://www.go.cd/\">GoCD</a>. Today we see several products that embrace deployment pipelines, including <a href=\"/radar/tools/concourse-ci\">ConcourseCI</a>, <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://spinnaker.io/\">Spinnaker,</a> <a href=\"https://github.com/drone\">Drone</a> and <a href=\"/radar/tools/gocd\">GoCD</a>."
}
]
},
{
"name": "assess",
"blips": [
{
"name": "Babel",
"description": "<a href=\"http://babeljs.io/\"><strong>Babel.js</strong></a> has become the default compiler for writing next-generation JavaScript. Its ecosystem is really taking off, thanks to its restructured <a href=\"http://babeljs.io/docs/plugins/#presets\">plugin system</a>. It allows developers to write <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> (and even ES7) code that runs in the browser or in the server without sacrificing backward compatibility for older browsers, and with very little configuration. It has first-class support for different build-and-test systems, which makes integration with any current workflow simple. It is a great piece of software that has become the main driver of ES6 (and ES7) adoption and innovation."
},
{
"name": "Consul",
"description": "We have moved <strong><a href=\"http://consul.io\">Consul</a></strong>, the service-discovery tool supporting both DNS- and HTTP-based discovery mechanisms, into Adopt. It goes beyond other discovery tools by providing customizable health checks for registered services, ensuring that unhealthy instances are marked accordingly. More tools have emerged to work with Consul to make it even more powerful. <a href=\"https://github.com/hashicorp/consul-template\">Consul Template</a> enables configuration files to be populated with information from Consul, making things like client-side load balancing using mod_proxy much easier. In the world of Docker, <a href=\"https://github.com/gliderlabs/registrator\">registrator</a> can automatically register Docker containers as they appear with Consul with extremely little effort, making it much easier to manage container-based setups. You should still think long and hard about whether you need a tool like this or whether something simpler will do, but if you decide you need service discovery, you won't go far wrong with Consul."
},
{
"name": "Grafana",
"description": "When combining modern techniques and architecture styles, such as <a href=\"/radar/techniques/microservices\">microservices</a>, <a href=\"/radar/techniques/devops\">DevOps</a> and <a href=\"/radar/techniques/qa-in-production\">QA in production</a>, development teams need increasingly sophisticated monitoring. Simply looking a graphs of disk usage and CPU utilization is not sufficient anymore, and many teams collect application and business-specific metrics using tools such a Graphite and Kibana. <strong><a href=\"http://grafana.org/\">Grafana</a></strong> makes it easy to create useful and elegant dashboards for data from a number of sources. A particularly useful feature allows timescales of different graphs to be synchronized, which helps with spotting correlations in the underlying data. The templating system that is being added shows a lot promise and will likely make managing sets of similar services even easier. Based on its strengths, Grafana has become our default choice in this category."
},
{
"name": "Packer",
"description": "Machine images have become a staple of modern deployment pipelines, and there are a number of tools and techniques to create the images. Because of its comprehensive feature set and the positive experiences we've had with it, we recommend <a href=\"http://packer.io\"><strong>Packer</strong></a> over the alternatives. We also recommend against trying to write custom scripts to do what Packer does out of the box."
},
{
"name": "Apache Kafka",
"description": "Many organizations are now looking closely at new data architectures that capture information as immutable sequences of events at scale. <a href=\"http://kafka.apache.org/\"><strong>Apache Kafka</strong></a> continues to build momentum as an open source messaging framework that provides a solution for publishing ordered event feeds to large numbers of independent, lightweight consumers. Configuring Kafka is nontrivial, but our teams are reporting positive experiences with the framework."
},
{
"name": "Espresso",
"description": "At the top of the testing pyramid for Android application development, our teams are increasingly using <strong><a href=\"https://google.github.io/android-testing-support-library/docs/espresso/index.html\">Espresso</a></strong> as the functional-testing tool. Its small-core API hides the messy implementation details and helps in writing concise tests, with faster and reliable test execution. With Espresso, you can run automated UI tests simulating user interactions within a single target app on both emulators and real devices across different Android versions."
},
{
"name": "fastlane",
"description": "<strong><a href=\"https://fastlane.tools/\">fastlane</a></strong> is our go-to tool for automating most of the boring activities involved in getting iOS and Android mobile apps built, tested, documented and provisioned. Simple configuration, a range of tooling and multiple pipelines make this a key ingredient in doing <a href=\"/radar/techniques/continuous-delivery-cd\">continuous delivery</a> for mobile."
},
{
"name": "Galen",
"description": "Testing that layout and styling of responsive websites is working as expected across various form factors can be a slow and often manual process. <strong><a href=\"http://galenframework.com/\">Galen</a></strong> helps ease this problem by providing a simple language, running on top of <a href=\"http://www.seleniumhq.org/\">Selenium</a>, that allows you to specify expectations for the appearance of your website in various screen sizes. Although Galen suffers from the typical brittleness and speed issues of any end-to-end testing approach, we have found benefit in the early feedback on design issues."
},
{
"name": "HashiCorp Vault",
"description": "Having a way to securely manage secrets is increasingly becoming a huge project issue. The old practice of keeping secrets in a file or in environment variables is becoming hard to manage, especially in environments with multiple applications and large numbers of <a href=\"/radar/techniques/microservices\">microservices</a>. <a href=\"https://github.com/hashicorp/vault\"><strong>HashiCorp Vault</strong></a> addresses the problem by providing mechanisms for securely accessing secrets through a unified interface. It has served us well on a number of projects, and our teams liked how easy it was to integrate Vault with their services. Storing and updating secrets is a bit cumbersome, because it relies on a command-line tool and a fair amount of discipline from the team."
},
{
"name": "JSONassert",
"description": ""
},
{
"name": "Let's Encrypt",
"description": "<a href=\"https://letsencrypt.org/\"><strong>Let's Encrypt</strong></a> first appeared on the Radar last edition, and since December 2015 this project has moved its beta status from private to public, meaning users will no longer be required to have an invitation in order to try it. Let's Encrypt grants access to a simpler mechanism to obtain and manage certificates for a larger set of users who are seeking a way to secure their websites. It also promotes a big step forward in terms of security and privacy. This trend has already begun within ThoughtWorks, and many of our projects now have certificates verified by Let's Encrypt."
},
{
"name": "Load Impact",
"description": "<strong><a href=\"https://loadimpact.com/\">Load Impact</a> </strong>is a SaaS load-testing tool that can generate highly realistic loads of up to 1.2 million concurrent users. Record and playback web interactions using a Chrome plugin simulate network connections for mobile or desktop users and generate load from up to 10 different locations around the world. While not the only on-demand load-testing tool we've used—we also like <a href=\"https://blazemeter.com/\">BlazeMeter</a>—our teams were very enthusiastic about Load Impact."
},
{
"name": "OWASP Dependency-Check",
"description": "In a world full of libraries and tools that simplify the life of many software developers, deficiencies in their security have become visible and have increased the vulnerability surface in the applications that use them. <a href=\"https://www.owasp.org/index.php/OWASP_Dependency_Check\"><strong>OWASP Dependency-Check</strong></a> automatically identifies potential security problems in the code, checking if there are any known publicly disclosed vulnerabilities, then using methods to constantly update the database of public vulnerabilities. Dependency-Check has some interfaces and plugins to automate this verification in Java and .NET (which we have used successfully) as well as Ruby, Node.js and Python."
},
{
"name": "Pa11y",
"description": "<a href=\"http://pa11y.org/\"><strong>Pa11y</strong></a> is an automatic accessibility tester that can run from the command line and be embedded into a build pipeline. Our teams have had success using Pa11y on a highly dynamic site by first creating a static HTML version, then running the accessibility tests against that. For many systems—especially government websites—accessibility testing is a requirement, and Pa11y makes it all a lot easier."
},
{
"name": "Serverspec",
"description": "In the past we have included automated <a href=\"/radar/techniques/provisioning-testing\">Provisioning Testing</a> as a recommended technique, and in this issue we highlight <a href=\"http://serverspec.org/\"><strong>Serverspec</strong></a> as a popular tool for implementing those tests. Although this tool is not new, we are seeing its use become more common as more cross-functional delivery teams take on responsibility for infrastructure provisioning. Serverspec is built on the Ruby library RSpec and comes with a comprehensive set of helpers for asserting that server configuration is correct."
},
{
"name": "Talisman",
"description": "With the maturity of tools such as <a href=\"/radar/tools/hashicorp-vault\">Vault</a>, there is no longer an excuse for storing secrets in code repositories, particularly since this often ends up being the soft underbelly of important systems. We’ve previously mentioned repository-scanning tools such as <a href=\"/radar/tools/gitrob\">Gitrob</a>, but we are now pushing proactive tools such as (the ThoughtWorks-created) <strong><a href=\"https://github.com/thoughtworks/talisman\">Talisman</a></strong>, which is a prepush hook for Git that scans commits for secrets matching predefined patterns."
},
{
"name": "Terraform",
"description": "With <strong><a href=\"https://www.terraform.io/\">Terraform</a></strong>, you can manage cloud infrastructure by writing declarative definitions. The configuration of the servers instantiated by Terraform is usually left to tools like Puppet, Chef or Ansible. We like Terraform because the syntax of its files is quite readable and because it supports a number of cloud providers while making no attempt to provide an artificial abstraction across those providers. Following our first, more cautious, mention of Terraform almost two years ago, it has seen continued development and has evolved into a stable product that has proven its value in our projects. The issue with state file management can now be sidestepped by using what Terraform calls a \"remote state backend.\" We’ve successfully used <a href=\"/radar/tools/consul\">Consul</a> for that purpose."
},
{
"name": "tmate",
"description": "Pair programming is an essential technique for us, and—given that we’re seeing more and more teams whose members are distributed across multiple locations—we have experimented with a number of tools to support remote pairing. We certainly liked <a href=\"https://screenhero.com/\">ScreenHero</a> but are concerned about its future. For teams that don’t rely on a graphical IDE, using <strong> <a href=\"https://tmate.io/\">tmate</a></strong> for pairing has turned out to be a great solution. tmate is a fork of the popular tmux tool, and compared to <a href=\"http://hamvocke.com/blog/remote-pair-programming-with-tmux/\">tmux for remote pairing</a>, the setup is much easier. Compared to graphical screen-sharing solutions, the bandwidth and resource requirements are modest, and it obviously never suffers from blurry screens. Teams can also set up their own server, thus retaining full control of the privacy and integrity of the solution."
},
{
"name": "Webpack",
"description": "<a href=\"http://webpack.github.io/\"><strong>Webpack</strong></a> has solidified itself as our go-to JavaScript module bundler. With its ever-growing <a href=\"https://github.com/webpack/docs/wiki/list-of-loaders\">list of loaders</a>, it provides a single dependency tree for all your static assets, allowing flexible manipulation of JavaScript, CSS, etc. and minimizing what needs to be sent to the browser and when. Of particular relevance is the smooth integration among AMD, CommonJS and <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> modules and how it has enabled teams to work in ES6 and seamlessly transpile (using <a href=\"http://babeljs.io/\">Babel</a>) to earlier versions for browser compatibility. Many of our teams also value <a href=\"http://browserify.org/\">Browserify</a>, which covers a similar space but is more focused on making Node.js modules available for client-side use."
},
{
"name": "Zipkin",
"description": "Development on <a href=\"https://github.com/openzipkin/zipkin\"><strong>Zipkin</strong></a> has continued apace, and since the middle of 2015 it has moved to the <em>openzipkin/zipkin</em> organization at GitHub. There are now bindings for Python, Go, Java, Ruby, Scala and C#; and there are Docker images available for those wanting to get started quickly. We still like this tool. There is an active and growing community around usage of it, and implementation is getting easier. If you need a way of measuring the end-to-end latency of many logical requests,&nbsp;Zipkin continues to be a strong choice."
},
{
"name": "Android-x86",
"description": "<strong><a href=\"http://www.android-x86.org/\">Android-x86</a></strong> is a port of the <a href=\"http://source.android.com/\">Android open source</a> project to x86 platforms. The project started by hosting various patches from the community for x86 support but then later created its own codebase to provide support for different x86 platforms. We have seen significant time savings by utilizing Android-x86 in our CI servers instead of emulators for hermetic UI testing. However, for UI-specific tests targeting a particular device resolution—simulating low memory, bandwidth and battery—it is better to stick with emulators."
},
{
"name": "axios",
"description": "Our teams have had success with <a href=\"https://github.com/mzabriskie/axios\"><strong>axios</strong></a>, a promises-based HTTP client in JavaScript that they describe as \"better than <a href=\"/radar/languages-and-frameworks/fetch\">Fetch</a>.\" The project has lots of endorsements and activity on GitHub, and it gets a thumbs-up from us."
},
{
"name": "Bottled Water",
"description": "With the growth of interest in streaming data architectures and the downstream data lakes they feed, we have seen an increased reliance on \"change data capture\" tooling to connect transactional data stores to stream-processing systems. <strong><a href=\"https://github.com/confluentinc/bottledwater-pg\">Bottled Water</a></strong> is a welcome addition to this field, converting changes in <a href=\"/radar/platforms/postgresql-for-nosql\">PostgreSQL</a>’s write-ahead log into <a href=\"/radar/tools/apache-kafka\">Kafka</a> events. One downside of this approach, however, is that you are tied to low-level database events rather than the higher-level <a href=\"/radar/techniques/capture-domain-events-explicitly\">business events</a> we recommend as the foundation for an event-oriented architecture."
},
{
"name": "Clojure.spec",
"description": "One of those perpetual developer debates involves language typing: How much is just right? <a href=\"/radar/languages-and-frameworks/clojure\">Clojure</a>, the dynamically typed functional Lisp on the JVM, added a new entry into this discussion that blurs the lines. <strong><a href=\"https://clojure.org/about/spec\">Clojure.spec</a></strong> is a new facility built into Clojure that allows developers to wrap type and other verification criteria around data structures, such as allowable value ranges. Once they are established, Clojure uses these specifications to provide a slew of benefits: generated tests, validation, destructuring of data structures and others. Clojure.spec is a promising way to have the benefits of types and ranges where developers need them but not everywhere."
},
{
"name": "FBSnapshotTestcase",
"description": "Testing the visual portion of iOS applications can be painful, slow and flakey, which is why we’re happy to include <strong><a href=\"https://github.com/facebook/ios-snapshot-test-case\">FBSnapshotTestcase</a></strong> in our toolkit. It automates taking, storing and diff-ing snapshots of UI components so you can keep your interfaces pixel-perfect. Since it runs as a unit test (in the simulator), it is faster and more reliable than functional-testing approaches."
},
{
"name": "Grasp",
"description": "We had our collective minds blown by a little JavaScript command-line refactoring tool called <strong><a href=\"http://www.graspjs.com/\">Grasp</a></strong>. Providing a rich set of selectors and operating against the abstract syntax tree, it is leagues ahead of fiddling with sed and grep. A useful addition to the toolkit in our ongoing quest to treat <a href=\"/radar/languages-and-frameworks/javascript-as-a-first-class-language\">JavaScript as a first-class language</a>."
},
{
"name": "LambdaCD",
"description": "<a href=\"http://www.lambda.cd\"><strong>LambdaCD</strong></a> provides teams with a way to define Continuous Delivery pipelines in Clojure. This brings the benefits of <a href=\"/radar/tools/infrastructure-as-code\">Infrastructure as code</a> to the configuration of CD servers: source-control management, unit testing, refactoring and code reuse. In the \"pipelines as code\" space, LambdaCD stands out for being lightweight, self-contained and fully programmable, allowing teams to work with their pipelines in the same way that they do with their code."
},
{
"name": "Pinpoint",
"description": "Teams using the Phoenix Server or <a href=\"/radar/techniques/phoenix-environments\">Phoenix Environment </a> techniques have found little in the way of support from Application Performance Management (APM) tools. Their licensing models, based on long-running, limited amounts of tin, and their difficulty in dealing with ephemeral hardware, have meant that they are often more trouble than they are worth. However, distributed systems need monitoring, and at some point many teams recognize the need for an APM tool. We think <strong><a href=\"https://github.com/naver/pinpoint\">Pinpoint</a></strong>, an open source tool in this space, is worth investigating as an alternative to AppDynamics and Dynatrace. Pinpoint is written in Java, with plugins available for many servers, databases and frameworks. While we think you can go a long way using a combination of other lightweight open source tools—<a href=\"/radar/tools/zipkin\">Zipkin</a>, for example—if you are in the market for an APM, Pinpoint is worth considering."
},
{
"name": "Pitest",
"description": "<a href=\"http://pitest.org\"><strong>Pitest</strong></a> is a test coverage analysis tool for Java that uses a mutation-testing technique. Traditional test coverage analysis tends to measure the number of lines that are executed by your tests. It is therefore only able to identify code that is definitely not tested. Mutation testing, on the other hand, tries to test the quality of those lines that are executed by your test code and yet might contain general errors. Several problems can be spotted this way, helping the team to measure and grow a healthy test suite. Most of such tools tend to be slow and difficult to use, but Pitest has proven to have better performance, is easy to set up, and is actively supported."
},
{
"name": "Repsheet",
"description": "Attacks on web properties using bots are becoming more sophisticated. Identifying these bad actors and their behaviors is the goal of the <a href=\"http://getrepsheet.com/\"><strong>Repsheet</strong></a> project. It's a plugin for either Apache or NGINX that records user activity, fingerprints actors using predefined and user-defined rules, and then allows action to be taken, including the ability to block offensive actors. It includes a utility that visualizes current actors; this puts the ability to manage bot-based threats in the hands of team members, increasing security awareness for teams. We like this since it's a good example of a simple tool solving a very real but often invisible problem—bot-based attacks."
},
{
"name": "Scikit-learn",
"description": "<a href=\"http://scikit-learn.org/stable/\"><strong>Scikit-learn</strong></a> is an increasingly popular machine-learning library written in Python. It provides a robust set of machine-learning models such as clustering, classification, regression and dimensionality reduction, and a rich set of functionality for companion tasks like model selection, model evaluation and data preparation. Since it is designed to be simple, reusable in various contexts and well documented, we see this tool accessible even to nonexperts to explore the machine-learning space."
},
{
"name": "Jenkins as a deployment pipeline",
"description": "We know we're in perilous territory here, since we build a competing tool, but we feel we have to address a persistent problem. Continuous Integration tools like CruiseControl and Jenkins are valuable for software development, but as your build process gets more complex it requires something beyond just Continuous Integration: It requires a <a href=\"http://martinfowler.com/bliki/DeploymentPipeline.html\">deployment pipeline</a>. We frequently see people trying to use <strong>Jenkins as a Deployment Pipeline</strong> with the aid of plugins, but our experience is that these quickly become a tangle. Jenkins 2.0 introduces \"Pipeline as Code\" but continues to model pipelines using plugins and fails to change the core Jenkins product to model pipelines directly. In our experience, tools that are built around a first-class representation of deployment pipelines are much more suitable, and this is what drove us to replace CruiseControl with <a href=\"https://www.go.cd/\">GoCD</a>. Today we see several products that embrace deployment pipelines, including <a href=\"/radar/tools/concourse-ci\">ConcourseCI</a>, <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://spinnaker.io/\">Spinnaker,</a> <a href=\"https://github.com/drone\">Drone</a> and <a href=\"/radar/tools/gocd\">GoCD</a>."
}
]
},
{
"name": "hold",
"blips": [
{
"name": "Babel",
"description": "<a href=\"http://babeljs.io/\"><strong>Babel.js</strong></a> has become the default compiler for writing next-generation JavaScript. Its ecosystem is really taking off, thanks to its restructured <a href=\"http://babeljs.io/docs/plugins/#presets\">plugin system</a>. It allows developers to write <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> (and even ES7) code that runs in the browser or in the server without sacrificing backward compatibility for older browsers, and with very little configuration. It has first-class support for different build-and-test systems, which makes integration with any current workflow simple. It is a great piece of software that has become the main driver of ES6 (and ES7) adoption and innovation."
},
{
"name": "Consul",
"description": "We have moved <strong><a href=\"http://consul.io\">Consul</a></strong>, the service-discovery tool supporting both DNS- and HTTP-based discovery mechanisms, into Adopt. It goes beyond other discovery tools by providing customizable health checks for registered services, ensuring that unhealthy instances are marked accordingly. More tools have emerged to work with Consul to make it even more powerful. <a href=\"https://github.com/hashicorp/consul-template\">Consul Template</a> enables configuration files to be populated with information from Consul, making things like client-side load balancing using mod_proxy much easier. In the world of Docker, <a href=\"https://github.com/gliderlabs/registrator\">registrator</a> can automatically register Docker containers as they appear with Consul with extremely little effort, making it much easier to manage container-based setups. You should still think long and hard about whether you need a tool like this or whether something simpler will do, but if you decide you need service discovery, you won't go far wrong with Consul."
},
{
"name": "Grafana",
"description": "When combining modern techniques and architecture styles, such as <a href=\"/radar/techniques/microservices\">microservices</a>, <a href=\"/radar/techniques/devops\">DevOps</a> and <a href=\"/radar/techniques/qa-in-production\">QA in production</a>, development teams need increasingly sophisticated monitoring. Simply looking a graphs of disk usage and CPU utilization is not sufficient anymore, and many teams collect application and business-specific metrics using tools such a Graphite and Kibana. <strong><a href=\"http://grafana.org/\">Grafana</a></strong> makes it easy to create useful and elegant dashboards for data from a number of sources. A particularly useful feature allows timescales of different graphs to be synchronized, which helps with spotting correlations in the underlying data. The templating system that is being added shows a lot promise and will likely make managing sets of similar services even easier. Based on its strengths, Grafana has become our default choice in this category."
},
{
"name": "Packer",
"description": "Machine images have become a staple of modern deployment pipelines, and there are a number of tools and techniques to create the images. Because of its comprehensive feature set and the positive experiences we've had with it, we recommend <a href=\"http://packer.io\"><strong>Packer</strong></a> over the alternatives. We also recommend against trying to write custom scripts to do what Packer does out of the box."
},
{
"name": "Apache Kafka",
"description": "Many organizations are now looking closely at new data architectures that capture information as immutable sequences of events at scale. <a href=\"http://kafka.apache.org/\"><strong>Apache Kafka</strong></a> continues to build momentum as an open source messaging framework that provides a solution for publishing ordered event feeds to large numbers of independent, lightweight consumers. Configuring Kafka is nontrivial, but our teams are reporting positive experiences with the framework."
},
{
"name": "Espresso",
"description": "At the top of the testing pyramid for Android application development, our teams are increasingly using <strong><a href=\"https://google.github.io/android-testing-support-library/docs/espresso/index.html\">Espresso</a></strong> as the functional-testing tool. Its small-core API hides the messy implementation details and helps in writing concise tests, with faster and reliable test execution. With Espresso, you can run automated UI tests simulating user interactions within a single target app on both emulators and real devices across different Android versions."
},
{
"name": "fastlane",
"description": "<strong><a href=\"https://fastlane.tools/\">fastlane</a></strong> is our go-to tool for automating most of the boring activities involved in getting iOS and Android mobile apps built, tested, documented and provisioned. Simple configuration, a range of tooling and multiple pipelines make this a key ingredient in doing <a href=\"/radar/techniques/continuous-delivery-cd\">continuous delivery</a> for mobile."
},
{
"name": "Galen",
"description": "Testing that layout and styling of responsive websites is working as expected across various form factors can be a slow and often manual process. <strong><a href=\"http://galenframework.com/\">Galen</a></strong> helps ease this problem by providing a simple language, running on top of <a href=\"http://www.seleniumhq.org/\">Selenium</a>, that allows you to specify expectations for the appearance of your website in various screen sizes. Although Galen suffers from the typical brittleness and speed issues of any end-to-end testing approach, we have found benefit in the early feedback on design issues."
},
{
"name": "HashiCorp Vault",
"description": "Having a way to securely manage secrets is increasingly becoming a huge project issue. The old practice of keeping secrets in a file or in environment variables is becoming hard to manage, especially in environments with multiple applications and large numbers of <a href=\"/radar/techniques/microservices\">microservices</a>. <a href=\"https://github.com/hashicorp/vault\"><strong>HashiCorp Vault</strong></a> addresses the problem by providing mechanisms for securely accessing secrets through a unified interface. It has served us well on a number of projects, and our teams liked how easy it was to integrate Vault with their services. Storing and updating secrets is a bit cumbersome, because it relies on a command-line tool and a fair amount of discipline from the team."
},
{
"name": "JSONassert",
"description": ""
},
{
"name": "Let's Encrypt",
"description": "<a href=\"https://letsencrypt.org/\"><strong>Let's Encrypt</strong></a> first appeared on the Radar last edition, and since December 2015 this project has moved its beta status from private to public, meaning users will no longer be required to have an invitation in order to try it. Let's Encrypt grants access to a simpler mechanism to obtain and manage certificates for a larger set of users who are seeking a way to secure their websites. It also promotes a big step forward in terms of security and privacy. This trend has already begun within ThoughtWorks, and many of our projects now have certificates verified by Let's Encrypt."
},
{
"name": "Load Impact",
"description": "<strong><a href=\"https://loadimpact.com/\">Load Impact</a> </strong>is a SaaS load-testing tool that can generate highly realistic loads of up to 1.2 million concurrent users. Record and playback web interactions using a Chrome plugin simulate network connections for mobile or desktop users and generate load from up to 10 different locations around the world. While not the only on-demand load-testing tool we've used—we also like <a href=\"https://blazemeter.com/\">BlazeMeter</a>—our teams were very enthusiastic about Load Impact."
},
{
"name": "OWASP Dependency-Check",
"description": "In a world full of libraries and tools that simplify the life of many software developers, deficiencies in their security have become visible and have increased the vulnerability surface in the applications that use them. <a href=\"https://www.owasp.org/index.php/OWASP_Dependency_Check\"><strong>OWASP Dependency-Check</strong></a> automatically identifies potential security problems in the code, checking if there are any known publicly disclosed vulnerabilities, then using methods to constantly update the database of public vulnerabilities. Dependency-Check has some interfaces and plugins to automate this verification in Java and .NET (which we have used successfully) as well as Ruby, Node.js and Python."
},
{
"name": "Pa11y",
"description": "<a href=\"http://pa11y.org/\"><strong>Pa11y</strong></a> is an automatic accessibility tester that can run from the command line and be embedded into a build pipeline. Our teams have had success using Pa11y on a highly dynamic site by first creating a static HTML version, then running the accessibility tests against that. For many systems—especially government websites—accessibility testing is a requirement, and Pa11y makes it all a lot easier."
},
{
"name": "Serverspec",
"description": "In the past we have included automated <a href=\"/radar/techniques/provisioning-testing\">Provisioning Testing</a> as a recommended technique, and in this issue we highlight <a href=\"http://serverspec.org/\"><strong>Serverspec</strong></a> as a popular tool for implementing those tests. Although this tool is not new, we are seeing its use become more common as more cross-functional delivery teams take on responsibility for infrastructure provisioning. Serverspec is built on the Ruby library RSpec and comes with a comprehensive set of helpers for asserting that server configuration is correct."
},
{
"name": "Talisman",
"description": "With the maturity of tools such as <a href=\"/radar/tools/hashicorp-vault\">Vault</a>, there is no longer an excuse for storing secrets in code repositories, particularly since this often ends up being the soft underbelly of important systems. We’ve previously mentioned repository-scanning tools such as <a href=\"/radar/tools/gitrob\">Gitrob</a>, but we are now pushing proactive tools such as (the ThoughtWorks-created) <strong><a href=\"https://github.com/thoughtworks/talisman\">Talisman</a></strong>, which is a prepush hook for Git that scans commits for secrets matching predefined patterns."
},
{
"name": "Terraform",
"description": "With <strong><a href=\"https://www.terraform.io/\">Terraform</a></strong>, you can manage cloud infrastructure by writing declarative definitions. The configuration of the servers instantiated by Terraform is usually left to tools like Puppet, Chef or Ansible. We like Terraform because the syntax of its files is quite readable and because it supports a number of cloud providers while making no attempt to provide an artificial abstraction across those providers. Following our first, more cautious, mention of Terraform almost two years ago, it has seen continued development and has evolved into a stable product that has proven its value in our projects. The issue with state file management can now be sidestepped by using what Terraform calls a \"remote state backend.\" We’ve successfully used <a href=\"/radar/tools/consul\">Consul</a> for that purpose."
},
{
"name": "tmate",
"description": "Pair programming is an essential technique for us, and—given that we’re seeing more and more teams whose members are distributed across multiple locations—we have experimented with a number of tools to support remote pairing. We certainly liked <a href=\"https://screenhero.com/\">ScreenHero</a> but are concerned about its future. For teams that don’t rely on a graphical IDE, using <strong> <a href=\"https://tmate.io/\">tmate</a></strong> for pairing has turned out to be a great solution. tmate is a fork of the popular tmux tool, and compared to <a href=\"http://hamvocke.com/blog/remote-pair-programming-with-tmux/\">tmux for remote pairing</a>, the setup is much easier. Compared to graphical screen-sharing solutions, the bandwidth and resource requirements are modest, and it obviously never suffers from blurry screens. Teams can also set up their own server, thus retaining full control of the privacy and integrity of the solution."
},
{
"name": "Webpack",
"description": "<a href=\"http://webpack.github.io/\"><strong>Webpack</strong></a> has solidified itself as our go-to JavaScript module bundler. With its ever-growing <a href=\"https://github.com/webpack/docs/wiki/list-of-loaders\">list of loaders</a>, it provides a single dependency tree for all your static assets, allowing flexible manipulation of JavaScript, CSS, etc. and minimizing what needs to be sent to the browser and when. Of particular relevance is the smooth integration among AMD, CommonJS and <a href=\"/radar/languages-and-frameworks/es6\">ES6</a> modules and how it has enabled teams to work in ES6 and seamlessly transpile (using <a href=\"http://babeljs.io/\">Babel</a>) to earlier versions for browser compatibility. Many of our teams also value <a href=\"http://browserify.org/\">Browserify</a>, which covers a similar space but is more focused on making Node.js modules available for client-side use."
},
{
"name": "Zipkin",
"description": "Development on <a href=\"https://github.com/openzipkin/zipkin\"><strong>Zipkin</strong></a> has continued apace, and since the middle of 2015 it has moved to the <em>openzipkin/zipkin</em> organization at GitHub. There are now bindings for Python, Go, Java, Ruby, Scala and C#; and there are Docker images available for those wanting to get started quickly. We still like this tool. There is an active and growing community around usage of it, and implementation is getting easier. If you need a way of measuring the end-to-end latency of many logical requests,&nbsp;Zipkin continues to be a strong choice."
},
{
"name": "Android-x86",
"description": "<strong><a href=\"http://www.android-x86.org/\">Android-x86</a></strong> is a port of the <a href=\"http://source.android.com/\">Android open source</a> project to x86 platforms. The project started by hosting various patches from the community for x86 support but then later created its own codebase to provide support for different x86 platforms. We have seen significant time savings by utilizing Android-x86 in our CI servers instead of emulators for hermetic UI testing. However, for UI-specific tests targeting a particular device resolution—simulating low memory, bandwidth and battery—it is better to stick with emulators."
},
{
"name": "axios",
"description": "Our teams have had success with <a href=\"https://github.com/mzabriskie/axios\"><strong>axios</strong></a>, a promises-based HTTP client in JavaScript that they describe as \"better than <a href=\"/radar/languages-and-frameworks/fetch\">Fetch</a>.\" The project has lots of endorsements and activity on GitHub, and it gets a thumbs-up from us."
},
{
"name": "Bottled Water",
"description": "With the growth of interest in streaming data architectures and the downstream data lakes they feed, we have seen an increased reliance on \"change data capture\" tooling to connect transactional data stores to stream-processing systems. <strong><a href=\"https://github.com/confluentinc/bottledwater-pg\">Bottled Water</a></strong> is a welcome addition to this field, converting changes in <a href=\"/radar/platforms/postgresql-for-nosql\">PostgreSQL</a>’s write-ahead log into <a href=\"/radar/tools/apache-kafka\">Kafka</a> events. One downside of this approach, however, is that you are tied to low-level database events rather than the higher-level <a href=\"/radar/techniques/capture-domain-events-explicitly\">business events</a> we recommend as the foundation for an event-oriented architecture."
},
{
"name": "Clojure.spec",
"description": "One of those perpetual developer debates involves language typing: How much is just right? <a href=\"/radar/languages-and-frameworks/clojure\">Clojure</a>, the dynamically typed functional Lisp on the JVM, added a new entry into this discussion that blurs the lines. <strong><a href=\"https://clojure.org/about/spec\">Clojure.spec</a></strong> is a new facility built into Clojure that allows developers to wrap type and other verification criteria around data structures, such as allowable value ranges. Once they are established, Clojure uses these specifications to provide a slew of benefits: generated tests, validation, destructuring of data structures and others. Clojure.spec is a promising way to have the benefits of types and ranges where developers need them but not everywhere."
},
{
"name": "FBSnapshotTestcase",
"description": "Testing the visual portion of iOS applications can be painful, slow and flakey, which is why we’re happy to include <strong><a href=\"https://github.com/facebook/ios-snapshot-test-case\">FBSnapshotTestcase</a></strong> in our toolkit. It automates taking, storing and diff-ing snapshots of UI components so you can keep your interfaces pixel-perfect. Since it runs as a unit test (in the simulator), it is faster and more reliable than functional-testing approaches."
},
{
"name": "Grasp",
"description": "We had our collective minds blown by a little JavaScript command-line refactoring tool called <strong><a href=\"http://www.graspjs.com/\">Grasp</a></strong>. Providing a rich set of selectors and operating against the abstract syntax tree, it is leagues ahead of fiddling with sed and grep. A useful addition to the toolkit in our ongoing quest to treat <a href=\"/radar/languages-and-frameworks/javascript-as-a-first-class-language\">JavaScript as a first-class language</a>."
},
{
"name": "LambdaCD",
"description": "<a href=\"http://www.lambda.cd\"><strong>LambdaCD</strong></a> provides teams with a way to define Continuous Delivery pipelines in Clojure. This brings the benefits of <a href=\"/radar/tools/infrastructure-as-code\">Infrastructure as code</a> to the configuration of CD servers: source-control management, unit testing, refactoring and code reuse. In the \"pipelines as code\" space, LambdaCD stands out for being lightweight, self-contained and fully programmable, allowing teams to work with their pipelines in the same way that they do with their code."
},
{
"name": "Pinpoint",
"description": "Teams using the Phoenix Server or <a href=\"/radar/techniques/phoenix-environments\">Phoenix Environment </a> techniques have found little in the way of support from Application Performance Management (APM) tools. Their licensing models, based on long-running, limited amounts of tin, and their difficulty in dealing with ephemeral hardware, have meant that they are often more trouble than they are worth. However, distributed systems need monitoring, and at some point many teams recognize the need for an APM tool. We think <strong><a href=\"https://github.com/naver/pinpoint\">Pinpoint</a></strong>, an open source tool in this space, is worth investigating as an alternative to AppDynamics and Dynatrace. Pinpoint is written in Java, with plugins available for many servers, databases and frameworks. While we think you can go a long way using a combination of other lightweight open source tools—<a href=\"/radar/tools/zipkin\">Zipkin</a>, for example—if you are in the market for an APM, Pinpoint is worth considering."
},
{
"name": "Pitest",
"description": "<a href=\"http://pitest.org\"><strong>Pitest</strong></a> is a test coverage analysis tool for Java that uses a mutation-testing technique. Traditional test coverage analysis tends to measure the number of lines that are executed by your tests. It is therefore only able to identify code that is definitely not tested. Mutation testing, on the other hand, tries to test the quality of those lines that are executed by your test code and yet might contain general errors. Several problems can be spotted this way, helping the team to measure and grow a healthy test suite. Most of such tools tend to be slow and difficult to use, but Pitest has proven to have better performance, is easy to set up, and is actively supported."
},
{
"name": "Repsheet",
"description": "Attacks on web properties using bots are becoming more sophisticated. Identifying these bad actors and their behaviors is the goal of the <a href=\"http://getrepsheet.com/\"><strong>Repsheet</strong></a> project. It's a plugin for either Apache or NGINX that records user activity, fingerprints actors using predefined and user-defined rules, and then allows action to be taken, including the ability to block offensive actors. It includes a utility that visualizes current actors; this puts the ability to manage bot-based threats in the hands of team members, increasing security awareness for teams. We like this since it's a good example of a simple tool solving a very real but often invisible problem—bot-based attacks."
},
{
"name": "Scikit-learn",
"description": "<a href=\"http://scikit-learn.org/stable/\"><strong>Scikit-learn</strong></a> is an increasingly popular machine-learning library written in Python. It provides a robust set of machine-learning models such as clustering, classification, regression and dimensionality reduction, and a rich set of functionality for companion tasks like model selection, model evaluation and data preparation. Since it is designed to be simple, reusable in various contexts and well documented, we see this tool accessible even to nonexperts to explore the machine-learning space."
},
{
"name": "Jenkins as a deployment pipeline",
"description": "We know we're in perilous territory here, since we build a competing tool, but we feel we have to address a persistent problem. Continuous Integration tools like CruiseControl and Jenkins are valuable for software development, but as your build process gets more complex it requires something beyond just Continuous Integration: It requires a <a href=\"http://martinfowler.com/bliki/DeploymentPipeline.html\">deployment pipeline</a>. We frequently see people trying to use <strong>Jenkins as a Deployment Pipeline</strong> with the aid of plugins, but our experience is that these quickly become a tangle. Jenkins 2.0 introduces \"Pipeline as Code\" but continues to model pipelines using plugins and fails to change the core Jenkins product to model pipelines directly. In our experience, tools that are built around a first-class representation of deployment pipelines are much more suitable, and this is what drove us to replace CruiseControl with <a href=\"https://www.go.cd/\">GoCD</a>. Today we see several products that embrace deployment pipelines, including <a href=\"/radar/tools/concourse-ci\">ConcourseCI</a>, <a href=\"/radar/tools/lambdacd\">LambdaCD</a>, <a href=\"http://spinnaker.io/\">Spinnaker,</a> <a href=\"https://github.com/drone\">Drone</a> and <a href=\"/radar/tools/gocd\">GoCD</a>."
}
]
}
]
},
}
}
}
function ringNodeToBlipNodes(ringNode){
return ringNode.parentNode.parentNode.querySelectorAll('ul>li');
}
function blipNodeToDetails(blipNode){
var name = blipNode.querySelector('.blip-name').innerHTML;
var descNode = blipNode.querySelector('.blip-description p');
var description = descNode ? descNode.innerHTML : '';
return { name : name, description : description}
}
function getRingNodes(){
return document.querySelectorAll('.ring');
}
function nodesToArray(nodes){
return Array.prototype.slice.call(nodes);
}
JSON.stringify(nodesToArray(getRingNodes()).map(ringNode => {
var name = ringNode.classList[1];
var blips = nodesToArray(ringNodeToBlipNodes(ringNode)).map(blipNodeToDetails)
return {
name : name,
blips : blips
}
}), null, 2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment