$http and request param serialisation issues

gistfile1.md

Issues description

"Hard-coded" serialization format for request parameters

The current version of the $http service has a hard-coded way of serializing request parameters. This one-and-only-one way of doing things causes practical problems to people using backends that have different serialziation schemas (mostly Rails and PHP).

More specifically, given this $http call:

$http.get('http://google.com', {params: {foo: [1 ,2], bar: 'sth;else'}});

a URL produced by the default serialization schema will be:

http://google.com?foo=1&foo=2&bar=sth;else

This is unfortunatelly not the format expected by some (many?) people:

• PHP / Rails users would expect an array to be serialised to request params as: foo[]=1&foo[]=2&bar=sth;else
• there is a question of special chars encoding - in this particular case ; is expected to be percent-encoded by some backends

More generally, users might use backends that use a completly different serialisation schema, ex.:

http://google.com;foo=[1,2];bar=sth%3Belse

This per-backend variability of the URL serialization schema indicates that a URL building mechanism used by AngularJS should be configurable / overridable for users, instead of being fixed deep inside the framework.

Testing considerations

As of today people are using $httpBackend mock to test $http-based code. The problem here is that while writing they functional code people operate on a different abstraction level as compared to tests (that are expressed in terms of backend interactions and not higher-level $http interactions). One practical consequence of this approach is that people needs to re-construct a URL in their tests. In other words they need to duplicate logic burried deep-inside $http.

Issues - summary

Based on the above description we've got 2 main problems to solve:

• make URL-construction mechanism configurable to:
  • allow custom, per-backend serialization schemas - issues: #3740, #7429
  • allow users to deal with certain back-end quirks (ex.: %-encoding) - issues: #9224
• make it easier for people to generate URLs in their tests - issues: #3311

Proposed solution(s)

The general idea is to extract the logic of the buildUrl to a separate service ($$httpUrl? $urlBuilder?) thus allowing:

• easy customisation of the serialisation schema (just swap the default service)
• community to build custom serialisation schemas dedicated to a given backend type
• usage of a new service from the unit tests to prepare string representation of a URL the same way as $http would do
• easier unit-testing of the serialization service itself (both custom implementations from the community as well as built-in one)

After doing so we would introduce a new configuration param to the config object (paramsSerializer? urlBuilder?) that, by default, would invoke current logic from the extracted service. Adding a new config param would allow us to handle a use-case where users are hitting different backends with different serialization schemas.

Prior art

• $http's serialization schema as of version 1.3.8
• jQuery's param method: http://api.jquery.com/jquery.param/

So the registration / usage could look something like this?

//Register custom urlBuilder
myModule.config(function($httpUrlBuilderProvider) {
  $httpUrlBuilderProvider.addBuilder('myBuilder', myBuilder);
});

myApp.config(function($httpProvider) {
  //Set the default builder
  $httpProvider.setUrlBuilderDefault('myBuilder');
});

//Use it
myApp.controller(function($http) {
  //Use default builder
  $http({
    url: '...',
  });

  //Use other builder
  $http({
    url: '...',
    urlBuilder: 'myOtherBuilder'
  });

  //Use default builder?
  $http({
    url: '...',
    urlBuilder: undefined
  });
});

How would the unit tests know about the custom urlBilder?

@Narretz it is not my intention to have per-$http call to a URL serializer and as such a particular serialization strategy would not be part of $http call config. Rather it would be on a service-level (so for all the $http calls). If anyone needs to have different serialization modes they can manually stringify a URL or move this logic to an interceptor, as today.

In terms of code it would be:

//configure serialization schema for the whole application:

myModule.config(function($urlParamsSerializerProvider) {
  $urlParamsSerializerProvider.addArrayMarkers = true; //false by default, behaves as today
});

Then $http would be injected with an instance of $urlParamsSerializer - that's it. In your tests you could inject the same service.

And if you are after a custom serialization schema you would simply override the default $urlParamsSerializer service

@pawel - the "proposed solution" section above does suggest an opportunity to configure per $http call.

After doing so we would introduce a new configuration param to the config object (paramsSerializer? urlBuilder?) that, by default, would invoke current logic from the extracted service. Adding a new config param would allow us to handle a use-case where users are hitting different backends with different serialization schemas.

It would seem that this would be in keeping with the current arrangement, where we can set the default config service wide and then on the config of the call itself.

@petebacondarwin - oh, right - this was my idea at one point but I think it complexifies things for not much benefit...

If you have different API end points customzing the serializer once doesn't really help you. Interceptors are also not very intuitive as you have to filter the requests you want to modify. I say, if we make this flexible, let's go all the way.
Do you mean it will make the code more compley or the API?

i don't think we can get by with locking people into 1 serialization per app.

It should be more like this:

module.config($urlBuilderProvider) {
  $urlBuilderProvider.querySerializer('type2', function(paramsObject) {
    // do custom param building
  });
});

$http({
  method: 'GET',
  params: {
    // ... query params
  },
  queryMode: 'jquery'
}).then(...);

By default, this should ship with 'default', and 'jquery' and 'legacy' (or maybe just 2 out of 3 of those). easy to use, easy to extend, allows for good errors.

@catip your proposal is interesting, we just need to make sure that it also helps with testing. So I guess this could work if we have individual functions for each type exposed in DI as well - which could be easily done. Let me sleep on your proposal.

I like @catip's solution as well.

When I see @catip's solution, I get worry about the consistency of the AngularJS ecosystem. My vision is to adopt a universal way while maintaining enough options available to satisfy the different approaches.

I'll prefer add an option to the $http config like { array_strict_mode: true }.

PS : And make transformRequest available for $resource's GET query : angular/angular.js#11277