Skip to content

Instantly share code, notes, and snippets.

@danbev
Created Nov 26, 2012
Embed
What would you like to do?
Paging/Querying in AeroGear Controller

Paging/Querying in AeroGear Controller

This document is intended to discuss a solution for AEROGEAR-645 "Add Query/Paging support on RESTful endpoints".
This dev-aerogear mail list thread was used as a starting point to gather requirements and ideas.

Background

Lets just clarify the two concepts that we are dealing with here, Paging and Query.

Paging

Paging deals with limiting the number of results returned from a single request. One "page" can contain a certain number of items as the result of invoking a request, sometimes referred to the size of a page. Let say you want to limit you the number of result returned to 10 items, this could look something like this:

/cars/?limit=10

So, this would simply say we want to limit the result to 10 items, but it does not say which ten items to return. The first ten, the last ten, what gives?
Well there is another query parameter needed for this which is the offset:

/cars/?limit=10&offset=3

So, this is saying that we have a limit of 10, and an offset of 3 which would return the items in the range 30-40.
Another variant, which is used by GitHub, is:

/cars?per_page=10&page=3

The principal is the same, we are saying give us 10 items per page, and start from page 3.

Linking

Related to paging is also linking which is how a caller can navigate using links. This was also suggested in the dev-list thread which suggested at looking into how GitHub handles pagination. GitHub use Web Linking so that clients can determine next, previous, first, last, using the Http Response header "Link"?
Lets take a look at how this works:

curl -i "https://api.github.com/users/danbev/gists?page=1&per_page=1"

First lets take a look at the Link response header:

Link: <https://api.github.com/users/danbev/gists?page=2&per_page=1>; rel="next", 
      <https://api.github.com/users/danbev/gists?page=1000000&per_page=1>; rel="last"

The actual content is a javascript array containing a single entry with info about the first gist. Now, by using the next link header we can get the second item and so on. We could have specified a different value for the per_page but this was done so that it is easier to work with while testing.

Another option which was brought up in the dev-list mailing list thread is that of returning metadata to the caller in the body of the response. This would be returned by default or perhaps as an extra query parameters, for example named metadata or something like that. For example:

{
    "meta_data": {"total":100,"limit":10,"offset":3},
    "links" : {"next": "/cars?per_page=10&page=3", "previous": "...", "first": "...", "last": "..."},
    [
      // array or cars 
    ]
}

Additional information could be added, for example a total number of elements available. But one must also consider the performance impact of the information made available, for example it might be expensive to calculate the total number.

  • Which solution is preferred, using the Link header or using metadata in the response?
  • If metadata in the response is chosen should this be the default?

Query

Querying is concerned only with selecting what items to be returned and does not have anything to do with paging. You could very well use paging without a query. But you might only be interested in a subset of items, for example only a certain model of a car. For example:

/cars/?where={"brand":"BMW"} 

Usage in AeroGear Controller

  • How will paging and querying be used in AeroGear Controller?
    AeroGear Controller enables us to declare MVC routes where endpoints are simple POJOs. At the moment there is only support for routes that forward to a view and don't return anything to the calling client, well they actually do return the view to the client but not raw data like a JSON array of items for example. But RESTFul POJO endpoints are not supported yet.

  • Since AeroGear controller does not currently handle RESTful services neither paging of querying makes sense, which is leaving me a little puzzled as to what the task at hand should actually do?

@sebastienblanc

This comment has been minimized.

Copy link

@sebastienblanc sebastienblanc commented Nov 27, 2012

Some general remarks :

  • Is paging really in the scope of Aerogear ? There is a big chance that Aerogear will be used with other front end frameworks which has their own built-in paging mechanism, same thing on the server side, it could conflict ...
  • Query, as mentioned in the gist, is a totally different topic and IMHO a way more complex, what about to separate (different JIRA / Gist ) these 2 concepts ?
@sebastienblanc

This comment has been minimized.

Copy link

@sebastienblanc sebastienblanc commented Nov 27, 2012

Forget about the first remark, as the scope is Aerogear Controller ;) but the second remark is still valid

@danbev

This comment has been minimized.

Copy link
Owner Author

@danbev danbev commented Nov 27, 2012

Yeah, I agree with separating these issues. I'll do that after the meeting this afternoon as long as there are no objections.

@secondsun

This comment has been minimized.

Copy link

@secondsun secondsun commented Dec 4, 2012

Wearing his Android hat

For paging I would tend to favor headers over metadata in the body. If we put paging metadata in our response, then we would have to change the Android RestAdapter to detect the metadata and return only the data objects we are requesting. This isn't a big deal, but it would be easier to have some Object which can sniff the response headers and keep its own state up to date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment