Spreedly's main line of business is transacting with stored credit cards at other gateways and endpoints - which are all themselves some sort of remote service. As such, it is important that engineers at Spreedly understand the realities of dealing with remote services and distributed systems, in general. This work sample is designed to understand your comfort dealing with such systems.
We've deployed a server that you can start jobs on. What do the jobs do? Doesn't really matter. What is important is that the jobs will always take ~0.5 seconds to complete. You can queue up a job like this:
$ curl -H "Content-Type: application/json" -d '{
"account": "your email address",
"wait": true
}' http://jobs.asgateway.com/start
Since wait is true, the server will just hang until the job is complete, at which time it will return this:
{
"id":"4a40f6c1bf585ca9",
"state":"completed",
"startedAt":"2015-06-29T15:28:55.497Z",
"proof":"E850075A4063332DE3D1C254E2A255424361DBDC"
}
We've provided a jobclient you can run on your local box against the job server:
$ jobclient --account 'example'
50 / 50 [=========================================================] 100.00 % 23s
Succeeded: 50
Average Duration: 11.368634862 seconds
Total Duration: 23.49453871 seconds
WORK SAMPLE RESULT: FAILED
Uh oh - the jobclient couldn't complete successfully. What's going on? Well, the jobclient runs 50 jobs, and all of those jobs need to both succeed, and they all need to complete within 5 seconds. If any jobs return an error, or it takes the jobclient more than 5 seconds to run all the jobs, it will fail.
Why can't it complete them in time? Well, it turns out our little job server can run a lot of jobs simultaneously. But what it can't do is handle a lot of connections simultaneously. Actually, it turns out it can only handle one connection at a time per account! All connection attempts after the first one will queue up, waiting for the precious single handler.
But the job server provides a way around this - go asynchronous with a callback! If you set wait to false, and pass a callback url, the job server will immediately return a response:
$ curl -H "Content-Type: application/json" -d '{
"account": "example",
"wait": false,
"callback": "http://requestb.in/1573xwm1"
}' http://jobs.asgateway.com/start
Returns:
{
"state":"running",
"id":"4499a6e0c2ef4b98"
}
And then, later, it will POST to the callback url you supplied with a request like this:
POST http://requestb.in/1573xwm1
Content-Type: application/json
{
"id":"4499a6e0c2ef4b98",
"state":"completed",
"startedAt":"2015-06-29T15:34:16.850Z",
"proof":"078DC91F2980650231E94E67777078D5C926B9A3"
}
This sounds great, so what's the problem? Well, the jobclient which we've supplied and that verifies successful completion of the sample, doesn't know how to interact with the asynchronous API. It can open up a lot of connections, but since it always specifies wait: true, it'll quickly consume the available connection and spend all its time waiting for replies.
So your task is simple: write something that sits in-between the supplied client and the server, which lets the client hold open a lot of connections, and turns the synchronous client requests into asynchronous requests to the server. Note that you can pass a url into the jobclient to point it at your middleware:
$ jobclient --url http://jobs.asgateway.com/proxy --account example
50 / 50 [==========================================================] 100.00 % 1s
Succeeded: 50
Average Duration: 1.384823579 seconds
Total Duration: 1.636669279 seconds
WORK SAMPLE RESULT: PASSED
Proof (supply to Spreedly):
H4sIAAAJbogC/4RXOY4kyY69S8m/B0bSjEtrtp5gpGm04Ov9jzAvwhNfi0glqwoFPGfS3sZ...//+qbRsKqEQAA
Here are the constraints you should keep in mind when developing your async proxy:
- Because we need to be able to effectively assess your approach, please use one of these languages to build your proxy:
- Ruby
- Elixir
- JavaScript
- C#
- Java
- Python
- If these languages aren't in your wheelhouse, feel free to reach out to us to discuss if there's another language with which we are mutually familiar. We aren't trying to assess your ability to pick up a new language.
- The app itself should run as a single OS level process. Running it within an app server is fine, but don't rely on the app server to provide the requisite concurrency (e.g., no use of app server threading, processes or other). The concurrency must be provided and managed by the app itself.
- It should process all jobclient requests in under 5 seconds
- Spreedly does not need to be able to run your code (the proof from the jobclient serves as evidence of successful functioning). So don't spend any time worrying about helping us set up the appropriate environment.
- Write your code like you would a prototype that could end up being deployed into production in the future: straightforward, lightweight, but with an eye to good coding practice that will allow it to be improved further.
- In addition to functioning code, please also write a test suite to accompany the application.
Ready to go? We've built a little site for this sample, which has everything you should need - have at it!
As you can see above, when the client is able to complete enough jobs within the time limit, it will spit out a "proof". Once you have a successful proof, you should validate it using our online tool, which verifies proper functioning and performance of your proxy.
Once your app has been validated, submit the following to Spreedly via email:
- The email address used as the async proxy account
Also provide a link to a secret gist containing:
- The contents of the proof block.
- The command you used to start the server (we won't run it, but want to see the server options required for your solution to work).
- The code/app you wrote to get a successful proof. Please make sure all files have been scrubbed of any personal information (so we can blindly evaluate the submission). Also, please include the raw files themselves, not a zip of all the files.
- The terminal output from running your final test suite.
- Please provide a single markdown file with appropriate formatting discussing the following, in the context of your chosen implementation:
- What class of errors or conditions from the job server does your proxy service not account for?
- How would you update the proxy service in future development iterations to account for these error conditions? Not looking for a code or language-specific answer here - just talk in general concepts and patterns to implement for each error condition.
- If the requirements for the proxy service, still running as a single process on a single node, were increased to handle 10X, 100X, 1000X, etc... concurrent requests, what limitations would the current implementation hit? Assume the job server itself will never be the limiting factor.
- If your implementation graduates from the prototype phase and needs to be deployed into production, what changes will need to be made to support running as a distributed service across several disparate nodes? Imagine five instances of your proxy service running, each on one of the five different nodes - what changes will you have to make to your app to support this mode of operation?
- How can we improve this work sample?
Since this work sample does not need to be running somewhere accessible, you may find a tool like ngrok useful to provide a public URL loopback to localhost.
Version: 2.5 - 2019-02-22