karanth

SSL is an important security and privacy feature for all websites. Its details are outlined in this wikipedia [article] ("http://en.wikipedia.org/wiki/Secure_Sockets_Layer"). At Scibler, we use SSL certificates, encrypting all traffic to and fro from our servers. SSL is a public-key based asymmetric encryption scheme for symmetric key exchange. Symmetric keys are used for payload encryption. On our servers, we use embedded jetty (ring jetty adapter), with the clojure [ring] (https://github.com/ring-clojure) library to handle the http specific functionality.

This is a tutorial about installing SSL certificates on jetty webservers. SSL certificates are X.509 certificates that can be self-signed (authorized by Scibler) or can be signed by trusted third-parties. Trusted third-party certificates are the ones that a Internet user and browsers trust the most. Trusted third-party certification authorities issue certificates per domain and charge a nominal yearly fee.

####Pre-Requisites

• The Java JDK has to be

karanth

IMAP is a popular mail access protocol. Mail clients traditionally work on a polling model, i.e., they request the server for new messages and updates at fixed intervals. These are not "Real"-time updates as server updates are brought down to the client only at the end of each interval. The average time taken for an email to be seen by the client is half the size of the polling interval and not immediate.

However, IMAP4 has the IDLE command to help clients get updates instantly in "real"-time. RFC 2177 refers to the IDLE command and can be found [here] (https://tools.ietf.org/html/rfc2177). Executing the command has the following pre-requisites,

• A (TCP socket) network connection is open between the client and the server.
• The client is authenticated with the server using LOGIN or AUTHENTICATE.
• The client has issued a SELECT (read/write) or the EXAMINE (read-only) command on a particular mailbox.
• The server lists IDLE as a supported capability when the CAPABILITY command is issued.

Wh

karanth

SASL - Simple Authentication and Security Layer is NOT a security protocol. It is an interface or an abstraction providing pluggable authentication mechanisms to connection-oriented protocols. For example, the SASL interface in a system supporting the LDAP protocol may support Kerberos-based authentication though traditionally LDAP does not support Kerberos-based authentication or security. Another example is that SASL built into an IMAP server may allow Oauth2-based authentication or any new authentication mechanism to be plugged in, though traditionally an IMAP server may only be supporting password-based authentication.

From now on, mechanisms refer to authentication mechanisms like Kerberos-based authentication, OAuth2-based authentication or even plain text authentication and protocols refer to connection-oriented protocols like LDAP, IMAP, SMTP, etc.

The mechanisms are grouped and named. For example, Kerberos-based authentication falls under a mechanism name called GSSAPI.

How a _pr

karanth

There are many situations on the browser client when it would be desirable to make an AJAX call to a web-site or web service to fetch data. Many a time, the data source is on a different domain than the one from where the request is originating. These are called cross-domain requests. All browsers discourage scripts from making cross-domain requests as they are wary of the security implications in allowing arbitrary data requests across trust boundaries. At Scibler, the browser plugin that gets activated on the google mail domain needs to request data from Scibler servers to create an integrated experience for a user.

Partial mitigation of 2 important attacks is a reason why browsers advocate the Same Origin Policy (SOP) for data requests. One is called XSRF or CSRF (pronounced sea-surf) and the other XSS.

[XSRF] (http://en.wikipedia.org/wiki/Cross-site_request_forgery) stands for Cross-site Request Forgery, where in, trust on a user's browser that is providing a session with a particular site is hi

karanth

The IMAP protocol workflow consists of the following steps,

• A network connection established between the client and the server.
• A greeting message sent by the server indicating that the client has successfully connected.
• A series of interactions between the client and server.

The interactions consists of strings of lines, i.e. string terminated by a carriage return and a line feed (CRLF or \r\n). Interactions can be both commands (sent by clients) and data (sent by clients and servers). Both the client and the server strictly interact using lines or known length octet streams (8-bit characters) followed by a line.

####Client

An IMAP client issues commands to the server in a CRLF terminated string. The syntax of a command includes a tag, followed by the command and parameters. A tag is an alphanumeric identifier and each client command has a different tag for that session. A tag could be something like but not limited to A1, A2 etc.

karanth

A free book on Distributed Systems available at http://book.mixu.net/distsys/index.html

Chapter 1: Covers the basics of Distributed systems including the need for such systems and the definitions of terms that describe such systems. Terms such as Scalability, Fault Tolerance, Replication, Latency, Performance and Partitioning are defined in this chapter. In my mind, the first reason for using Replication in any system was to provide redundancy against data loss. This chapter sheds light on another subtle reason, caching. We replicate for performance by putting disk data in memory, memory data in the processor cache etc. For lower latency, we replicate data in different geographies using CDNs.

karanth

Installing a single node hadoop cluster is not a straight forward task. It involves a bunch of different things like creating users and groups to enabling password-less ssh. Thanks to virtualization technology and hortonworks' pre-configured OS images with Hadoop and a few of its ecosystem components, the task has been greatly simplified. Though this does not enable a first time Hadoop user to learn about the system level Hadoop complexities, it simplifies administration and deployment. The user can now focus on data management and analysis.

Downloads

The 2.4GB image for the Hortonworks Hadoop sandbox can be downloaded from [here] (http://hortonworks.com/products/hortonworks-sandbox/#install). I have chosen Oracle's VirtualBox as the virtualization technology. It can be downloaded from [here] (https://www.virtualbox.org/wiki/Downloads)

Configuration

I have tried installing VirtualBox on my Windows 8 PC, that has 4GB of RAM. The documentation clearly states that if Ambari and/or HBase have to b

karanth

####Basics####

• Memory Hierarchy - Tape, Disk, SSD, Memory, Cache
• Kryder's law
• Long Tail vs 80/20 rule
• Drawbacks of monolithic systems - Supercomputers
• Distributed Systems - Advantages & Problems
• CAP theorem - Consistency, Availability and Partition Tolerance
• PACELC - if(Partition){ Tradoffs: Consistency vs Availability } else { Tradeoffs: Consistency vs Latency }
• Concurrency vs Parallelism
• Parallel vs Distributed computing

karanth

WSDL is an XML document that describes the contract a Web Service provides to its clients. The best way to explain a WSDL is by looking at XSD, or rules that govern the WSDL XML document. An example WSDL from the web is broken down and each element type is visited. The version of the WSDL below is 1.1. There is a newer 2.0 version of the WSDL XSD that has streamlined a few elements. However, Exchange Web Services still seem to be using 1.1. This particular Web Service gives stock quotes based on symbols.

<wsdl:definitions xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"       
                  xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/"
                  xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" 
                  xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" 
                  xmlns:tns="http://www.webserviceX.NET/" 
                  xmlns:s="http://www.w3.org/2001/XMLSchema" 

xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soa

karanth

A previous [gist] (https://gist.github.com/karanth/8467944#file-jsonp-cors-pub-md) had notes about JSONP and CORS as methods for making cross-domain requests. After consideration, CORS was a secure and a better alternative. However, implementing CORS is not as straightforward as introducing a few headers on the server response.

There CORS W3C spec identifies cases where a pre-flight request is called for in a CORS situation. A pre-flight request is a client-iniated request using the OPTIONS HTTP verb that tries to understand the capabilities of the server. Only on a successful response (200) to the pre-flight request, and presence of certain headers in the server response, will the client initiate the actual CORS request.

A little bit of nosing around with the rapportive plugin revealed a CORS situation, that forces the client to execute the special pre-flight request. Rapportive is a neat browser plugin that gives details about a contact (by looking into the from field of the email) whe