Knock Junior Data Engineer Code Challenge

challenge.md

Data Engineer Challenge

A common task we deal with at Knock is mapping data from a variety of real estate systems into our own internal formats. Real-world data is often messy, and doesn't always conform exactly with what we want. We often have to make best-effort decisions as to how data maps onto our own formats.

Your challenge is to map an input real estate agent to the prescribed output format, as shown below.

// Output format (Go struct example):
type Agent struct {
	FirstName        string   `json:"first_name"`
	LastName         string   `json:"last_name"`
	PhoneNumbers     []string `json:"phone_numbers"`
	MLSID            string   `json:"mls_id"`
	AddressLine1     string   `json:"address_line_1"`
	AddressLine2     string   `json:"address_line_2"`
	Email            string   `json:"email"`
	OfficeID         string   `json:"office_id"`
	BoardMemberships []string `json:"board_memberships"`
}

Example output JSON:
{
  "first_name": "Steve",
  "last_name": "Smith",
  "phone_numbers": [
    "404-123-4567",
    "404-867-5309"
  ],
  "mls_id": "A1234",
  "address_line_1": "123 Main St",
  "address_line_2": "Atlanta, GA 30332",
  "email": "steve@knock.com",
  "office_id": "O5678",
  "board_memberships": [
      "Atlanta Board of REALTORS",
      "Georgia Association of Real Estate Professionals"
  ]
}

Sample input:
{
    "AgentFirstName": "Steve",
    "AgentLastName": "Smith",
    "CellPhone": "404-123-4567",
    "DirectPhone": "404-867-5309",
    "Agent_MLS_Id": "A1234",
    "StreetAddress": "123 Main St",
    "StreetCity": "Atlanta",
    "StreetState": "GA",
    "StreetZipOrPostalCode": "30332",
    "AgentEmail": "steve@knock.com",
    "Office_Id": "O5678",
    "AgentBoardMemberships": "Atlanta Board of REALTORS,Georgia Association of Real Estate Professionals"
}

Write a program that reads the example input JSON file, maps it to the output structure as defined above, and writes it to an output file. You may write the code in Scala, Python, Go, or Java. You should explain how to run the code in the README, as well as provide tests (and the instructions to run them) to ensure your code works as expected.

Implementation notes

• Values that are not present in the input should output as null in JSON
• If an output array contains no elements, the output value should be null.
• Any empty string value should be null in the output.
• Do not assume that all values will always be present. Any field in the input may be null. Your code should handle this gracefully.
• Don't overthink it!

input.json

{
    "AgentFirstName": "Steve",
    "AgentLastName": "Smith",
    "CellPhone": "404-123-4567",
    "DirectPhone": "404-867-5309",
    "Agent_MLS_Id": "A1234",
    "StreetAddress": "123 Main St",
    "StreetCity": "Atlanta",
    "StreetState": "GA",
    "StreetZipOrPostalCode": "30332",
    "AgentEmail": "steve@knock.com",
    "Office_Id": "O5678",
    "AgentBoardMemberships": "Atlanta Board of REALTORS,Georgia Association of Real Estate Professionals"
}