jq_cfdot_tutorial.md

Intro to JSON?

• A JSON string contains either an array of values, or an object (an associative array of name/value pairs).

• An array is surrounded by square brackets, [ and ], and contains a comma-separated list of values.

• An object is surrounded by curly brackets, { and }, and contains a comma-separated list of name/value pairs.

• A name/value pair consists of a field name (in double quotes), followed by a colon (:), followed by the field value.

• A value in an array or object can be:

  • A number (integer or floating point)
  • A string (in double quotes)
  • A boolean (true or false)
  • Another array (surrounded by square brackets, [ and ])
  • Another object (surrounded by curly brackets, { and })
  • The value null

{
  "cell_id": "b92b8733-6ca5-4460-80eb-ff908d3456c8",
  "rep_address": "http://10.0.16.18:1800",
  "zone": "z1",
  "capacity": {
    "memory_mb": 26127,
    "disk_mb": 62527,
    "containers": 249
  },
  "rootfs_provider_list": [
    {
      "name": "preloaded",
      "properties": [
        "cflinuxfs2"
      ]
    },
    {
      "name": "preloaded+layer",
      "properties": [
        "cflinuxfs2"
      ]
    },
    {
      "name": "docker"
    }
  ],
  "rep_url": "https://b92b8733-6ca5-4460-80eb-ff908d3456c8.cell.service.cf.internal:1801"
}

Lets break this down :

• At the top level, we've written curly braces ({ and }), which creates an object.
• Inside the object, we have several name/value pairs:

"cell_id": "b92b8733-6ca5-4460-80eb-ff908d3456c8"	A property with the name "cell_id" and the integer value b92b8733-6ca5-4460-80eb-ff908d3456c8
"rootfs_provider_list": [..]	A property with the name "rootfs_provide_list", whose value in an array
"capacity": {..}	A property with the name `capacity, whose value is an object

• Inside the rootfs_provider_list array, we have 3 properties of the type object which have further nested properties, one is a property with the name name and the second, an array which has one value

Review : https://codebeautify.org/jsonviewer/cb1bef4d

Intro to jq

jq invocation

stdin -> jq [filter] -> stdout : Read from stdin ; write to stdout

curl -s ... | jq ... cfdot ... | jq ...

jq [filter] file -> stdout ; read from file; write to stdout

slurp mode output as raw strings

Identity filter "jq ."

echo '{"x":1, "y:2}' | jq .

{
  "x": 1,
  "y": 2
}

Projection "jq .property"

• Project out the value of x" from {"x":1, "y":2}

echo '{"x":1, "y":2}'|jq .x

1

Netsted Projection "jq. nested.property"

e.g : {"x:{"y":"z"}}

• z is a property assigned to object y which is assigned to object x
• project out the value z by chaining the path expression

echo '{"x":{"y":"z"}}' | jq .
{
  "x": {
    "y": "z"
  }
}

Array

e.g : '[{"x":1}, {"y":2}]'

  {
    "x": 1
  },
  {
    "y": 2
  }
]

• Print the content of the array

echo '[{"x":1}, {"y":2}]'| jq .[]

{
  "x": 1
}
{
  "y": 2
}

The family of .[] is very versatile

• If the input is an array, .[] will iterate through the elements.
• If the input is an object .[] will iterate through the object values. for eg : echo '{"x":{"y":"z"}, "a":{"b":"c"}}'| jq .[]

{
 "y": "z"
}
{
 "b": "c"
} 

• If you provide an index, .[0] will access elements in arrays by their position. echo '[{"x":1}, {"y":2}]'| jq .[0]

{
   "x": 1
}

• If you provide a string, .["foo"] will access elements in objects by their keys.

Basic Operators

• Pipe

jq | filter[1] | filter[2]

echo '{"x":{"y":"z"}}' | jq .x
{
  "y": "z"
}

echo '{"x":{"y":"z"}}' | jq '.x | .y'
"z"

• Tee ( One record becomes two records)

echo '{"x":1}'|jq '. , .'

{
  "x": 1
}
{
  "x": 1
}

• Select a value from a key in an object

echo '{"foo": 1, "bar": 2}' | jq -c '{foo}'

{"foo":1}

• Build an object out the values of KV pair
  echo '{"x":{"y":"z"}, "a":{"b":"c"}}'| jq '{label1: .x.y, label2: .a.b}'

{
  "label1": "z",
  "label2": "c"
}

• Addition

Constructors

select

if-then-else

group-by

map

Print the values of the key/value pair : echo '[{"x":1}, {"y":2}]'| jq '.[]|.[]'

1
2

Print the values in an array :

echo '[{"x":1}, {"y":2}]'| jq '[.[]|.[]]'

[
  1,
  2
]

The equivalent of the above command is map as show below :

echo '[{"x":1}, {"y":2}]'| jq 'map(.[])'

[
  1,
  2
]

map will run the filter for each element of the input array and returns the output in a new array. In the above example the filter is .[]

cfdot

Some common commands :

1. actual-lrp-groups : to dump the actual LRPs
2. actual-lrp-groups-for-guid : list actual LRP for a process guid
3. task-events : subscribe to task events
4. lrp-events : subscribe to lrp events
5. desired-lrps : dump desired LRPs
6. locks : list locket locks

Use Case 1 :

How to find the instance_guid for a particular app_guid?

a) first find the process guid

cf curl /v2/apps/$(cf app <app_name> --guid)| jq -r '.metadata.guid + "-" + .entity.version'

https://jqplay.org/s/7kWJ2Hn3UX

b) identify the diego cell where the app instances are running : cf curl v2/apps/$(cf app healthwatch-api --guid)/stats | grep host

c) run cfdot on the cell

cfdot actual-lrp-groups-for-guid <process_guid>

c) Inspect json blob https://codebeautify.org/jsonviewer/cb46f834 d) Use jqplay to start operating on the json blob https://jqplay.org/s/EB0r_Jz5ay

cfdot actual-lrp-groups-for-guid eb2f93fb-b2fc-4f7b-998d-063c18cf8ba0-27cc5569-6f5d-4f47-baf6-5c3d43f0e785| jq '.instance | select (.address == "10.0.16.17")| .instance_guid'

d) Once you have the instance guid, then you have backdoor to the container.

e) optional : List all diego cells with their respective instance_guids for a specific process-guid

cfdot actual-lrp-groups-for-guid $pg | jq '.instance | "\(.index): Cell \(.cell_id), Instance \(.instance_guid)"' -r

Use Case 2

For a specific app-guid, find the desired LRP and inspect the json blob.

The desired LRP tells you a lot of useful things like:

• environment variables being set
• the URL from which the droplet is being downloaded
• the commands being used to run the app
• app start timeout
• info about the healthcheck to be used
• memory & disk limits + CPU weight
• routes bound & if those are in an isolation segment
• egress rules
• the URL from which the buildpack cache can be downloaded

Use Case 3

cfdot actual-lrp-groups | jq '.instance | select(.instance_address =="10.255.127.8") | {"process_guid": .process_guid, "index": .index, "instance_guid": .instance_guid, "cell_id": .cell_id, "address": .address, "instance_address": .instance_address, "ports": .ports, "state": .state}

Written with StackEdit.