misterdjules

This document describes the root cause investigation for the core dump available in Manta at /trent.mick/public/bugs/oneachnodecrash/core.node.46811.

The program that the dumped process was running seems to be a 32bits node binary:

[root@headnode (coal) ~]# pargs /var/tmp/trent-sdconeachnode-core
core '/var/tmp/trent-sdconeachnode-core' of 46811:      /zones/f26d87e2-f7bf-48b9-bf74-be2dc9e421aa/root/opt/smartdc/sdc/build/node/bin
argv[0]: /zones/f26d87e2-f7bf-48b9-bf74-be2dc9e421aa/root/opt/smartdc/sdc/build/node/bin/node
argv[1]: /zones/f26d87e2-f7bf-48b9-bf74-be2dc9e421aa/root/opt/smartdc/sdc/lib/sdc-oneachnode.js
argv[2]: -n
argv[3]: 564dd6c4-bea3-c59c-fbd9-02536df17513

misterdjules

[root@6fa9d8d9-20bc-4f6f-c491-a459c108d6db ~/manta-muskie]# ./build/node/bin/node node_modules/.bin/nodeunit test/auth.test.js 

auth.test.js
before:
NotEnoughSpaceError: not enough free space for 51200 MB
    at ClientRequest.onResponse (/root/manta-muskie/node_modules/manta/node_modules/restify-clients/lib/HttpClient.js:217:26)
    at ClientRequest.g (events.js:180:16)
    at ClientRequest.emit (events.js:95:17)
    at HTTPParser.parserOnIncomingClient (http.js:1751:21)
    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:152:23)

misterdjules

[2017-05-19T23:15:30.146Z]  INFO: sapi/12687 on 4c2bc81a-a004-4076-b47a-4350ac433b76: handled: 200 (route=listinstances, audit=true, remoteAddress=172.25.1.27, remotePort=62127, latency=2, _audit=true)
    GET /instances?service_uuid=1574c183-4d73-4164-9392-6891193b084f HTTP/1.1
    accept: application/json
    user-agent: restify/2.8.5 (ia32-sunos; v8/3.14.5.9; OpenSSL/1.0.1e) node/0.10.26
    accept-version: *
    date: Fri, 19 May 2017 23:15:30 GMT
    host: sapi.nightly-1.joyent.us
    connection: keep-alive
    --
    HTTP/1.1 200 OK

misterdjules

[root@headnode (nightly-1) ~]# sdcadm up -C experimental moray@8566e752-3c55-11e7-9d5f-ff436b91265c
Using channel experimental

This update will make the following changes:
    download 1 image (55 MiB):
        image 8566e752-3c55-11e7-9d5f-ff436b91265c
            (manta-moray@MORAY-104-20170519T053504Z-gf738f50)
    update "moray" service to image 8566e752-3c55-11e7-9d5f-ff436b91265c
        (manta-moray@MORAY-104-20170519T053504Z-gf738f50):

misterdjules

[root@headnode (nightly-1) ~]# zlogin -Q -i $(vmadm lookup -1 alias=manatee0) /usr/bin/env
PATH=/usr/sbin:/usr/bin
TERM=xterm-256color
[root@headnode (nightly-1) ~]# zlogin -Q $(vmadm lookup -1 alias=manatee0)
Last login: Tue May  9 19:02:06 on pts/2
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ; Instance (multiarch 13.3.1)

misterdjules

Introduction

When reviewing MORAY-104, Cody made the suggestion that that change should also include the fix for the issue where sending a findObjects request for a bucket that has fields being reindexed sends records that don't include these fields.

I have two concerns with this suggestion:

1. even though it makes sense in many use cases to have that behavior in conjunction with the behavior that checks that search fields have all usable

misterdjules

Introduction

ZAPI-685 and ZAPI-782 were two changes made to the same DeleteVm VMAPI endpoint, and they both broke VMAPI clients in some way.

ZAPI-685 was originally made to avoid starting workflow jobs, and thus potentially flooding the jobs queue, when handling DeleteVm requests, unless necessary.

misterdjules

Introduction

PUBAPI-1370 describes the problem when a volume is being created and the provisioning of its storage VM fails:

• after the provisioning workflow started
• before the actual VM was created on the CN

In this case, the volume being created will stay in the state creating, since

misterdjules

Docker supports an endpoint for getting/streaming

Does that mean getting and streaming? It might help to make this clearer as to not confuse the reader from the beginning. Maybe "getting" is good enough at this point in the document, even if technically the client is "streamimg" them.

Recent changes to the Docker cli for Docker version 1.13.0 have increased the priority of implementing the /events endpoint at least minimally, as the cli now depends on /events to gather the exit code for the docker run command and users will get errors and the wrong exit code for every docker run with the new

misterdjules

➜  node-dtrace-provider-issue-90 ls             
aside-gc.js  dtrace-gc.js node_modules package.json
➜  node-dtrace-provider-issue-90 ls -l node_modules 
total 8
lrwxr-xr-x  1 JulienGilli  staff  58 Jan 26 18:15 dtrace-provider -> ../../../../.nvm/v0.10.40/lib/node_modules/dtrace-provider
➜  node-dtrace-provider-issue-90 ls node_modules/dtrace-provider/build 
DTraceProviderBindings.target.mk Makefile                         config.gypi                      libusdt.target.mk
Debug                            binding.Makefile                 gyp-mac-tool
➜  node-dtrace-provider-issue-90 cat aside-gc.js 
var dtrace = require('./dtrace-gc');