Skip to content

Instantly share code, notes, and snippets.

package main
import (
"errors"
"fmt"
"io"
"os"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"

Generic format for registry documents

Why do we need standard registry document formats anyway?

A registry is a data store for content addressed data. At the lowest level it is just a large key (hash) to value (document) store, but one of the things we have learned about data stores is that they are more useful if they allow for structured data (a Merkle tree, technically a dag), with data being allowed to point to (hashes of) other data items. The complexity added by this is that if the data store needs to follow the links in documents it needs to know how to parse them. The primary use case for this parsing is for garbage collection: the usual storage model allows for an object that is not referenced internally or from a tag (a generic name that can be given to items in the store) may be garbage collected. Without this it is difficult to remove any items from the store. Tags exist to give human friendly names and to anchor items into the store while they exist.

Probably the best developed system alo

@justincormack
justincormack / notes
Created September 5, 2018 14:41
criu setup Ubuntu 18.04
apt update
apt upgrade
apt install build-essential
apt install pkg-config
apt install libnet-dev python-yaml libaio-dev
apt install libprotobuf-dev libprotobuf-c0-dev protobuf-c-compiler protobuf-compiler python-protobuf libnl-3-dev libcap-dev python-future
# criu install
curl -O -sSL http://download.openvz.org/criu/criu-3.10.tar.bz2
tar xjf criu-3.10.tar.bz2
## Modifiers for ss tokens
The K, X, KK and IK patterns have an "ss" token in the first line. These are there on the basis that this adds some
authentication for the initiator, but it is not at that point capable of using the standard "se", as it does not have
the responders ephemeral key yet. In some cases this may not be needed, for example if the initiator is not sending
a premessage at this point in the handshake, especially for the two way handshakes. For this reason there is a "noss"
modifier that omits the "ss" token from these handshakes. This improves performance slightly by omitting one DH
operation.
Conversely there are use cases where adding an "ss" token is useful, as it can strengthen the authentication for a
@justincormack
justincormack / content.go
Created July 8, 2018 13:48
Enforcing content addressed storage on S3 with signed URLs and sha256 hashes
package main
import (
"crypto/sha256"
"encoding/hex"
"flag"
"fmt"
"io"
"net/http"
"os"
I have run an nginx container...
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6d67de07731d nginx "nginx -g 'daemon ..." 40 minutes ago Up 40 minutes 80/tcp, 443/tcp epic_goldberg
I want to use Debian for debug:
docker run -it --pid=container:6d67de07731d --net=container:6d67de07731d --cap-add sys_admin debian
I can see the nginx process:
execve("/sbin/apk", ["apk", "update"], [/* 9 vars */]) = 0
arch_prctl(ARCH_SET_FS, 0x7fec6536bb48) = 0
set_tid_address(0x7fec6536bb80) = 2469
open("/etc/ld-musl-x86_64.path", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/libssl.so.39", O_RDONLY|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=331864, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@n\1\0\0\0\0\0"..., 960) = 960
mmap(NULL, 2428928, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x7fec64e8f000
mmap(0x7fec650d7000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x48000) = 0x7fec650d7000
dev/
dev/agpgart
dev/audio
dev/audio1
dev/audio2
dev/audio3
dev/audioctl
dev/console
dev/core
dev/dsp
FROM alpine
RUN touch /bin/sh && \
ldd /bin/sh
time="2016-01-19T14:51:15.264108533Z" level=info msg="Layer sha256:4e0e182967f847e292d3b8c8ebff2cf567c6a38713ddbad9f40e8dad2e603ff4 cleaned up"
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x815856]
goroutine 1 [running]:
github.com/docker/docker/layer.NewStoreFromGraphDriver(0x7fd78c5eee38, 0xc820616d80, 0x7fd78c5eed38, 0xc820357220, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/docker/docker/layer/layer_store.go:90 +0x446
github.com/docker/docker/layer.NewStoreFromOptions(0xc820611800, 0xf, 0xc820258140, 0x20, 0x7ffd15737edb, 0x4, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/docker/docker/layer/layer_store.go:66 +0x496
github.com/docker/docker/daemon.NewDaemon(0xc82032f080, 0xc8203580e0, 0x0, 0x0, 0x0)