floehopper

Achievements

• Added functionality to Asset Manager to support Whitehall attachments, e.g. access-limiting
• Identified a problem in serving draft attachments in the AWS integration environment

Ongoing and next

• Continued preparations to start serving Whitehall attachments from Asset Manager
  • Updating state of assets in Asset Manager when state changes in Whitehall
  • Preview of CSV attachments

floehopper

commit 3253a2b7cdf1ba8b60365638ad1b517bbc1e30fd
Author: James Mead <james@floehopper.org>
Date:   Sun Feb 18 12:21:40 2018 +0000

    Publish event for force_publish within EditionUnwithdrawer#perform!
    
    The EditionUnwithdrawer was already publishing an 'unwithdrawn' event to
    the notifier for the originally withdrawn edition which ends up as
    superseded, but it was not publishing a 'force_publish' event for the
    new draft edition which is force-published.

floehopper

vagrant@development:~$ cd /var/apps/whitehall
vagrant@development:/var/apps/whitehall$ bundle
Fetching gem metadata from https://rubygems.org/.........
Using rake 12.3.0
Using Ascii85 1.0.3
Using PriorityQueue 0.1.2
Using concurrent-ruby 1.0.5
Using i18n 0.9.5
Using minitest 5.11.3
Using thread_safe 0.3.6

floehopper

$ systemctl --version
systemd 229
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN

$ sudo systemctl show -p StartLimitInterval nginx.service
StartLimitInterval=10000000

$ sudo systemctl set-property nginx.service StartLimitInterval=0
Unknown assignment StartLimitInterval=0

floehopper

Rails console

irb> a = Asset.create!(file: File.open('README.md'), access_limited: true, organisation_slug: 'organisation-slug')
=> #<Asset _id: 5a68c81c3e489e7bb5baac3e, deleted_at(deleted_at): nil, created_at: 2018-01-24 17:53:32 UTC, updated_at: 2018-01-24 17:53:32 UTC, state: "unscanned", filename_history: [], uuid: "0b0d0cb0-c49f-4f47-b204-a991c8af9cbf", access_limited: true, organisation_slug: "organisation-slug", etag: "5a68c81c-2f81", last_modified: 2018-01-24 17:53:32 UTC, md5_hexdigest: "d463b46b2c42d1f043e56920a03de458", file: "README.md", _type: "Asset">
irb(main):003:0> a.access_limited?
=> true
irb(main):004:0> a.organisation_slug
=> "organisation-slug"

floehopper

Create mainstream asset

$ echo `date` > tmp.txt
$ curl http://localhost:3000/assets --form "asset[file]=@tmp.txt" --form "asset[access_limited]=1" --form "asset[organisation_slug]=foo"

Rails log

floehopper

puts subject.call.resources.detect { |r| r.type == 'Nginx::Config::Site' && r['name'] == 'assets-origin.dev.gov.uk' }['content']

floehopper

2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "Host: assets-origin.dev.gov.uk"
2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "Connection: keep-alive"
2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safar
i/537.36"
2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "Upgrade-Insecure-Requests: 1"
2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "Accept-Encoding: gzip, deflate"
2018/01/23 16:17:58 [debug] 18102#0: *15 http header: "Accept-Language: en-GB,en-US;q=0.9,en;q=0.8"

GET /media/5a67581d759b7424f4e81ba1/README.md?jm345 HTTP/1.0

floehopper

2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "Host: asset-manager.dev.gov.uk"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "X-Real-IP: 127.0.0.1"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "X-Forwarded-Server: static.dev.gov.uk"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "X-Forwarded-For: 127.0.0.1"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "X-Forwarded-Host: static.dev.gov.uk"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "Connection: close"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari
/537.36"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "Upgrade-Insecure-Requests: 1"
2018/01/23 16:11:22 [debug] 17761#0: *5 http header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"

floehopper

Achievements

• Migrated more non-attachment assets from Whitehall to Asset Manager
• Started serving the migrated Whitehall non-attachment assets from Asset Manager

Ongoing and next

• Continuing to migrate remaining non-attachment assets from Whitehall to Asset Manager and removing them from NFS
• Continuing to switch these Whitehall assets over to be served from Asset Manager
• Spiking on supporting attachment assets from Whitehall in Asset Manager (authorisation for drafts)