chrisroos

Keybase proof

I hereby claim:

• I am chrisroos on github.
• I am chrisroos (https://keybase.io/chrisroos) on keybase.
• I have a public key ASAHSRfXQ5NzVNyEmSsHC02aqeI4TZmm-_XI-RiUSAYQXAo

To claim this, I am signing this object:

chrisroos

Testing Rails ActionMailer and AWS SES

I added two verified email addresses to AWS SES so that I could send emails from/to them while in the SES Sandbox.

UserMailer

class UserMailer < ApplicationMailer
  default from: 'accounts+aws-ses@gofreerange.com'

chrisroos

# These were the IDs returned by the identify-affected-attachments.rb script when it was run in staging on 20 Mar 2018.
# 
[3625, 6002, 6007, 7874, 7875, 12248, 12249, 16333, 16381, 16383, 27263, 27607, 28153, 28154, 48876, 65687, 69606, 97881, 118296, 137087, 137587, 151994, 151995, 151996, 152690, 156319, 163466, 163467, 163468, 163469, 163470, 163471, 167849, 167851, 172237, 172238, 172239, 176752, 176753, 179306, 180955, 182610, 182636, 184031, 190436, 190437, 190438, 190439, 198104, 198144, 203773, 203952, 206956, 210410, 210933, 211644, 211646, 212271, 222544, 222545, 226017, 226119, 230027, 232152, 245882, 246519, 249795, 249796, 251493, 255473, 256038, 256798, 257471, 259192, 259868, 260418, 260904, 263013, 263014, 263359, 266717, 266834, 268678, 268679, 268680, 268681, 268682, 268683, 268684, 268685, 268686, 268687, 270767, 271065, 273957, 273959, 276018, 276347,276696, 276698, 278305, 279330, 281943, 284025, 284200, 284202, 284203, 284204, 284205, 284208, 284962, 286668, 286670, 287941, 288107, 2881

chrisroos

# See https://github.com/alphagov/asset-manager/issues/525 for more information

ids = [1, 2, 3, 4, 5, 6, 7, 11, 17, 33, 44, 47, 79, 80, 82, 88, 94, 98, 99, 101, 106, 108, 135, 136, 137, 138, 139, 140, 145, 146, 153, 154, 160, 168, 182, 187, 231, 350, 374, 378, 379, 382, 389, 413, 437, 636, 765, 773, 774, 776, 863, 876, 966, 1223, 1224, 1456, 1486, 1519, 1724, 1725, 1751, 1752, 1753, 1754, 1755, 1765, 1766, 1767, 1773, 1776, 1779, 1781, 1782, 1783, 1785, 1786, 1787, 1788, 1789, 1792, 1793, 1794, 1795, 1796, 1797, 1798, 1799, 1800, 1801, 1802, 1803, 1804, 1805, 1806, 1807, 1808, 1809, 1819, 1827, 1831, 1832, 1833, 1834, 1835, 1836, 1837, 1838, 1839, 1840, 1841, 1842, 1843, 1844, 1845, 1846, 1848, 1849, 1850, 1851, 1852, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1860, 1861, 1862, 1863, 1864, 1865, 1866, 1867, 1868, 1869, 1870, 1871, 1872, 1873, 1874, 1875, 1876, 1877, 1878, 1879, 1880, 1881, 1882, 1883, 1884, 1885, 1886, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894, 1895, 1896, 1897, 1898, 1899, 1900, 1901, 1

chrisroos

# Script used to save the URL paths of all the AttachmentData records in the Whitehall database
#
# To use:
#
# 1. SSH to a whitehall_backend box in integration/staging
# 2. Save the script as /tmp/save-attachment-data-urls.rb
# 3. Run `sudo su - deploy`
# 4. Run `cd /var/apps/whitehall`
# 5. Run `govuk_setenv whitehall bundle exec rails r /tmp/save-attachment-data-urls.rb`

chrisroos

[1, 2, 3, 4, 5, 6, 7, 11, 17, 33, 44, 47, 79, 80, 82, 88, 94, 98, 99, 101, 106, 108, 135, 136, 137, 138, 139, 140, 145, 146, 153, 154, 160, 168, 182, 187, 231, 350, 374, 378, 379, 382, 389, 413, 437, 636, 765, 773, 774, 776, 863, 876, 966, 1223, 1224, 1456, 1486, 1519, 1724, 1725, 1751, 1752, 1753, 1754, 1755, 1765, 1766, 1767, 1773, 1776, 1779, 1781, 1782, 1783, 1785, 1786, 1787, 1788, 1789, 1792, 1793, 1794, 1795, 1796, 1797, 1798, 1799, 1800, 1801, 1802, 1803, 1804, 1805, 1806, 1807, 1808, 1809, 1819, 1827, 1831, 1832, 1833, 1834, 1835, 1836, 1837, 1838, 1839, 1840, 1841, 1842, 1843, 1844, 1845, 1846, 1848, 1849, 1850, 1851, 1852, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1860, 1861, 1862, 1863, 1864, 1865, 1866, 1867, 1868, 1869, 1870, 1871, 1872, 1873, 1874, 1875, 1876, 1877, 1878, 1879, 1880, 1881, 1882, 1883, 1884, 1885, 1886, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894, 1895, 1896, 1897, 1898, 1899, 1900, 1901, 1902, 1903, 1904, 1905, 1906, 1907, 1908, 1909, 1910, 1911, 1912, 1913, 1914, 1915, 1916

chrisroos

require 'net/http'

file_path = Rails.root.join('tmp', 'tmp.txt')
File.open(file_path, 'w') { |file| file.puts(Time.now) }
asset = Asset.create!(file: File.open(file_path))

def display_headers(response)
  puts "Code:            #{response.code}"
  puts "Etag:            #{response.header['Etag']}"
  puts "Content-Type:    #{response.header['Content-Type']}"

chrisroos

assets_checked = 0
assets_without_content_type = 0

percentage_of_assets_to_test = 0.1

assets = Asset.all
assets.each.with_index do |asset, index|
  next unless (rand * 100) < percentage_of_assets_to_test

  id = asset.id

chrisroos

Nginx proxy to signed S3 URLs

• Backend app sets X-Accel-Redirect header to a location that nginx is configured to handle (e.g. /s3-proxy/)

• Backend app generates signed URL and stores it in HTTP header

• Backend app stores s3 host in separate HTTP header

• nginx location (e.g. /s3-proxy/) extracts content from HTTP headers and proxies requests to S3

chrisroos

Debugging nginx config

• Use the debug log level to the error_log directive

  • I think this writes to the log for all requests, not just errors.

• Use return 302 http://localhost to force a redirect to localhost.

• Use add_header to write debug info to HTTP headers for later inspection