Varnish persistent storage crash - see https://www.varnish-cache.org/trac/ticket/1077

varnish-crash.log

Dec 23 11:14:12 varnishd[3802]: Child (17362) said Out of space in persistent silo
Dec 23 11:14:12 varnishd[3802]: Child (17362) said Committing suicide, restart will make space
Dec 23 11:14:12 varnishd[3802]: Child (17362) ended
Dec 23 11:14:12 varnishd[3802]: Child cleanup complete
Dec 23 11:14:12 varnishd[3802]: child (20093) Started
Dec 23 11:14:12 varnishd[3802]: Child (20093) said Child starts
Dec 23 11:14:12 varnishd[3802]: Child (20093) said Dropped 10 segments to make free_reserve
Dec 23 11:14:12 varnishd[3802]: Child (20093) said Dropped 0 segments to make free_reserve
Dec 23 11:14:12 varnishd[3802]: Child (20093) said Dropped 0 segments to make free_reserve
Dec 23 11:14:13 varnishd[3802]: Child (20093) said Silo completely loaded
Dec 23 11:15:01 varnishd[3802]: last message repeated 2 times
Dec 23 15:08:56 varnishd[3802]: Child (20093) said Out of space in persistent silo
Dec 23 15:08:56 varnishd[3802]: Child (20093) said Committing suicide, restart will make space
Dec 23 15:08:56 varnishd[3802]: Child (20093) ended
Dec 23 15:08:56 varnishd[3802]: Child cleanup complete
Dec 23 15:08:56 varnishd[3802]: child (23662) Started
Dec 23 15:08:56 varnishd[3802]: Child (23662) said Child starts
Dec 23 15:08:56 varnishd[3802]: Child (23662) said Dropped 11 segments to make free_reserve
Dec 23 15:08:56 varnishd[3802]: Child (23662) said Dropped 0 segments to make free_reserve
Dec 23 15:08:56 varnishd[3802]: Child (23662) said Dropped 0 segments to make free_reserve
Dec 23 15:09:02 varnishd[3802]: Child (23662) said Silo completely loaded
Dec 23 15:10:01 varnishd[3802]: last message repeated 2 times
Dec 23 17:45:25 varnishd[3802]: Child (23662) died signal=6
Dec 23 17:45:25 varnishd[3802]: Child (23662) Panic message: Assert error in smp_oc_getobj(), storage_persistent_silo.c line 400:#012  Condition((const void*)(o) >= (const void*)((sg->sc)->base) && (const void*)(o) < (const void *)((sg->sc)->base + (sg->sc)->mediasize)) not true.#012errno = 22 (Invalid argument)#012thread = (cache-worker)#012ident = Linux,2.6.32-341-ec2,x86_64,-spersistent,-spersistent,-spersistent,-smalloc,-hcritbit,epoll#012Backtrace:#012  0x42f118: /usr/sbin/varnishd() [0x42f118]#012  0x44c525: /usr/sbin/varnishd() [0x44c525]#012  0x428986: /usr/sbin/varnishd(HSH_Lookup+0x3a6) [0x428986]#012  0x415a79: /usr/sbin/varnishd() [0x415a79]#012  0x419165: /usr/sbin/varnishd(CNT_Session+0x6d5) [0x419165]#012  0x430aa8: /usr/sbin/varnishd() [0x430aa8]#012  0x430f31: /usr/sbin/varnishd() [0x430f31]#012  0x7f7c7e0b69ca: /lib/libpthread.so.0(+0x69ca) [0x7f7c7e0b69ca]#012  0x7f7c7de1370d: /lib/libc.so.6(clone+0x6d) [0x7f7c7de1370d]#012sp = 0x7f7417bce008 {#012  fd = 32, id = 32, xid = 779726692,#012  client = 10.210.214.8 55620,#012  step = STP_LOOKUP,#012  handling = hash,#012  restarts = 0, esi_level = 0#012  flags = #012  bodystatus = 4#012  ws = 0x7f7417bce080 { #012    id = "sess",#012    {s,f,r,e} = {0x7f7417bcec90,+912,+65536,+65536},#012  },#012  http[req] = {#012    ws = 0x7f7417bce080[sess]#012      "GET",#012      "/path/to/a.jpg",#012      "HTTP/1.1",#012      "host: myserver",#012      "Accept: image/png,image/*;q=0.8,*/*;q=0.5",#012      "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7",#012      "Accept-Language: en-gb,en;q=0.5",#012      "Cache-Control: max-age=86400"…
Dec 23 17:45:25 varnishd[3802]: Child cleanup complete
Dec 23 17:45:25 varnishd[3802]: child (26689) Started
Dec 23 17:45:26 varnishd[3802]: Pushing vcls failed:#012CLI communication error (hdr)
Dec 23 17:45:26 varnishd[3802]: Stopping Child
Dec 23 17:45:26 varnishd[3802]: Child (26689) died signal=6
Dec 23 17:45:26 varnishd[3802]: Child (26689) Panic message: Assert error in smp_open_segs(), storage_persistent.c line 239:#012  Condition(sg1->p.offset != sg->p.offset) not true.#012thread = (cache-main)#012ident = Linux,2.6.32-341-ec2,x86_64,-spersistent,-spersistent,-spersistent,-smalloc,-hcritbit,no_waiter#012Backtrace:#012  0x42f118: /usr/sbin/varnishd() [0x42f118]#012  0x44a96d: /usr/sbin/varnishd() [0x44a96d]#012  0x44aacb: /usr/sbin/varnishd() [0x44aacb]#012  0x447f57: /usr/sbin/varnishd(STV_open+0x27) [0x447f57]#012  0x42ddfa: /usr/sbin/varnishd(child_main+0xca) [0x42ddfa]#012  0x4406ee: /usr/sbin/varnishd() [0x4406ee]#012  0x441037: /usr/sbin/varnishd() [0x441037]#012  0x7f7c7f1db6a2: /usr/lib/varnish/libvarnish.so(+0x96a2) [0x7f7c7f1db6a2]#012  0x7f7c7f1dbcf8: /usr/lib/varnish/libvarnish.so(vev_schedule+0x88) [0x7f7c7f1dbcf8]#012  0x4408d9: /usr/sbin/varnishd(MGT_Run+0x139) [0x4408d9]#012