Skip to content

Instantly share code, notes, and snippets.

@alfondotnet
Created February 23, 2024 12:34
Show Gist options
  • Save alfondotnet/2f233cd0aa1380a9ccfe08cf1cd161fe to your computer and use it in GitHub Desktop.
Save alfondotnet/2f233cd0aa1380a9ccfe08cf1cd161fe to your computer and use it in GitHub Desktop.
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><link rel="preconnect" href="https://static.linear.app" crossorigin="anonymous"/><link rel="preconnect" href="https://cdn.sanity.io" crossorigin="anonymous"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/><link rel="preload" as="image" href="https://cdn.sanity.io/images/ornj730p/production/dff9dd98ea36ac93a38ef6405abc866dda659f68-492x492.jpg?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2"/><link rel="preload" as="image" href="https://cdn.sanity.io/images/ornj730p/production/4d081a7a9ee2575014ab150f2d74cc3e2266a937-300x300.jpg?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2"/><link rel="preload" as="image" href="https://cdn.sanity.io/images/ornj730p/production/c5229881e58cfb3a30c31b036e2965f9f327dcc8-512x512.png?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2"/><link rel="preload" as="image" href="https://cdn.sanity.io/images/ornj730p/production/2e281ae34028ff175a63af027de95bca66b9bcb1-2056x944.png?q=95&amp;auto=format&amp;dpr=2" fetchPriority="high"/><link rel="preload" as="image" href="https://cdn.sanity.io/images/ornj730p/production/e078697b78690170e26ec6ed13fd77483d5269c4-2056x944.png?q=95&amp;auto=format&amp;dpr=2" fetchPriority="high"/><link rel="stylesheet" href="/_next/static/css/697eabb2bbc445a1.css" crossorigin="" data-precedence="next"/><link rel="stylesheet" href="/_next/static/css/66815da86973e6a3.css" crossorigin="" data-precedence="next"/><link rel="stylesheet" href="/_next/static/css/c6672b3b5da5c45f.css" crossorigin="" data-precedence="next"/><link rel="preload" as="script" fetchPriority="low" href="/_next/static/chunks/webpack-5d7fdfd3db0f75de.js" crossorigin=""/><script src="/_next/static/chunks/ea5af1dd-dd789632f0059c21.js" async="" crossorigin=""></script><script src="/_next/static/chunks/6185-c2e2b3a97664c646.js" async="" crossorigin=""></script><script src="/_next/static/chunks/main-app-3c0bb951bcc68f7c.js" async="" crossorigin=""></script><script src="/_next/static/chunks/597e48ed-804b33f585e7b335.js" async=""></script><script src="/_next/static/chunks/2853-b6ae42a21e872220.js" async=""></script><script src="/_next/static/chunks/4320-fecd6f2023ea247a.js" async=""></script><script src="/_next/static/chunks/1249-0b42965bd6dc6539.js" async=""></script><script src="/_next/static/chunks/4884-fc3796079191e457.js" async=""></script><script src="/_next/static/chunks/64-79db7aac0ce71bb5.js" async=""></script><script src="/_next/static/chunks/705-f288e190c72681e1.js" async=""></script><script src="/_next/static/chunks/app/layout-67c8bbf39822b51d.js" async=""></script><script src="/_next/static/chunks/4002-e41702bacedfdc74.js" async=""></script><script src="/_next/static/chunks/3831-f7c2823b994006f3.js" async=""></script><script src="/_next/static/chunks/1475-da89268f4995c903.js" async=""></script><script src="/_next/static/chunks/1795-ee1cd5610d48545e.js" async=""></script><script src="/_next/static/chunks/app/error-2292dde8b1c24f73.js" async=""></script><script src="/_next/static/chunks/app/not-found-d6645cca11b417f0.js" async=""></script><script src="/_next/static/chunks/app/global-error-b8704b25c3e7cbc6.js" async=""></script><script src="/_next/static/chunks/2065-1fba709f93bc0b20.js" async=""></script><script src="/_next/static/chunks/5984-fe6969300ebc9026.js" async=""></script><script src="/_next/static/chunks/2226-b8801cbd0f1f64c3.js" async=""></script><script src="/_next/static/chunks/3866-a9d8aacd2f46c894.js" async=""></script><script src="/_next/static/chunks/8400-34793ad87919c231.js" async=""></script><script src="/_next/static/chunks/9427-58a8df3ca7616cc2.js" async=""></script><script src="/_next/static/chunks/3938-6cae752823dd8243.js" async=""></script><script src="/_next/static/chunks/3293-1d0c3dc323cbeb08.js" async=""></script><script src="/_next/static/chunks/2021-024b4f80edf8ab1b.js" async=""></script><script src="/_next/static/chunks/app/(marketing)/blog/%5Bslug%5D/page-549e721e0d4ff138.js" async=""></script><link rel="preload" href="https://www.googletagmanager.com/gtag/js?id=G-Z4CF4937R9" as="script"/><link rel="preload" href="/static/fonts/inter-var-subset-latin-opsz-sep27.woff2" as="font" type="font/woff2" crossorigin="anonymous"/><title>Post mortem on Linear incident from Jan 24th, 2024 | Linear</title><meta name="description" content="Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages"/><meta name="application-name" content="Linear"/><link rel="manifest" href="https://linear.app/static/pwa.webmanifest?v=3"/><meta name="format-detection" content="telephone=no, date=no, address=no, email=no"/><meta name="apple-mobile-web-app-capable" content="yes"/><meta name="apple-mobile-web-app-status-bar-style" content="black"/><meta property="og:title" content="Post mortem on Linear incident from Jan 24th, 2024 | Linear"/><meta property="og:description" content="Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages"/><meta property="og:image" content="https://cdn.sanity.io/images/ornj730p/production/b35aa350e5f84242286a65f1464c4b25462947a5-2400x1256.jpg?q=95&amp;auto=format&amp;dpr=2"/><meta name="twitter:card" content="summary"/><meta name="twitter:site" content="@linear"/><meta name="twitter:creator" content="@linear"/><meta name="twitter:title" content="Post mortem on Linear incident from Jan 24th, 2024 | Linear"/><meta name="twitter:description" content="Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages"/><meta name="twitter:image" content="https://cdn.sanity.io/images/ornj730p/production/b35aa350e5f84242286a65f1464c4b25462947a5-2400x1256.jpg?q=95&amp;auto=format&amp;dpr=2"/><link rel="icon" href="/favicon.ico" sizes="any"/><link rel="icon" href="/static/favicon.svg" type="image/svg+xml"/><link rel="apple-touch-icon" href="/static/apple-touch-icon.png" sizes="180x180"/><script>(()=>{let e=document.documentElement,t=window.navigator;(function(){try{let r=8;void 0!==t&&"hardwareConcurrency"in t&&(r=t.hardwareConcurrency),e.classList.toggle("enhanced",r>4)}catch(e){}})(),function(){try{document.cookie&&document.cookie.indexOf("loggedIn=1")>-1&&e.classList.add("logged-in")}catch(e){}}(),function(){try{e.classList.add("js")}catch(e){}}()})();</script><script>/* <!-- CLIENT_ENV --> */</script><script src="/_next/static/chunks/polyfills-c67a75d1b6f99dc8.js" crossorigin="" noModule=""></script><style data-styled="" data-styled-version="5.3.6">.cJUdsU{height:auto;max-width:100%;}/*!sc*/
@supports (font:-apple-system-body) and (-webkit-appearance:none){.cJUdsU[loading="lazy"]:not([data-loaded="true"]){-webkit-clip-path:inset(0.6px);clip-path:inset(0.6px);}}/*!sc*/
html.js .sc-cf331c07-0[data-fade="true"]{opacity:0;-webkit-mask:var(--mask-start);mask:var(--mask-start);--mask-start:linear-gradient(to right,#000 25%,#000000e6 50%,#00000000) 150% 0 / 400% no-repeat;--mask-end:linear-gradient(to right,#000 25%,#000000e6 50%,#00000000) 0 / 400% no-repeat;}/*!sc*/
html.js .sc-cf331c07-0[data-fade="true"][data-loaded="true"]{-webkit-animation:lbEHvc 800ms both;animation:lbEHvc 800ms both;}/*!sc*/
html.js [data-rmiz-modal-content] .sc-cf331c07-0[data-fade="true"]{opacity:1;-webkit-mask:none;mask:none;-webkit-animation:none;animation:none;}/*!sc*/
data-styled.g8[id="sc-cf331c07-0"]{content:"cJUdsU,"}/*!sc*/
.dZeLXf{display:none;}/*!sc*/
data-styled.g9[id="sc-b850125d-0"]{content:"dZeLXf,"}/*!sc*/
.cVJRDH{position:relative;}/*!sc*/
data-styled.g10[id="sc-b850125d-1"]{content:"cVJRDH,"}/*!sc*/
.fCOUsV{position:absolute;top:0;bottom:0;left:0;right:0;pointer-events:none;}/*!sc*/
.fCOUsV[href="#"]{display:none;}/*!sc*/
.sc-b850125d-0:target ~ * .sc-b850125d-2,.sc-b850125d-0:checked ~ * .fCOUsV{display:none;}/*!sc*/
.sc-b850125d-0:target ~ * .sc-b850125d-2[href="#"],.sc-b850125d-0:checked ~ * .fCOUsV[href="#"]{display:block;pointer-events:unset;}/*!sc*/
data-styled.g11[id="sc-b850125d-2"]{content:"fCOUsV,"}/*!sc*/
.gRfBMV{display:block;position:absolute;top:0;bottom:0;left:0;right:0;cursor:pointer;}/*!sc*/
.sc-b850125d-0:target ~ * .sc-b850125d-3{pointer-events:none;}/*!sc*/
data-styled.g12[id="sc-b850125d-3"]{content:"gRfBMV,"}/*!sc*/
.bkPaLG{--visibility-delay:var(--mobile-menu-animation-duration);visibility:hidden;-webkit-transition:visibility 0s var(--visibility-delay);transition:visibility 0s var(--visibility-delay);}/*!sc*/
.sc-b850125d-0:target ~ * .sc-b850125d-4,.sc-b850125d-0:checked ~ * .bkPaLG{--visibility-delay:0s;visibility:visible;}/*!sc*/
data-styled.g13[id="sc-b850125d-4"]{content:"bkPaLG,"}/*!sc*/
.cdFcqp{position:absolute;border:0;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;-webkit-clip:rect(0,0,0,0);clip:rect(0,0,0,0);white-space:nowrap;word-wrap:normal;}/*!sc*/
data-styled.g14[id="sc-b850125d-5"]{content:"cdFcqp,"}/*!sc*/
.hMxkas{color:currentColor;font-size:21px;line-height:28px;-webkit-letter-spacing:-0.37px;-moz-letter-spacing:-0.37px;-ms-letter-spacing:-0.37px;letter-spacing:-0.37px;font-weight:var(--font-weight-medium);}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.fTnzTg{color:var(--color-text-quaternary);text-align:center;font-size:13px;line-height:19px;-webkit-letter-spacing:-0.12px;-moz-letter-spacing:-0.12px;-ms-letter-spacing:-0.12px;letter-spacing:-0.12px;}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.erIhHf{color:var(--color-text-secondary);font-size:13px;line-height:19px;-webkit-letter-spacing:-0.12px;-moz-letter-spacing:-0.12px;-ms-letter-spacing:-0.12px;letter-spacing:-0.12px;}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.bpPUhY{color:currentColor;text-wrap:balance;font-weight:var(--font-weight-semibold);font-size:40px;line-height:44px;-webkit-letter-spacing:-0.015em;-moz-letter-spacing:-0.015em;-ms-letter-spacing:-0.015em;letter-spacing:-0.015em;}/*!sc*/
@media (max-width:640px){.bpPUhY{font-size:32px;line-height:36px;-webkit-letter-spacing:-0.69px;-moz-letter-spacing:-0.69px;-ms-letter-spacing:-0.69px;letter-spacing:-0.69px;}}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.jGnIx{color:currentColor;font-weight:var(--font-weight-semibold);font-size:24px;line-height:28px;-webkit-letter-spacing:-0.47px;-moz-letter-spacing:-0.47px;-ms-letter-spacing:-0.47px;letter-spacing:-0.47px;}/*!sc*/
@media (max-width:768px){.jGnIx{font-size:21px;line-height:28px;-webkit-letter-spacing:-0.37px;-moz-letter-spacing:-0.37px;-ms-letter-spacing:-0.37px;letter-spacing:-0.37px;}}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.csHZOu{color:currentColor;font-weight:var(--font-weight-semibold);font-size:21px;line-height:28px;-webkit-letter-spacing:-0.37px;-moz-letter-spacing:-0.37px;-ms-letter-spacing:-0.37px;letter-spacing:-0.37px;}/*!sc*/
@media (max-width:768px){.csHZOu{font-size:17px;line-height:20px;-webkit-letter-spacing:-0.22px;-moz-letter-spacing:-0.22px;-ms-letter-spacing:-0.22px;letter-spacing:-0.22px;}}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.hLtvGC{color:var(--color-text-primary);display:-webkit-box;-webkit-box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;overflow-wrap:anywhere;font-size:14px;line-height:21px;-webkit-letter-spacing:-0.18px;-moz-letter-spacing:-0.18px;-ms-letter-spacing:-0.18px;letter-spacing:-0.18px;font-weight:var(--font-weight-medium);}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
.fFZeMR{color:currentColor;font-size:13px;line-height:19px;-webkit-letter-spacing:-0.12px;-moz-letter-spacing:-0.12px;-ms-letter-spacing:-0.12px;letter-spacing:-0.12px;}/*!sc*/
u.sc-8978f737-0{-webkit-text-decoration:underline;text-decoration:underline;-webkit-text-decoration-style:solid;text-decoration-style:solid;-webkit-text-decoration-thickness:1.5px;text-decoration-thickness:1.5px;-webkit-text-decoration-color:var(--color-text-quaternary);text-decoration-color:var(--color-text-quaternary);text-underline-offset:2.5px;}/*!sc*/
data-styled.g15[id="sc-8978f737-0"]{content:"hMxkas,fTnzTg,erIhHf,bpPUhY,jGnIx,csHZOu,hLtvGC,fFZeMR,"}/*!sc*/
.biVEgX{position:fixed;top:0;left:0;right:0;margin-right:var(--removed-body-scroll-bar-size,0);background:var(--color-bg-primary);border-bottom:1px solid var(--color-border-primary);height:var(--header-height);z-index:var(--layer-header);}/*!sc*/
data-styled.g18[id="sc-28207a6c-2"]{content:"biVEgX,"}/*!sc*/
.ibiJBp{position:relative;width:100%;z-index:1;}/*!sc*/
data-styled.g19[id="sc-28207a6c-3"]{content:"ibiJBp,"}/*!sc*/
.bBCxVF{position:relative;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;gap:24px;}/*!sc*/
@media (max-width:640px){.bBCxVF{-webkit-box-pack:justify;-webkit-justify-content:space-between;-ms-flex-pack:justify;justify-content:space-between;}}/*!sc*/
data-styled.g20[id="sc-28207a6c-4"]{content:"bBCxVF,"}/*!sc*/
.bNDfqC{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;margin:0;font-size:14px;font-weight:var(--font-weight-medium);color:var(--color-text-primary);}/*!sc*/
data-styled.g21[id="sc-28207a6c-5"]{content:"bNDfqC,"}/*!sc*/
.jSSMAU.jSSMAU{-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;margin-left:auto;}/*!sc*/
data-styled.g22[id="sc-28207a6c-6"]{content:"jSSMAU,"}/*!sc*/
.emloIa{display:block;width:100%;height:100%;white-space:nowrap;}/*!sc*/
data-styled.g23[id="sc-28207a6c-7"]{content:"emloIa,"}/*!sc*/
.hnIPoq{display:none;}/*!sc*/
.hnIPoq.hnIPoq{margin-left:0;margin-right:calc(-1 * var(--page-padding-right));}/*!sc*/
@media (max-width:640px){.hnIPoq{display:block;}}/*!sc*/
data-styled.g24[id="sc-28207a6c-8"]{content:"hnIPoq,"}/*!sc*/
.klEjtY{height:var(--header-height);min-width:calc(18px + (2 * var(--page-padding-right)));}/*!sc*/
.klEjtY label{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-webkit-justify-content:center;-ms-flex-pack:center;justify-content:center;}/*!sc*/
#mobile-menu-state:target ~ * .sc-28207a6c-9 svg rect:first-of-type,#mobile-menu-state:checked ~ * .klEjtY svg rect:first-of-type{-webkit-transform:rotate(45deg) !important;-ms-transform:rotate(45deg) !important;transform:rotate(45deg) !important;}/*!sc*/
#mobile-menu-state:target ~ * .sc-28207a6c-9 svg rect:last-of-type,#mobile-menu-state:checked ~ * .klEjtY svg rect:last-of-type{-webkit-transform:rotate(-45deg) !important;-ms-transform:rotate(-45deg) !important;transform:rotate(-45deg) !important;}/*!sc*/
data-styled.g25[id="sc-28207a6c-9"]{content:"klEjtY,"}/*!sc*/
.frsHKg{position:fixed;top:0;left:0;width:100%;height:calc(100 * var(--svh));overflow-y:auto;z-index:-1;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:column;-ms-flex-direction:column;flex-direction:column;background:var(--color-bg-primary);padding-bottom:24px;padding-top:calc(var(--header-height) + 32px);padding-left:var(--page-padding-left);padding-right:var(--page-padding-right);-webkit-clip-path:inset(0% 0% 100% 0%);clip-path:inset(0% 0% 100% 0%);-webkit-transition:clip-path var(--mobile-menu-animation-duration) var(--ease-out-quad);transition:clip-path var(--mobile-menu-animation-duration) var(--ease-out-quad);}/*!sc*/
@media (min-width:640){.frsHKg{display:none;}}/*!sc*/
.frsHKg > *{opacity:0;-webkit-transform:translateY(-32px);-ms-transform:translateY(-32px);transform:translateY(-32px);-webkit-transition-timing-function:var(--ease-out-quad);transition-timing-function:var(--ease-out-quad);-webkit-transition-duration:var(--mobile-menu-animation-duration);transition-duration:var(--mobile-menu-animation-duration);-webkit-transition-property:none;transition-property:none;-webkit-transition-delay:0s;transition-delay:0s;}/*!sc*/
@media (max-width:640px){#mobile-menu-state:target ~ * .sc-28207a6c-10,#mobile-menu-state:checked ~ * .frsHKg{-webkit-clip-path:inset(0% 0% 0% 0%);clip-path:inset(0% 0% 0% 0%);}#mobile-menu-state:target ~ * .sc-28207a6c-10 > *,#mobile-menu-state:checked ~ * .frsHKg > *{opacity:1;-webkit-transform:none;-ms-transform:none;transform:none;-webkit-transition-property:opacity,-webkit-transform;-webkit-transition-property:opacity,transform;transition-property:opacity,transform;-webkit-transition-delay:calc(var(--mobile-menu-animation-duration) / 3);transition-delay:calc(var(--mobile-menu-animation-duration) / 3);}}/*!sc*/
data-styled.g26[id="sc-28207a6c-10"]{content:"frsHKg,"}/*!sc*/
.KsLvU{list-style-type:none;display:grid;grid-template-columns:1fr 1fr;grid-auto-flow:row;}/*!sc*/
.KsLvU li{margin:0;}/*!sc*/
.KsLvU li a{padding:8px 0;}/*!sc*/
.KsLvU li a:hover{color:var(--color-text-tertiary);}/*!sc*/
data-styled.g27[id="sc-28207a6c-11"]{content:"KsLvU,"}/*!sc*/
.jZznLV{margin-top:auto;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;gap:16px;}/*!sc*/
data-styled.g28[id="sc-28207a6c-12"]{content:"jZznLV,"}/*!sc*/
.fWYOno{margin-left:auto;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;gap:16px;}/*!sc*/
data-styled.g29[id="sc-28207a6c-13"]{content:"fWYOno,"}/*!sc*/
.gbGOAV{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;height:var(--header-height);-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;color:var(--color-text-primary);}/*!sc*/
data-styled.g30[id="sc-28207a6c-14"]{content:"gbGOAV,"}/*!sc*/
.iGAWSG{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:none;padding:0;margin:0;font:inherit;color:inherit;-webkit-tap-highlight-color:transparent;cursor:pointer;border-radius:6px;padding:4px 8px;margin:0 -8px;-webkit-transition:160ms var(--ease-out-quad);transition:160ms var(--ease-out-quad);-webkit-transition-property:background;transition-property:background;}/*!sc*/
.iGAWSG svg{margin-left:2px;width:14px;height:14px;fill:var(--color-text-secondary);vertical-align:middle;-webkit-transition:160ms var(--ease-out-quad);transition:160ms var(--ease-out-quad);}/*!sc*/
.iGAWSG:hover,.iGAWSG[data-state="open"]{background:var(--color-bg-tertiary);color:var(--color-text-primary);}/*!sc*/
.iGAWSG[data-state="open"] svg{-webkit-transform:rotate(-180deg);-ms-transform:rotate(-180deg);transform:rotate(-180deg);}/*!sc*/
data-styled.g31[id="sc-28207a6c-15"]{content:"iGAWSG,"}/*!sc*/
.jCSimZ{--menu-transition:240ms var(--ease-out-quad);position:absolute;margin:0 auto;top:calc(100% - 4px);left:0;width:100%;-webkit-perspective:2000px;-moz-perspective:2000px;-ms-perspective:2000px;perspective:2000px;z-index:var(--layer-header);-webkit-transform:none;-ms-transform:none;transform:none;-webkit-transition:var(--menu-transition);transition:var(--menu-transition);-webkit-transition-property:-webkit-transform;-webkit-transition-property:transform;transition-property:transform;}/*!sc*/
.jCSimZ[data-transition="false"]{-webkit-transition:none;transition:none;}/*!sc*/
data-styled.g32[id="sc-28207a6c-16"]{content:"jCSimZ,"}/*!sc*/
.bljAEN{position:relative;overflow:hidden;height:var(--radix-navigation-menu-viewport-height);width:var(--radix-navigation-menu-viewport-width);contain:strict;background:var(--color-bg-secondary);border-radius:12px;box-shadow:0 0 0 1px var(--color-border-primary),var(--shadow-high);}/*!sc*/
[data-theme="glass"] .sc-28207a6c-17{--blur:80px;-webkit-backdrop-filter:blur(var(--blur));backdrop-filter:blur(var(--blur));}/*!sc*/
@supports (font:-apple-system-body) and (-webkit-appearance:none){[data-theme="glass"] .sc-28207a6c-17{-webkit-backdrop-filter:none;backdrop-filter:none;}[data-theme="glass"] .sc-28207a6c-17::before{content:"";position:absolute;inset:0;background:var(--color-bg-tertiary);-webkit-animation:jgSSyl 0.16s var(--ease-out-quad) both;animation:jgSSyl 0.16s var(--ease-out-quad) both;}}/*!sc*/
@media (prefers-reduced-motion:no-preference){.bljAEN{-webkit-transform-origin:top left;-ms-transform-origin:top left;transform-origin:top left;-webkit-transition:var(--menu-transition);transition:var(--menu-transition);-webkit-transition-property:width,height;transition-property:width,height;}.sc-28207a6c-16[data-transition="false"] .sc-28207a6c-17{-webkit-transition:none;transition:none;}.bljAEN[data-state="open"]{-webkit-animation:cqekGh 200ms;animation:cqekGh 200ms;}.bljAEN[data-state="closed"]{-webkit-animation:dEXkLd 400ms;animation:dEXkLd 400ms;}}/*!sc*/
data-styled.g33[id="sc-28207a6c-17"]{content:"bljAEN,"}/*!sc*/
.cxOdMO{position:absolute;top:0;left:0;padding:8px;}/*!sc*/
@media (prefers-reduced-motion:no-preference){.cxOdMO{-webkit-animation:var(--menu-transition) both;animation:var(--menu-transition) both;--anim-amount:200px;}.cxOdMO[data-motion="from-start"]{-webkit-animation-name:erMfa-D;animation-name:erMfa-D;}.cxOdMO[data-motion="from-end"]{-webkit-animation-name:knpUHJ;animation-name:knpUHJ;}.cxOdMO[data-motion="to-start"]{-webkit-animation-name:iMzVmT;animation-name:iMzVmT;}.cxOdMO[data-motion="to-end"]{-webkit-animation-name:dshhSx;animation-name:dshhSx;}}/*!sc*/
data-styled.g34[id="sc-28207a6c-18"]{content:"cxOdMO,"}/*!sc*/
.jwaIYb{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:none;padding:0;margin:0;font:inherit;color:inherit;-webkit-tap-highlight-color:transparent;cursor:pointer;background:var(--color-bg-primary);border:1px solid var(--color-border-primary);padding:3px;border-radius:var(--rounded-full);display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;gap:4px;-webkit-transition:120ms;transition:120ms;-webkit-transition-property:border-color;transition-property:border-color;}/*!sc*/
@media (any-hover:hover){.jwaIYb:hover{border-color:var(--color-border-tertiary);}}/*!sc*/
.jwaIYb div{border-radius:var(--rounded-full);height:100%;padding:2px 5px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;color:var(--color-text-tertiary);}/*!sc*/
.jwaIYb div[data-active="true"]{color:var(--color-text-primary);background:var(--color-bg-quaternary);}/*!sc*/
data-styled.g47[id="sc-15e0216c-0"]{content:"jwaIYb,"}/*!sc*/
@-webkit-keyframes erMfa-D{from{-webkit-transform:translateX(calc(-1 * var(--anim-amount)));-ms-transform:translateX(calc(-1 * var(--anim-amount)));transform:translateX(calc(-1 * var(--anim-amount)));opacity:0;}to{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}}/*!sc*/
@keyframes erMfa-D{from{-webkit-transform:translateX(calc(-1 * var(--anim-amount)));-ms-transform:translateX(calc(-1 * var(--anim-amount)));transform:translateX(calc(-1 * var(--anim-amount)));opacity:0;}to{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}}/*!sc*/
data-styled.g117[id="sc-keyframes-erMfa-D"]{content:"erMfa-D,"}/*!sc*/
@-webkit-keyframes knpUHJ{from{-webkit-transform:translateX(var(--anim-amount));-ms-transform:translateX(var(--anim-amount));transform:translateX(var(--anim-amount));opacity:0;}to{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}}/*!sc*/
@keyframes knpUHJ{from{-webkit-transform:translateX(var(--anim-amount));-ms-transform:translateX(var(--anim-amount));transform:translateX(var(--anim-amount));opacity:0;}to{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}}/*!sc*/
data-styled.g118[id="sc-keyframes-knpUHJ"]{content:"knpUHJ,"}/*!sc*/
@-webkit-keyframes iMzVmT{from{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}to{-webkit-transform:translateX(calc(-1 * var(--anim-amount)));-ms-transform:translateX(calc(-1 * var(--anim-amount)));transform:translateX(calc(-1 * var(--anim-amount)));opacity:0;}}/*!sc*/
@keyframes iMzVmT{from{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}to{-webkit-transform:translateX(calc(-1 * var(--anim-amount)));-ms-transform:translateX(calc(-1 * var(--anim-amount)));transform:translateX(calc(-1 * var(--anim-amount)));opacity:0;}}/*!sc*/
data-styled.g119[id="sc-keyframes-iMzVmT"]{content:"iMzVmT,"}/*!sc*/
@-webkit-keyframes dshhSx{from{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}to{-webkit-transform:translateX(var(--anim-amount));-ms-transform:translateX(var(--anim-amount));transform:translateX(var(--anim-amount));opacity:0;}}/*!sc*/
@keyframes dshhSx{from{-webkit-transform:translateX(0);-ms-transform:translateX(0);transform:translateX(0);opacity:1;}to{-webkit-transform:translateX(var(--anim-amount));-ms-transform:translateX(var(--anim-amount));transform:translateX(var(--anim-amount));opacity:0;}}/*!sc*/
data-styled.g120[id="sc-keyframes-dshhSx"]{content:"dshhSx,"}/*!sc*/
@-webkit-keyframes jgSSyl{0%{-webkit-backdrop-filter:none;backdrop-filter:none;}100%{-webkit-backdrop-filter:blur(var(--blur));backdrop-filter:blur(var(--blur));}}/*!sc*/
@keyframes jgSSyl{0%{-webkit-backdrop-filter:none;backdrop-filter:none;}100%{-webkit-backdrop-filter:blur(var(--blur));backdrop-filter:blur(var(--blur));}}/*!sc*/
data-styled.g121[id="sc-keyframes-jgSSyl"]{content:"jgSSyl,"}/*!sc*/
@-webkit-keyframes cqekGh{from{-webkit-transform:scale(0.98);-ms-transform:scale(0.98);transform:scale(0.98);opacity:0;}to{-webkit-transform:scale(1);-ms-transform:scale(1);transform:scale(1);opacity:1;}}/*!sc*/
@keyframes cqekGh{from{-webkit-transform:scale(0.98);-ms-transform:scale(0.98);transform:scale(0.98);opacity:0;}to{-webkit-transform:scale(1);-ms-transform:scale(1);transform:scale(1);opacity:1;}}/*!sc*/
data-styled.g122[id="sc-keyframes-cqekGh"]{content:"cqekGh,"}/*!sc*/
@-webkit-keyframes dEXkLd{from{-webkit-transform:scale(1);-ms-transform:scale(1);transform:scale(1);opacity:1;}to{-webkit-transform:scale(0.98);-ms-transform:scale(0.98);transform:scale(0.98);opacity:0;}}/*!sc*/
@keyframes dEXkLd{from{-webkit-transform:scale(1);-ms-transform:scale(1);transform:scale(1);opacity:1;}to{-webkit-transform:scale(0.98);-ms-transform:scale(0.98);transform:scale(0.98);opacity:0;}}/*!sc*/
data-styled.g123[id="sc-keyframes-dEXkLd"]{content:"dEXkLd,"}/*!sc*/
@-webkit-keyframes lbEHvc{to{opacity:1;-webkit-mask:var(--mask-end);mask:var(--mask-end);}}/*!sc*/
@keyframes lbEHvc{to{opacity:1;-webkit-mask:var(--mask-end);mask:var(--mask-end);}}/*!sc*/
data-styled.g125[id="sc-keyframes-lbEHvc"]{content:"lbEHvc,"}/*!sc*/
.cMOHUV{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;position:relative;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-webkit-justify-content:center;-ms-flex-pack:center;justify-content:center;line-height:0;-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;aspect-ratio:1 / 1;width:18px;height:18px;font-size:9px;}/*!sc*/
data-styled.g148[id="sc-95fd29f1-0"]{content:"cMOHUV,"}/*!sc*/
.isXwKW{border-radius:50%;width:100%;height:100%;pointer-events:none;}/*!sc*/
data-styled.g149[id="sc-7f4a689d-0"]{content:"isXwKW,"}/*!sc*/
.htdaYb{cursor:help;display:-webkit-inline-box;display:-webkit-inline-flex;display:-ms-inline-flexbox;display:inline-flex;}/*!sc*/
data-styled.g152[id="sc-91730d2e-1"]{content:"htdaYb,"}/*!sc*/
.CBiJQ{pointer-events:none;max-width:calc(var(--page-max-width) * 0.7);margin:48px auto;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-webkit-justify-content:center;-ms-flex-pack:center;justify-content:center;}/*!sc*/
data-styled.g278[id="sc-fe71cd7d-0"]{content:"CBiJQ,"}/*!sc*/
.mTUxM{border-radius:16px;}/*!sc*/
data-styled.g279[id="sc-fe71cd7d-1"]{content:"mTUxM,"}/*!sc*/
.kfbLle{padding-top:48px;}/*!sc*/
@media (max-width:640px){.kfbLle{padding-top:24px;}}/*!sc*/
data-styled.g280[id="sc-fe71cd7d-2"]{content:"kfbLle,"}/*!sc*/
</style></head><body><script>!function(){try{var d=document.documentElement,n='data-theme',s='setAttribute';var e=localStorage.getItem('website-theme');if('system'===e||(!e&&false)){var t='(prefers-color-scheme: dark)',m=window.matchMedia(t);if(m.media!==t||m.matches){d.style.colorScheme = 'dark';d[s](n,'dark')}else{d.style.colorScheme = 'light';d[s](n,'light')}}else if(e){d[s](n,e|| '')}else{d[s](n,'dark')}if(e==='light'||e==='dark'||!e)d.style.colorScheme=e||'dark'}catch(e){}}()</script><script>self.__wrap_n=self.__wrap_n||(self.CSS&&CSS.supports("text-wrap","balance")?1:2);self.__wrap_b=(e,t,r)=>{let n=(r=r||document.querySelector(`[data-br="${e}"]`)).parentElement,o=e=>r.style.maxWidth=e+"px";r.style.maxWidth="";let i=n.clientWidth,a=n.clientHeight,s=i/2-.25,u=i+.5,l;if(i){for(o(s),s=Math.max(r.scrollWidth,s);s+1<u;)o(l=Math.round((s+u)/2)),n.clientHeight===a?u=l:s=l;o(u*t+i*(1-t))}r.__wrap_o||"undefined"!=typeof ResizeObserver&&(r.__wrap_o=new ResizeObserver(()=>{self.__wrap_b(0,+r.dataset.brr,r)})).observe(n)};</script><a class="SkipNav_root__DcHPR" href="#skip-nav">Skip to content →</a><div class="Layout_container__BVtmP"><header class="sc-28207a6c-2 biVEgX"><input type="checkbox" id="mobile-menu-state" class="sc-b850125d-0 dZeLXf"/><div class="LayoutContent_root__ZBUya"><nav aria-label="Main" data-orientation="horizontal" dir="ltr" class="sc-28207a6c-3 ibiJBp"><div style="position:relative"><ul data-orientation="horizontal" class="sc-28207a6c-4 bBCxVF contentContainer" aria-label="Site navigation" dir="ltr"><li data-state="closed" style="-webkit-touch-callout:none" class="sc-28207a6c-5 bNDfqC"><a aria-label="Navigate home" data-radix-collection-item="" class="sc-28207a6c-14 gbGOAV Link_root__cNtak" rel="noopener" href="/homepage"><svg height="18" viewBox="0 0 400 100" fill="currentColor" size="18" aria-label="Linear Logotype"><path fill-rule="evenodd" clip-rule="evenodd" d="M12.9266 16.3713c-.5283.5806-.4933 1.4714.0617 2.0265l68.5946 68.5946c.5551.555 1.4459.59 2.0265.0617 10.0579-9.1522 16.3713-22.3478 16.3713-37.0179C99.9807 22.402 77.5788 0 49.9445 0 35.2744 0 22.0788 6.31337 12.9266 16.3713ZM4.35334 29.3894c-.25348.5589-.12567 1.2142.30824 1.6481L68.9432 95.3191c.4339.4339 1.0892.5617 1.6481.3083 1.485-.6736 2.9312-1.4176 4.3344-2.2277.8341-.4815.9618-1.6195.2808-2.3005L8.88146 24.7742c-.68097-.681-1.81894-.5532-2.30045.2808-.81013 1.4032-1.55411 2.8494-2.22767 4.3344ZM.453579 47.796c-.300979-.301-.46112014-.7158-.4327856-1.1405.1327026-1.9891.3816396-3.9463.7400796-5.865.214926-1.1505 1.620727-1.5497 2.448307-.7222L59.9124 96.7715c.8275.8276.4283 2.2334-.7222 2.4483-1.9187.3585-3.8759.6074-5.865.7401-.4247.0283-.8395-.1318-1.1405-.4328L.453579 47.796ZM3.93331 61.7589c-1.0331-1.0331-2.70028-.1429-2.32193 1.2683C6.22104 80.2203 19.7604 93.7597 36.9535 98.3693c1.4112.3784 2.3014-1.2888 1.2683-2.3219L3.93331 61.7589ZM201.602 27.535c3.587 0 6.494-2.918 6.494-6.5175S205.189 14.5 201.602 14.5c-3.586 0-6.493 2.918-6.493 6.5175s2.907 6.5175 6.493 6.5175Zm-55.621 56.8396V14.5039h11.54v59.648h31.115v10.2227h-42.655Zm82.136-28.511v28.511h-11.166V34.8555h11.026v8.4876l.14-.0937c1.121-2.6573 2.928-4.8769 5.42-6.6589 2.491-1.8132 5.668-2.7198 9.531-2.7198 3.426 0 6.54.766 9.344 2.2978 2.803 1.5006 5.045 3.7045 6.727 6.6119 1.682 2.9074 2.523 6.4713 2.523 10.6916v30.9026h-11.166V55.0195c0-3.7514-.997-6.5963-2.99-8.5345-1.962-1.9695-4.594-2.9543-7.896-2.9543-2.118 0-4.049.4377-5.793 1.313-1.744.8754-3.13 2.2196-4.158 4.0328-1.028 1.8132-1.542 4.1422-1.542 6.9871Zm101.105 27.6669c2.554 1.0942 5.482 1.6413 8.783 1.6413 2.71 0 5.03-.3439 6.961-1.0317 1.932-.719 3.52-1.6725 4.766-2.8605 1.277-1.1879 2.289-2.4853 3.037-3.8921h.187v6.9871h10.699V50.2833c0-2.4072-.468-4.6111-1.402-6.6119-.934-2.0008-2.289-3.7358-4.065-5.2051-1.744-1.4694-3.862-2.5948-6.354-3.3763-2.491-.8129-5.295-1.2193-8.409-1.2193-4.267 0-7.958.7347-11.073 2.204-3.084 1.4381-5.497 3.3763-7.242 5.8148-1.744 2.4384-2.694 5.1895-2.85 8.2531h10.793c.124-1.438.623-2.7198 1.495-3.8452.872-1.1254 2.056-2.0008 3.551-2.626 1.495-.6565 3.223-.9848 5.186-.9848 1.962 0 3.628.3283 4.999.9848 1.401.6565 2.476 1.5475 3.223 2.6729.748 1.1254 1.122 2.4384 1.122 3.939v.3752c0 1.1254-.39 1.9538-1.168 2.4853-.748.5314-2.025.9222-3.831 1.1723-1.776.2501-4.205.5471-7.289.891-2.523.2813-4.952.7034-7.288 1.2661-2.336.5627-4.423 1.3912-6.261 2.4853-1.806 1.0942-3.239 2.5479-4.298 4.3611-1.059 1.8132-1.588 4.1422-1.588 6.987 0 3.2826.747 6.0336 2.242 8.2532 1.495 2.1884 3.52 3.8453 6.074 4.9707Zm18.081-8.3001c-1.807.9691-4.034 1.4537-6.681 1.4537-2.679 0-4.813-.5627-6.401-1.6881-1.589-1.1567-2.383-2.7355-2.383-4.7362 0-1.5631.436-2.8293 1.308-3.7984.904-.9691 2.087-1.735 3.551-2.2977 1.464-.5628 3.052-.9535 4.765-1.1724 1.246-.1875 2.461-.3751 3.645-.5627 1.183-.2188 2.289-.422 3.317-.6096 1.028-.2188 1.9-.4377 2.616-.6565.748-.2188 1.293-.4533 1.635-.7034v5.5334c0 1.9382-.451 3.7202-1.355 5.3458-.872 1.5944-2.211 2.8917-4.017 3.8921Zm26.094 9.1442V34.8555h10.745v8.1594h.141c.903-2.8136 2.32-4.955 4.251-6.4244 1.962-1.5005 4.532-2.2508 7.709-2.2508.779 0 1.48.0312 2.102.0938.655.0312 1.2.0625 1.636.0937v10.082c-.405-.0625-1.122-.1406-2.149-.2344-1.028-.0938-2.118-.1407-3.271-.1407-1.838 0-3.519.422-5.046 1.2661-1.526.8441-2.741 2.1415-3.644 3.8921-.872 1.7195-1.308 3.8922-1.308 6.5182v28.4641h-11.166Zm-177.401 0V34.8555h11.166v49.5191h-11.166Zm84.238-2.204c3.582 2.2196 7.834 3.3294 12.755 3.3294 3.8 0 7.257-.6878 10.372-2.0633 3.146-1.4068 5.762-3.3294 7.849-5.7678 2.087-2.4697 3.442-5.3146 4.065-8.5346h-10.512c-.468 1.4693-1.231 2.7667-2.29 3.8921-1.027 1.0942-2.32 1.9539-3.877 2.5792-1.558.6252-3.364.9378-5.42.9378-2.772 0-5.155-.6252-7.148-1.8757-1.962-1.2505-3.457-2.9855-4.485-5.2051-.933-2.043-1.443-4.3564-1.529-6.9402h35.915v-3.0012c0-3.8139-.561-7.284-1.682-10.4102-1.121-3.1575-2.71-5.8773-4.766-8.1594-2.055-2.3134-4.531-4.0953-7.428-5.3458-2.866-1.2505-6.058-1.8757-9.578-1.8757-4.578 0-8.627 1.1098-12.147 3.3294-3.52 2.2196-6.276 5.2833-8.27 9.191-1.993 3.9078-2.99 8.3782-2.99 13.4114 0 5.0019.966 9.4568 2.897 13.3645 1.931 3.8765 4.688 6.9246 8.269 9.1442Zm23.501-32.7783c-1.028-2.1258-2.492-3.767-4.392-4.9237-1.9-1.1567-4.142-1.7351-6.728-1.7351-2.554 0-4.781.5784-6.681 1.7351-1.868 1.1567-3.332 2.7979-4.391 4.9237-.756 1.5396-1.234 3.2903-1.434 5.2521h25.059c-.2-1.9618-.678-3.7125-1.433-5.2521Z"></path></svg></a></li><li class="sc-28207a6c-5 bNDfqC hide-mobile"><button id="radix-:Rkquuuucmla:-trigger-features" data-state="closed" aria-expanded="false" aria-controls="radix-:Rkquuuucmla:-content-features" class="sc-28207a6c-15 iGAWSG" data-radix-collection-item="">Features <svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M4.53 5.47a.75.75 0 0 0-1.06 1.06l4 4a.75.75 0 0 0 1.054.007l4-3.903a.75.75 0 0 0-1.048-1.073l-3.47 3.385L4.53 5.47Z"></path></svg></button></li><li class="sc-28207a6c-5 bNDfqC hide-laptop"><a href="/method" data-radix-collection-item="" data-text="Method" class="sc-28207a6c-7 emloIa">Method</a></li><li class="sc-28207a6c-5 bNDfqC hide-laptop"><a href="/customers" data-radix-collection-item="" data-text="Customers" class="sc-28207a6c-7 emloIa">Customers</a></li><li class="sc-28207a6c-5 bNDfqC hide-tablet"><a href="/changelog" data-radix-collection-item="" data-text="Changelog" class="sc-28207a6c-7 emloIa">Changelog</a></li><li class="sc-28207a6c-5 bNDfqC hide-mobile"><a href="/pricing" data-radix-collection-item="" data-text="Pricing" class="sc-28207a6c-7 emloIa">Pricing</a></li><li class="sc-28207a6c-5 bNDfqC hide-mobile"><button id="radix-:Rkquuuucmla:-trigger-company" data-state="closed" aria-expanded="false" aria-controls="radix-:Rkquuuucmla:-content-company" class="sc-28207a6c-15 iGAWSG" data-radix-collection-item="">Company <svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M4.53 5.47a.75.75 0 0 0-1.06 1.06l4 4a.75.75 0 0 0 1.054.007l4-3.903a.75.75 0 0 0-1.048-1.073l-3.47 3.385L4.53 5.47Z"></path></svg></button></li><li class="sc-28207a6c-5 bNDfqC hide-mobile"><a href="/contact" data-radix-collection-item="" data-text="Contact" class="sc-28207a6c-7 emloIa">Contact</a></li><li data-show="logged-in" class="sc-28207a6c-5 sc-28207a6c-6 bNDfqC jSSMAU hide-mobile"><a data-radix-collection-item="" class="Link_root__cNtak" rel="noopener" href="/docs">Docs</a></li><li data-show="logged-in" class="sc-28207a6c-5 bNDfqC hide-mobile"><a type="button" data-header-cta="" data-radix-collection-item="" data-hide="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/signup">Sign up</a><a type="button" data-header-cta="" data-radix-collection-item="" data-show="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/login">Open app <svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M5.46967 11.4697C5.17678 11.7626 5.17678 12.2374 5.46967 12.5303C5.76256 12.8232 6.23744 12.8232 6.53033 12.5303L10.5303 8.53033C10.8207 8.23999 10.8236 7.77014 10.5368 7.47624L6.63419 3.47624C6.34492 3.17976 5.87009 3.17391 5.57361 3.46318C5.27713 3.75244 5.27128 4.22728 5.56054 4.52376L8.94583 7.99351L5.46967 11.4697Z"></path></svg></a></li><li data-hide="logged-in" class="sc-28207a6c-5 sc-28207a6c-6 bNDfqC jSSMAU hide-mobile"><a data-radix-collection-item="" class="Link_root__cNtak" rel="noopener" href="/login">Log in</a></li><li data-hide="logged-in" class="sc-28207a6c-5 bNDfqC hide-mobile"><a type="button" data-header-cta="" data-radix-collection-item="" data-hide="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/signup">Sign up</a><a type="button" data-header-cta="" data-radix-collection-item="" data-show="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/login">Open app <svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M5.46967 11.4697C5.17678 11.7626 5.17678 12.2374 5.46967 12.5303C5.76256 12.8232 6.23744 12.8232 6.53033 12.5303L10.5303 8.53033C10.8207 8.23999 10.8236 7.77014 10.5368 7.47624L6.63419 3.47624C6.34492 3.17976 5.87009 3.17391 5.57361 3.46318C5.27713 3.75244 5.27128 4.22728 5.56054 4.52376L8.94583 7.99351L5.46967 11.4697Z"></path></svg></a></li><li class="sc-28207a6c-5 sc-28207a6c-8 bNDfqC hnIPoq"><div class="sc-b850125d-1 cVJRDH sc-28207a6c-9 klEjtY"><a href="#mobile-menu-state" role="button" class="sc-b850125d-2 fCOUsV"><span class="sc-b850125d-5 cdFcqp">Open <!-- -->Site Navigation<!-- --> Menu</span></a><a href="#" role="button" class="sc-b850125d-2 fCOUsV"><span class="sc-b850125d-5 cdFcqp">Close <!-- -->Site Navigation<!-- --> Menu</span></a><label for="mobile-menu-state" aria-hidden="true" class="sc-b850125d-3 gRfBMV"><svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor" xmlns="http://www.w3.org/2000/svg"><rect x="1" y="7.5" width="14" height="1" rx="0.5" style="transform-origin:center;transition:160ms var(--ease-out-quad);transform:translateY(-3.5px)"></rect><rect x="1" y="7.5" width="14" height="1" rx="0.5" style="transform-origin:center;transition:160ms var(--ease-out-quad);transform:translateY(3.5px)"></rect></svg></label></div></li></ul></div><div style="--mobile-menu-animation-duration:400ms" class="sc-b850125d-4 bkPaLG"><div class="sc-28207a6c-10 frsHKg"><ul class="sc-28207a6c-11 KsLvU"><li><a href="/features" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Features</span></a></li><li><a href="/pricing" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Pricing</span></a></li><li><a href="/method" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Method</span></a></li><li><a href="/contact" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Contact</span></a></li><li><a href="/customers" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Customers</span></a></li><li><a href="/about" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">About</span></a></li><li><a href="/changelog" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Changelog</span></a></li><li><a data-active="" aria-current="page" href="/blog" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Blog</span></a></li><li><a href="/integrations" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Integrations</span></a></li><li><a href="/careers" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Careers</span></a></li><li><a href="/docs" data-radix-collection-item="" class="sc-28207a6c-7 emloIa"><span class="sc-8978f737-0 hMxkas">Docs</span></a></li></ul><div class="sc-28207a6c-12 jZznLV"><a type="button" data-header-cta="" data-show="logged-in" data-hide="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/signup">Sign up</a><a type="button" data-header-cta="" data-show="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/login">Open app <svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M5.46967 11.4697C5.17678 11.7626 5.17678 12.2374 5.46967 12.5303C5.76256 12.8232 6.23744 12.8232 6.53033 12.5303L10.5303 8.53033C10.8207 8.23999 10.8236 7.77014 10.5368 7.47624L6.63419 3.47624C6.34492 3.17976 5.87009 3.17391 5.57361 3.46318C5.27713 3.75244 5.27128 4.22728 5.56054 4.52376L8.94583 7.99351L5.46967 11.4697Z"></path></svg></a><a type="button" data-header-cta="" data-hide="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/signup">Sign up</a><a type="button" data-header-cta="" data-hide="logged-in" data-show="logged-in" class="Button_root__Stmhv Button_kind-primary__oPUio Button_size-small__sjtMt Button_kind__Z_LDa Link_root__cNtak" rel="noopener" href="/login">Open app <svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M5.46967 11.4697C5.17678 11.7626 5.17678 12.2374 5.46967 12.5303C5.76256 12.8232 6.23744 12.8232 6.53033 12.5303L10.5303 8.53033C10.8207 8.23999 10.8236 7.77014 10.5368 7.47624L6.63419 3.47624C6.34492 3.17976 5.87009 3.17391 5.57361 3.46318C5.27713 3.75244 5.27128 4.22728 5.56054 4.52376L8.94583 7.99351L5.46967 11.4697Z"></path></svg></a><a href="/login" type="button" data-hide="logged-in" class="Button_root__Stmhv Button_kind-secondary__C4t01 Button_size-small__sjtMt Button_kind__Z_LDa">Log in</a><div class="sc-28207a6c-13 fWYOno"><a href="https://twitter.com/linear" target="_blank" rel="noopener noreferrer" aria-label="Twitter"><svg class="" style="--icon-color:var(--color-text-tertiary)" width="16" height="16" viewBox="0 0 16 16" fill="var(--color-text-tertiary)" role="img" focusable="false" aria-hidden="true"><path fill="var(--icon-color, #1da1f2)" d="M16 3.42a6.239 6.239 0 0 1-1.767.477 3.046 3.046 0 0 0 1.353-1.676c-.595.347-1.253.6-1.954.735A3.099 3.099 0 0 0 11.386 2c-1.7 0-3.078 1.356-3.078 3.03 0 .237.028.467.08.69a8.786 8.786 0 0 1-6.344-3.166 2.992 2.992 0 0 0-.416 1.523c0 1.05.544 1.979 1.37 2.522a3.108 3.108 0 0 1-1.395-.38v.04c0 1.467 1.06 2.692 2.469 2.97a3.146 3.146 0 0 1-1.39.052 3.073 3.073 0 0 0 2.875 2.103 6.238 6.238 0 0 1-3.822 1.297c-.249 0-.494-.014-.735-.042A8.804 8.804 0 0 0 5.717 14c5.661 0 8.756-4.616 8.756-8.619 0-.13-.003-.261-.009-.391A6.205 6.205 0 0 0 16 3.423L16 3.42Z"></path></svg></a><a href="https://github.com/linear" target="_blank" rel="noopener noreferrer" aria-label="GitHub"><svg class="" style="--icon-color:var(--color-text-tertiary)" width="16" height="16" viewBox="0 0 16 16" fill="var(--color-text-tertiary)" role="img" focusable="false" aria-hidden="true"><path fill="var(--icon-color, #ffffff)" d="M11.5127 1.96272C10.4401 1.32091 9.27013 1 7.99946 1C6.72987 1 5.5588 1.32091 4.48573 1.96272C3.41428 2.60397 2.56465 3.47445 1.939 4.57358C1.31228 5.67383 1 6.87359 1 8.17567C1 9.73883 1.44527 11.1443 2.33528 12.3933C3.22529 13.64 4.37482 14.5066 5.78494 14.9846C5.94862 15.0159 6.0703 14.9941 6.14891 14.9192C6.22806 14.8437 6.26736 14.7515 6.26736 14.6396L6.26306 14.1337C6.25983 13.8161 6.25767 13.5394 6.25767 13.3023L6.04877 13.3392C5.91524 13.3644 5.74617 13.375 5.54265 13.3717C5.33967 13.3689 5.12807 13.3471 4.90947 13.3063C4.69249 13.266 4.4895 13.1732 4.30105 13.0267C4.11261 12.8802 3.97908 12.6885 3.89993 12.452L3.8084 12.2362C3.7481 12.0931 3.65226 11.9343 3.52196 11.761C3.39112 11.5854 3.25921 11.4669 3.12568 11.4049L3.06107 11.3585C3.01908 11.3271 2.98031 11.2902 2.94262 11.2466C2.90655 11.203 2.87962 11.1594 2.86186 11.1158C2.84355 11.0711 2.85809 11.0359 2.90655 11.0079C2.955 10.9788 3.0433 10.966 3.17037 10.966L3.35343 10.9939C3.47404 11.0185 3.6248 11.0935 3.80409 11.2176C3.98339 11.3428 4.13091 11.5044 4.2456 11.704C4.38559 11.9589 4.55411 12.1535 4.75171 12.2876C4.94931 12.4218 5.14853 12.4889 5.34936 12.4889C5.54857 12.4889 5.72248 12.4721 5.86839 12.4419C6.01377 12.4106 6.15107 12.3637 6.27759 12.3022C6.33359 11.884 6.48219 11.5631 6.72448 11.3389C6.37882 11.3009 6.06761 11.245 5.7914 11.17C5.51465 11.0957 5.22929 10.9744 4.93531 10.8055C4.63918 10.6378 4.39528 10.4298 4.20091 10.1794C4.00708 9.93227 3.84717 9.60353 3.72333 9.201C3.59949 8.79456 3.5365 8.32885 3.5365 7.79885C3.5365 7.04579 3.77556 6.40397 4.25583 5.87397C4.03077 5.30708 4.05123 4.67197 4.32044 3.96755C4.49542 3.91164 4.75656 3.95357 5.1033 4.09334C5.4495 4.2331 5.70256 4.3533 5.86355 4.45226C6.02507 4.55289 6.1543 4.63675 6.25121 4.70496C6.81494 4.54618 7.39751 4.46624 7.99946 4.46624C8.60142 4.46624 9.18399 4.5473 9.74933 4.70887L10.0955 4.48413C10.3324 4.33485 10.6124 4.199 10.9344 4.07321C11.2574 3.9491 11.504 3.91499 11.6742 3.9709C11.9472 4.67533 11.9719 5.30987 11.7469 5.87733C12.2261 6.40621 12.4662 7.04802 12.4662 7.80165C12.4662 8.33053 12.4037 8.79959 12.2794 9.20771C12.1555 9.61583 11.994 9.94457 11.7959 10.19C11.5988 10.436 11.3533 10.6428 11.0582 10.8117C10.7637 10.9794 10.4767 11.1007 10.2021 11.1751C9.92539 11.25 9.61526 11.3059 9.26744 11.3428C9.58349 11.6223 9.74125 12.064 9.74125 12.6695V14.6413C9.74125 14.7531 9.77894 14.8459 9.8554 14.9208C9.93077 14.9952 10.0508 15.0176 10.2151 14.9863C11.6257 14.5055 12.7747 13.6423 13.6647 12.3922C14.5547 11.1454 15 9.73827 15 8.17567C15 6.87415 14.6866 5.67383 14.061 4.57414C13.4348 3.475 12.5857 2.60453 11.5143 1.96328L11.5127 1.96272Z"></path></svg></a><a href="https://linear.app/join-slack" target="_blank" rel="noopener noreferrer" aria-label="Slack"><svg class="" style="--icon-color:var(--color-text-tertiary)" width="16" height="16" viewBox="0 0 16 16" fill="var(--color-text-tertiary)" role="img" focusable="false" aria-hidden="true"><path d="M3.95144 9.85024C3.95144 10.6639 3.29432 11.3216 2.48128 11.3216C1.66823 11.3216 1.01111 10.6639 1.01111 9.85024C1.01111 9.03655 1.66823 8.37891 2.48128 8.37891H3.95144V9.85024ZM4.68653 9.85024C4.68653 9.03655 5.34365 8.37891 6.15669 8.37891C6.96974 8.37891 7.62686 9.03655 7.62686 9.85024V13.5286C7.62686 14.3423 6.96974 14.9999 6.15669 14.9999C5.34365 14.9999 4.68653 14.3423 4.68653 13.5286V9.85024Z" fill="var(--icon-color, #E01E5A)"></path><path d="M9.84333 12.0573C10.6564 12.0573 11.3135 12.7149 11.3135 13.5286C11.3135 14.3423 10.6564 14.9999 9.84333 14.9999C9.03029 14.9999 8.37317 14.3423 8.37317 13.5286V12.0573H9.84333ZM9.84333 11.3216C9.03029 11.3216 8.37317 10.6639 8.37317 9.85024C8.37317 9.03655 9.03029 8.37891 9.84333 8.37891H13.5299C14.3429 8.37891 15.0001 9.03655 15.0001 9.85024C15.0001 10.6639 14.3429 11.3216 13.5299 11.3216H9.84333Z" fill="var(--icon-color, #ECB22E)"></path><path d="M12.0486 6.16083C12.0486 5.34713 12.7057 4.68949 13.5188 4.68949C14.3318 4.68949 14.9889 5.34713 14.9889 6.16083C14.9889 6.97452 14.3318 7.63217 13.5188 7.63217H12.0486V6.16083ZM11.3135 6.16083C11.3135 6.97452 10.6564 7.63217 9.84334 7.63217C9.03029 7.63217 8.37317 6.97452 8.37317 6.16083V2.47134C8.37317 1.65764 9.03029 1 9.84334 1C10.6564 1 11.3135 1.65764 11.3135 2.47134V6.16083Z" fill="var(--icon-color, #2EB67D)"></path><path d="M6.15672 3.94268C5.34368 3.94268 4.68655 3.28503 4.68655 2.47134C4.68655 1.65764 5.34368 1 6.15672 1C6.96977 1 7.62689 1.65764 7.62689 2.47134V3.94268H6.15672ZM6.15672 4.68949C6.96977 4.68949 7.62689 5.34713 7.62689 6.16083C7.62689 6.97452 6.96977 7.63217 6.15672 7.63217H2.47017C1.65712 7.63217 1 6.97452 1 6.16083C1 5.34713 1.65712 4.68949 2.47017 4.68949H6.15672Z" fill="var(--icon-color, #36C5F0)"></path></svg></a><a href="https://www.youtube.com/@LinearApp" target="_blank" rel="noopener noreferrer" aria-label="YouTube"><svg class="" style="--icon-color:var(--color-text-tertiary)" width="16" height="16" viewBox="0 0 16 16" fill="var(--color-text-tertiary)" role="img" focusable="false" aria-hidden="true"><path d="M13.4527 3.40075C14.0558 3.56204 14.5262 4.03428 14.6893 4.63734C15.1159 6.35783 15.0881 9.61061 14.6983 11.358C14.537 11.961 14.0647 12.4315 13.4617 12.5946C11.7591 13.0157 4.13336 12.9638 2.52936 12.5946C1.92629 12.4333 1.45584 11.961 1.29275 11.358C0.890404 9.71814 0.918183 6.2503 1.28379 4.6463C1.44508 4.04324 1.91733 3.5728 2.5204 3.40971C4.79647 2.93478 12.6426 3.08801 13.4527 3.40075ZM6.64251 5.90007V10.0937L10.2986 7.9969L6.64251 5.90007Z"></path></svg></a></div></div></div></div><div data-transition="false" class="sc-28207a6c-16 jCSimZ"></div></nav></div></header><main class="Layout_content__PrPCk"><div id="skip-nav"></div><div class="sc-fe71cd7d-2 kfbLle LayoutContent_root__ZBUya"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_justify-space-between__Txsv7"><a class="Article_backLink__lmkbt" href="/blog"><svg class="" style="--icon-color:currentColor" width="16" height="16" viewBox="0 0 16 16" fill="currentColor" role="img" focusable="false" aria-hidden="true"><path d="M10.7803 4.78033C11.0732 4.48744 11.0732 4.01256 10.7803 3.71967C10.4874 3.42678 10.0126 3.42678 9.71967 3.71967L5.71967 7.71967C5.42933 8.01001 5.42643 8.47986 5.71318 8.77376L9.61581 12.7738C9.90508 13.0702 10.3799 13.0761 10.6764 12.7868C10.9729 12.4976 10.9787 12.0227 10.6895 11.7262L7.30417 8.25649L10.7803 4.78033Z"></path></svg>Blog</a><div class="hide-mobile Flex_root__DOQCW Flex_align-center__oXK9F Flex_justify-center__GW4P8" style="gap:8px"><span class="sc-8978f737-0 fTnzTg">Last edited: <time dateTime="2024-01-30T15:15:09.574Z">January 30, 2024</time></span><a data-state="closed" class="sc-91730d2e-1 htdaYb"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_inline__8Gx2p" style="gap:8px"><div class="sc-95fd29f1-0 cMOHUV"><img src="https://cdn.sanity.io/images/ornj730p/production/dff9dd98ea36ac93a38ef6405abc866dda659f68-492x492.jpg?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2" width="18" height="18" alt="Avatar of Jori Lallo" class="sc-7f4a689d-0 isXwKW"/></div></div></a><a data-state="closed" class="sc-91730d2e-1 htdaYb"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_inline__8Gx2p" style="gap:8px"><div class="sc-95fd29f1-0 cMOHUV"><img src="https://cdn.sanity.io/images/ornj730p/production/4d081a7a9ee2575014ab150f2d74cc3e2266a937-300x300.jpg?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2" width="18" height="18" alt="Avatar of Tuomas Artman" class="sc-7f4a689d-0 isXwKW"/></div></div></a><a data-state="closed" class="sc-91730d2e-1 htdaYb"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_inline__8Gx2p" style="gap:8px"><div class="sc-95fd29f1-0 cMOHUV"><img src="https://cdn.sanity.io/images/ornj730p/production/c5229881e58cfb3a30c31b036e2965f9f327dcc8-512x512.png?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2" width="18" height="18" alt="Avatar of Tom Moor" class="sc-7f4a689d-0 isXwKW"/></div></div></a></div><button data-state="closed" class="sc-15e0216c-0 jwaIYb"><div><svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor"><path d="M11.3907 10.1063C8.11878 10.1063 6.02726 8.05327 6.02726 4.77878C6.02726 4.09975 6.19047 3.13453 6.40679 2.62716C6.46202 2.49235 6.46974 2.40113 6.46974 2.34328C6.46974 2.18506 6.34882 2 6.11494 2C6.05067 2 5.91076 2.01413 5.76953 2.06806C3.53361 2.9611 2.03345 5.36441 2.03345 7.90669C2.03345 11.4585 4.74167 14 8.28971 14C10.8935 14 13.1548 12.4214 13.9062 10.4502C13.9538 10.3102 13.9666 10.169 13.9666 10.1112C13.9666 9.8875 13.7841 9.73961 13.6117 9.73961C13.5373 9.73961 13.4692 9.75375 13.355 9.78712C12.8866 9.94023 12.1361 10.1063 11.3907 10.1063Z"></path></svg></div><div><svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M8.75881 14.2396C8.75881 14.6593 8.41855 14.9996 7.99882 14.9996C7.57908 14.9996 7.23881 14.6593 7.23881 14.2396V12.7596C7.23881 12.3398 7.57908 11.9996 7.99882 11.9996C8.41855 11.9996 8.75881 12.3398 8.75881 12.7596V14.2396Z"></path><path d="M5.17157 10.8267C4.87477 10.5299 4.39357 10.5299 4.09677 10.8267L3.05025 11.8733C2.75345 12.1701 2.75345 12.6513 3.05025 12.9481C3.34705 13.2449 3.82825 13.2449 4.12505 12.9481L5.17157 11.9015C5.46837 11.6047 5.46837 11.1235 5.17157 10.8267Z"></path><path d="M10.8286 10.8266C10.5318 11.1234 10.5318 11.6046 10.8286 11.9014L11.8751 12.9479C12.1719 13.2447 12.6531 13.2447 12.9499 12.9479C13.2467 12.6511 13.2467 12.1699 12.9499 11.8731L11.9034 10.8266C11.6066 10.5298 11.1254 10.5298 10.8286 10.8266Z"></path><path d="M10.1144 5.8839C11.286 7.05547 11.286 8.95497 10.1144 10.1265C8.94285 11.2981 7.04335 11.2981 5.87178 10.1265C4.70021 8.95497 4.70021 7.05547 5.87178 5.8839C7.04335 4.71233 8.94285 4.71233 10.1144 5.8839Z"></path><path d="M3.05042 4.1232C2.75362 3.8264 2.75362 3.3452 3.05042 3.0484C3.34722 2.7516 3.82843 2.7516 4.12523 3.0484L5.17174 4.09492C5.46854 4.39172 5.46854 4.87292 5.17174 5.16972C4.87495 5.46652 4.39374 5.46652 4.09694 5.16972L3.05042 4.1232Z"></path><path d="M11.8749 3.04856C12.1717 2.75176 12.6529 2.75176 12.9497 3.04856C13.2465 3.34536 13.2465 3.82656 12.9497 4.12336L11.9032 5.16988C11.6064 5.46668 11.1252 5.46668 10.8284 5.16988C10.5316 4.87308 10.5316 4.39188 10.8284 4.09508L11.8749 3.04856Z"></path><path d="M14.2394 7.2389C14.6592 7.2389 14.9994 7.57916 14.9994 7.9989C14.9994 8.41864 14.6592 8.7589 14.2394 8.7589H12.7594C12.3397 8.7589 11.9994 8.41864 11.9994 7.9989C11.9994 7.57916 12.3397 7.2389 12.7594 7.2389H14.2394Z"></path><path d="M4 7.9983C4 7.57856 3.65974 7.2383 3.24 7.2383H1.76C1.34026 7.2383 1 7.57856 1 7.99829C1 8.41803 1.34026 8.7583 1.76 8.7583H3.24C3.65974 8.7583 4 8.41803 4 7.9983Z"></path><path d="M7.99821 4.00012C8.41795 4.00012 8.75821 3.65986 8.75821 3.24012V1.76012C8.75821 1.34039 8.41795 1.00012 7.99821 1.00012C7.57847 1.00012 7.23821 1.34039 7.23821 1.76012V3.24012C7.23821 3.65986 7.57847 4.00012 7.99821 4.00012Z"></path></svg></div></button></div><div class="sc-fe71cd7d-0 CBiJQ"><div class="hide-dark"><img alt="The linear logo with a warning sign" data-nosnippet="true" data-loaded="false" fetchPriority="high" width="2056" height="944" decoding="async" data-nimg="1" class="sc-cf331c07-0 cJUdsU sc-fe71cd7d-1 mTUxM" style="color:transparent" src="https://cdn.sanity.io/images/ornj730p/production/2e281ae34028ff175a63af027de95bca66b9bcb1-2056x944.png?q=95&amp;auto=format&amp;dpr=2"/></div><div class="hide-light"><img alt="The linear logo with a warning sign" data-nosnippet="true" data-loaded="false" fetchPriority="high" width="2056" height="944" decoding="async" data-nimg="1" class="sc-cf331c07-0 cJUdsU sc-fe71cd7d-1 mTUxM" style="color:transparent" src="https://cdn.sanity.io/images/ornj730p/production/e078697b78690170e26ec6ed13fd77483d5269c4-2056x944.png?q=95&amp;auto=format&amp;dpr=2"/></div></div><div kind="prose" class="LayoutContent_root__ZBUya LayoutContent_prose__Eddy0"><span style="display:block" class="sc-8978f737-0 erIhHf">News</span><div class="Spacer_root__uoSvA" style="--height:12px"></div><h1 class="sc-8978f737-0 bpPUhY"><span data-br=":Rctauuuucmla:" data-brr="1" style="display:inline-block;vertical-align:top;text-decoration:inherit;text-wrap:balance">Post mortem on Linear incident from Jan 24th, 2024</span><script>self.__wrap_n!=1&&self.__wrap_b(":Rctauuuucmla:",1)</script></h1><div class="Spacer_root__uoSvA" style="--height:24px"></div></div><div class="Prose_prose__fW4cc"><p>On Wednesday January 24, Linear experienced a temporary <a href="https://linearstatus.com/incidents/01HMX98N26X50PZR72156MN1F4">data loss incident</a> from 04:47 to 09:56 UTC (about five hours) due to restoration from backup. This affected workspaces that made changes during this window or updated via automated processes (e.g. cycle automation), especially those in the EU due to the timing.</p><p>As part of the disaster recovery process, we took Linear offline for one hour, impacting all users. We restored over 99% of lost data within 36h of the start of the incident. But, in some cases we couldn&#x27;t restore data due to conflicts.</p><p>A database migration caused the incident, which accidentally deleted data from production servers. We put Linear into maintenance mode and reverted the database to a backup taken a few hours prior. We immediately began restoring missing data, which took two days to complete.</p><p>We’re deeply sorry for the inconvenience and frustration this incident caused for our customers. Reliability and security are top priorities for us at Linear and last week we fell short from our promise to our users and ourselves. In learning from this incident, we&#x27;ll improve our tools and processes to prevent such mistakes and recover more quickly.</p><p>Below is a timeline of the incident, and more details around its cause and the ultimate fix.</p><h2 class="sc-8978f737-0 jGnIx"><a data-id="incident-timeline" id="incident-timeline" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#incident-timeline"><strong>Incident timeline</strong></a></h2><p><em>All times are in Coordinated Universal Time (UTC)</em></p><h3 class="sc-8978f737-0 csHZOu"><a data-id="january-24" id="january-24" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#january-24"><strong>January 24</strong></a></h3><ul><li><strong>04:47</strong>: Full database backup completed (pre-incident).</li><li><strong>07:01</strong>: Merge of migration that caused data loss.</li><li><strong>07:20</strong>: Migration completed.</li><li><strong>07:52</strong>: Strange notifications noticed in the app, engineering and support notified to check other reports.</li><li><strong>08:10</strong>: Critical incident initiated with Zoom call to investigate, additional engineers paged.</li><li><strong>08:36</strong>: Update posted to Linear status page and shared on X: &quot;<a href="https://x.com/linear/status/1750074910456602761?s=20">Investigating data access issues</a>&quot;</li><li><strong>09:20</strong>: Additional update posted on Linear <a href="https://linearstatus.com/">status site</a>.</li><li><strong>09:56</strong>: Linear put in maintenance mode to prevent further changes and recover from backup.</li><li><strong>10:48</strong>: Linear access restored with the database backup from 04:47.</li><li><strong>11:09</strong>: Status site updated to monitoring.</li><li><strong>11:30</strong>: Restoration started for all data entered between 04:47 and 09:56.</li><li><strong>13:50</strong>: Emails sent to customers who had created workspaces between 04:47 and 09:56 asking them to recreate their workspace as we could not rebuild it.</li><li><strong>15:35</strong>: Emails sent to affected users and workspace administrators with information about the incident and restoration process.</li></ul><h3 class="sc-8978f737-0 csHZOu"><a data-id="january-25" id="january-25" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#january-25"><strong>January 25</strong></a></h3><ul><li><strong>14:00</strong>: Data recovery page published to application settings and shared with admins via email.</li><li><strong>14:25</strong>: Bug fixed with the data recovery page by forcing clients to refresh (which then overloaded our API and caused application loading issues).</li><li><strong>16:40</strong>: Dry-runs of the restoration process started.</li><li><strong>17:49</strong>: Workspace data restoration started.</li><li><strong>19:48</strong>: 98% of affected workspaces restored.</li><li><strong>23:20</strong>: 99% of affected workspaces restored. </li></ul><h3 class="sc-8978f737-0 csHZOu"><a data-id="january-26" id="january-26" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#january-26"><strong>January 26</strong></a></h3><ul><li><strong>07:37</strong>: Restoration for all workspaces except one completed.</li><li><strong>08:39</strong>: Restoration of final workspace completed.</li></ul><h2 class="sc-8978f737-0 jGnIx"><a data-id="what-happened" id="what-happened" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#what-happened"><strong>What happened</strong></a></h2><p>Linear uses trunk-based development and changes land in production on an ongoing basis as part of new feature development. We gate execution using feature flags.</p><p>While developing new features, we created two new tables in the production database. We also filled these tables with data from existing tables in preparation for rolling out the new functionality. We considered the data faulty and planned a new migration to update it. We had to drop the existing data in the new tables as part of that migration.</p><p>A pull request was created, reviewed and accepted that added a database migration that first dropped data in the new tables. Then, it copied data from existing tables to the new ones. The new tables were only used by engineers developing the feature and not used by actual users yet, so we deemed deletion safe. We used the following SQL statement for data deletion:</p><p><code>TRUNCATE TABLE &lt;new_table&gt; CASCADE;</code></p><p>The intention here was to delete any data in the new table as well as any rows of test data with foreign keys pointing to the new table. However, <code>CASCADE</code> simply truncates the entire contents of any tables that have foreign keys to the table.</p><p>This caused the full deletion of production data for issue and document descriptions, comments, notifications, favorites, and reactions.</p><p>When pull requests include migration, we do include CI warnings for dangerous migrations, like large indexing operations. The pull request was tested locally and reviewed by multiple engineers but the cascade operation was missed by both author and peer reviewers.</p><h2 class="sc-8978f737-0 jGnIx"><a data-id="outage-and-investigation" id="outage-and-investigation" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#outage-and-investigation"><strong>Outage and investigation</strong></a></h2><p>With the data deleted, symptoms weren&#x27;t immediately visible to end-users, including our own team. Linear stores most workspace data in a local client cache, and there’s also a database cache which sits in front of the Postgres database. Our <a href="https://linear.app/blog/scaling-the-linear-sync-engine">sync engine</a> updates this database cache programmatically to add and remove &quot;sync&quot; packets describing database mutations.</p><p>As this is part of the application logic, deleting rows directly from the database won’t create sync packets and thus won’t update the cache or clients. This meant users would continue to see and receive data for the deleted tables in the Postgres database until they reloaded their local data and the backend cache was invalidated, which usually takes 24 hours. Data caching was also one of the primary reasons the problem was missed in local development.</p><p>The data loss went unnoticed for 30 minutes due to the multiple layers of cache involved. It was possible to reload the Linear client and not see any visible problems at all. Only mutations caused errors due to underlying database entities no longer existing.</p><p>The outage first became apparent with notifications, as it is one of our most frequently updated tables, with notifications created and deleted based on many events. There is always a small lag in synchronization state between the backend and its clients. We often see the client trying to update deleted entities and see these as a <em>synchronization lag </em>warning. As an expected part of the system these did not count towards error metrics or trigger alert monitors.</p><p>An engineer started an internal incident to investigate when they saw problems while using their own notification inbox. User reports confirmed the synchronization issues that the problems pointed to. This happened around 30 minutes after the migration. This caused a critical incident to start and personnel to be paged.</p><p>The cause wasn’t clear at first due to the caching. But, a spike in warnings from clients to the <em>notificationUpdate</em> GraphQL endpoint offered a clue about when the problem started. The faulty commit was quickly correlated. Our engineers verified that the commit did cause similar symptoms when run locally. Shortly thereafter, an engineer found all notifications had been deleted in their local database. At this point, production data was checked and we found multiple tables had been entirely deleted.</p><p>Linear was promptly taken into maintenance mode, which blocks any updates and presents a maintenance screen for all users. We created a new backup to capture all the sync packets created up until that point. Then the database was restored from the latest backup from before the faulty migration ran.</p><p>Linear takes daily full backups in addition to having point-in-time recovery. The last backup was taken at 04:47 UTC and the bad commit landed at 07:01 UTC. With point-in-time recovery engineering could have restored services to the state they were in at 07:01. However, the engineering team had never tested point-in-time recovery, nor had engineering developed tooling to quickly restore the database using point-in-time recovery, so that option wasn’t considered. Restoring from a full backup was a frequently tested procedure and thus executed to get the application up and running as quickly as possible, while initially losing more data than point-in-time recovery.</p><p>After backup restoration was completed, Linear was brought back online again with data as of the 04:47 UTC backup.</p><p>The service recovered somewhat quickly, with intermittent problems affecting synchronization. The sync service uses a cursor to get changes not yet sent to clients and caches this cursor in Redis. However, the code did not account for the ID moving backwards, as was the case with a full database restore. Resetting the cache and restarting the sync service resolved these issues.</p><h2 class="sc-8978f737-0 jGnIx"><a data-id="data-restoration" id="data-restoration" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#data-restoration"><strong>Data restoration</strong></a></h2><p>After Linear was back online, our engineering team started restoring data that was affected by the rollback.</p><p>As mentioned earlier, Linear&#x27;s realtime sync engine keeps a log of every change that affects entities and properties in a form of sync packet (&quot;action&quot;) that is sent to clients. This includes the type of action taken (insert, update, archive, delete), a snapshot of the entire entity after the change, and a delta of properties changed in case of an update action. Each change also contains an actor, usually the user that made the change.</p><p>We used these actions to find all users affected by changes in the outage. We then contacted them to tell them about the problem and when data would be restored. We also sent an email to all administrators of the affected workspaces in the interest of transparency and awareness.</p><p>Part of our team implemented a new settings page to show administrators the progress of the restoration procedure.This page also listed any lost data and errors during restoration so they could then fix their data, if needed.</p><p>At the same time, we added a restoration script to run through the lost actions for each workspace. It reapplied the actions by users, integrations, API calls, and Linear automations. The script was first executed as a dry-run to expose and fix any edge cases in the replaying of these changes. When a workspace didn&#x27;t have any errors in the dry-run, recovery was run for it. We investigated workspaces that had errors to see if we could fix them.</p><p>Due to the large number of affected workspaces, this investigation took the majority of the recovery time. Some actions were unrecoverable because of unresolvable conflicts and were listed in the individual workspace&#x27;s recovery page. Most of the errors revolved around:</p><ul><li><strong>Document content already existed:</strong> This meant the description for an issue or document had been recently created. We did not want to override changes made by users.</li><li><strong>User created:</strong> Users created during the outage were not recovered as we had no way to tie them back to an authentication. Ultimately, all the actions involving a new user (e.g. being assigned to an issue) and issues created by these new users were skipped as well.</li><li><strong>Modifications to entities no longer existed:</strong> If an update for an entity was archived or deleted, those updates were skipped.</li></ul><p>We also excluded some objects from recovery entirely, such as integrations allowed to act in Linear on behalf of a workspace. We did not want to bring these back automatically after this delay. Instead, we chose to let users reconnect any missing integrations themselves. Similarly, we did not recover newly-created workspaces. Instead, we sent the workspace administrators an email telling them about the situation.</p><h2 class="sc-8978f737-0 jGnIx"><a data-id="impact" id="impact" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#impact"><strong>Impact</strong></a></h2><p>All users experienced one hour of platform downtime while the backup restored data.</p><p>In all, 12% of workspaces had data that was unavailable until the restoration finished. Another 7% had automated changes (like generated Cycles) that were also unavailable.</p><p>We restored over 99% of data within 36 hours. The remaining 4,136 sync packets for unresolvable conflicts represent an average of 0.44 per workspace.</p><h2 class="sc-8978f737-0 jGnIx"><a data-id="next-steps-and-improvements" id="next-steps-and-improvements" class="LinkAnchor_root__3QL80 Link_root__cNtak Link_kind-none__72aR_" rel="noopener" href="#next-steps-and-improvements"><strong>Next steps and improvements</strong></a></h2><p>This incident was the largest in Linear’s five-year history. We strive to build the fastest and most reliable product for our users. This incident revealed many areas where we can do better to find, stop, and recover from similar outages. We learned important lessons from the outage, which we’ll be making actionable in the coming weeks and months. We’ve listed some of the changes we’ll implement below, with more to come as we continue assessing our response:</p><ul><li>No user on the production databases should have <code>TRUNCATE</code> privileges.</li><li>Improve how migrations are created and applied to the database. This includes better review practices from database admins—separate from code reviews—and linting of dangerous operations.</li><li>Make testing of database migrations in a staging environment easier and automated to reduce friction.</li><li>Create and test tooling to quickly re-create the database from point-in-time backups.</li><li>Various changes to internal tooling, addressing weaknesses or friction uncovered by the incident response.</li><li>Improved monitors for data integrity.</li><li>Implement the ability to turn on a read-only mode for Linear, so that clients have read access even when no changes are allowed to reduce the effects of downtime.</li></ul><p>Again, we’re extremely sorry that this outage happened and it has involved most of our engineering team over the past week to resolve. We’ll keep working to improve as a team and to commit to a high level of transparency around our incident response, both during and after.</p></div><div class="Spacer_root__uoSvA" style="--height:72px"></div><div kind="prose" class="LayoutContent_root__ZBUya LayoutContent_prose__Eddy0"><div class="Eyebrow_root__E_290 Eyebrow_size-small__MsgdE">Authors</div><div class="Flex_root__DOQCW Flex_wrap__E4ecg" style="grid-column:4 / span 6;gap:16px"><a data-state="closed" class="sc-91730d2e-1 htdaYb"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_inline__8Gx2p" style="gap:8px"><div class="sc-95fd29f1-0 cMOHUV"><img src="https://cdn.sanity.io/images/ornj730p/production/dff9dd98ea36ac93a38ef6405abc866dda659f68-492x492.jpg?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2" width="18" height="18" alt="Avatar of Jori Lallo" class="sc-7f4a689d-0 isXwKW"/></div><span class="sc-8978f737-0 hLtvGC">Jori Lallo</span></div></a><a data-state="closed" class="sc-91730d2e-1 htdaYb"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_inline__8Gx2p" style="gap:8px"><div class="sc-95fd29f1-0 cMOHUV"><img src="https://cdn.sanity.io/images/ornj730p/production/4d081a7a9ee2575014ab150f2d74cc3e2266a937-300x300.jpg?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2" width="18" height="18" alt="Avatar of Tuomas Artman" class="sc-7f4a689d-0 isXwKW"/></div><span class="sc-8978f737-0 hLtvGC">Tuomas Artman</span></div></a><a data-state="closed" class="sc-91730d2e-1 htdaYb"><div class="Flex_root__DOQCW Flex_align-center__oXK9F Flex_inline__8Gx2p" style="gap:8px"><div class="sc-95fd29f1-0 cMOHUV"><img src="https://cdn.sanity.io/images/ornj730p/production/c5229881e58cfb3a30c31b036e2965f9f327dcc8-512x512.png?w=128&amp;h=128&amp;q=95&amp;auto=format&amp;dpr=2" width="18" height="18" alt="Avatar of Tom Moor" class="sc-7f4a689d-0 isXwKW"/></div><span class="sc-8978f737-0 hLtvGC">Tom Moor</span></div></a></div><div class="Spacer_root__uoSvA" style="--height:48px"></div><div class="Eyebrow_root__E_290 Eyebrow_size-small__MsgdE">Share</div><div><button type="button" class="Button_root__Stmhv Button_kind-secondary__C4t01 Button_size-small__sjtMt Button_kind__Z_LDa"><svg class="" style="--icon-color:#8A8F98" width="16" height="16" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M9.30558 10.206C9.57224 10.4726 9.59447 10.8912 9.37225 11.1831L9.30558 11.2594L6.84751 13.7175C5.58692 14.9781 3.54311 14.9781 2.28252 13.7175C1.0654 12.5004 1.02344 10.5531 2.15661 9.28564L2.28252 9.15251L4.74059 6.69443C5.0315 6.40353 5.50315 6.40353 5.79405 6.69443C6.06071 6.9611 6.08294 7.37963 5.86072 7.67161L5.79405 7.74789L3.33598 10.206C2.6572 10.8847 2.6572 11.9853 3.33598 12.664C3.98082 13.3089 5.00628 13.3411 5.68918 12.7608L5.79405 12.664L8.25212 10.206C8.54303 9.91506 9.01468 9.91506 9.30558 10.206ZM9.82982 6.17019C10.1207 6.46109 10.1207 6.93274 9.82982 7.22365L7.34921 9.70427C7.0583 9.99518 6.58665 9.99518 6.29575 9.70427C6.00484 9.41337 6.00484 8.94172 6.29575 8.65081L8.77637 6.17019C9.06727 5.87928 9.53892 5.87928 9.82982 6.17019ZM13.7175 2.2825C14.9346 3.49962 14.9766 5.44688 13.8434 6.71436L13.7175 6.84749L11.2594 9.30557C10.9685 9.59647 10.4969 9.59647 10.206 9.30557C9.93931 9.03891 9.91709 8.62037 10.1393 8.32839L10.206 8.25211L12.664 5.79403C13.3428 5.11525 13.3428 4.01474 12.664 3.33596C12.0192 2.69112 10.9938 2.65888 10.3109 3.23923L10.206 3.33596L7.74791 5.79403C7.457 6.08494 6.98535 6.08494 6.69445 5.79403C6.42779 5.52737 6.40556 5.10883 6.62778 4.81686L6.69445 4.74057L9.15252 2.2825C10.4131 1.02191 12.4569 1.02191 13.7175 2.2825Z"></path></svg><span class="sc-8978f737-0 fFZeMR">Copy link</span></button></div></div><div class="Spacer_root__uoSvA" style="--height:128px"></div></div></main><footer class="Footer_footer__lJt10"><div class="Footer_inner__vHgz_ LayoutContent_root__ZBUya"><div class="Footer_left__7jJvA"><a class="Footer_logoWrapper__9IJLI Link_root__cNtak" rel="noopener" href="/"><svg width="18" height="18" viewBox="0 0 100 100" fill="#5E6AD2" size="18" aria-label="Linear Logo"><path d="M1.22541 61.5228c-.2225-.9485.90748-1.5459 1.59638-.857L39.3342 97.1782c.6889.6889.0915 1.8189-.857 1.5964C20.0515 94.4522 5.54779 79.9485 1.22541 61.5228ZM.00189135 46.8891c-.01764375.2833.08887215.5599.28957165.7606L52.3503 99.7085c.2007.2007.4773.3075.7606.2896 2.3692-.1476 4.6938-.46 6.9624-.9259.7645-.157 1.0301-1.0963.4782-1.6481L2.57595 39.4485c-.55186-.5519-1.49117-.2863-1.648174.4782-.465915 2.2686-.77832 4.5932-.92588465 6.9624ZM4.21093 29.7054c-.16649.3738-.08169.8106.20765 1.1l64.77602 64.776c.2894.2894.7262.3742 1.1.2077 1.7861-.7956 3.5171-1.6927 5.1855-2.684.5521-.328.6373-1.0867.1832-1.5407L8.43566 24.3367c-.45409-.4541-1.21271-.3689-1.54074.1832-.99132 1.6684-1.88843 3.3994-2.68399 5.1855ZM12.6587 18.074c-.3701-.3701-.393-.9637-.0443-1.3541C21.7795 6.45931 35.1114 0 49.9519 0 77.5927 0 100 22.4073 100 50.0481c0 14.8405-6.4593 28.1724-16.7199 37.3375-.3903.3487-.984.3258-1.3542-.0443L12.6587 18.074Z"></path></svg>Linear - Designed Worldwide</a><div class="Footer_icons__iLRg0"><a class="Footer_iconAnchor__YT5wz" href="https://twitter.com/linear" target="_blank" rel="noopener noreferrer" aria-label="Twitter"><svg class="" style="--icon-color:#8A8F98" width="18" height="18" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path fill="var(--icon-color, #1da1f2)" d="M16 3.42a6.239 6.239 0 0 1-1.767.477 3.046 3.046 0 0 0 1.353-1.676c-.595.347-1.253.6-1.954.735A3.099 3.099 0 0 0 11.386 2c-1.7 0-3.078 1.356-3.078 3.03 0 .237.028.467.08.69a8.786 8.786 0 0 1-6.344-3.166 2.992 2.992 0 0 0-.416 1.523c0 1.05.544 1.979 1.37 2.522a3.108 3.108 0 0 1-1.395-.38v.04c0 1.467 1.06 2.692 2.469 2.97a3.146 3.146 0 0 1-1.39.052 3.073 3.073 0 0 0 2.875 2.103 6.238 6.238 0 0 1-3.822 1.297c-.249 0-.494-.014-.735-.042A8.804 8.804 0 0 0 5.717 14c5.661 0 8.756-4.616 8.756-8.619 0-.13-.003-.261-.009-.391A6.205 6.205 0 0 0 16 3.423L16 3.42Z"></path></svg></a><a class="Footer_iconAnchor__YT5wz" href="https://github.com/linear" target="_blank" rel="noopener noreferrer" aria-label="GitHub"><svg class="" style="--icon-color:#8A8F98" width="18" height="18" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path fill="var(--icon-color, #ffffff)" d="M11.5127 1.96272C10.4401 1.32091 9.27013 1 7.99946 1C6.72987 1 5.5588 1.32091 4.48573 1.96272C3.41428 2.60397 2.56465 3.47445 1.939 4.57358C1.31228 5.67383 1 6.87359 1 8.17567C1 9.73883 1.44527 11.1443 2.33528 12.3933C3.22529 13.64 4.37482 14.5066 5.78494 14.9846C5.94862 15.0159 6.0703 14.9941 6.14891 14.9192C6.22806 14.8437 6.26736 14.7515 6.26736 14.6396L6.26306 14.1337C6.25983 13.8161 6.25767 13.5394 6.25767 13.3023L6.04877 13.3392C5.91524 13.3644 5.74617 13.375 5.54265 13.3717C5.33967 13.3689 5.12807 13.3471 4.90947 13.3063C4.69249 13.266 4.4895 13.1732 4.30105 13.0267C4.11261 12.8802 3.97908 12.6885 3.89993 12.452L3.8084 12.2362C3.7481 12.0931 3.65226 11.9343 3.52196 11.761C3.39112 11.5854 3.25921 11.4669 3.12568 11.4049L3.06107 11.3585C3.01908 11.3271 2.98031 11.2902 2.94262 11.2466C2.90655 11.203 2.87962 11.1594 2.86186 11.1158C2.84355 11.0711 2.85809 11.0359 2.90655 11.0079C2.955 10.9788 3.0433 10.966 3.17037 10.966L3.35343 10.9939C3.47404 11.0185 3.6248 11.0935 3.80409 11.2176C3.98339 11.3428 4.13091 11.5044 4.2456 11.704C4.38559 11.9589 4.55411 12.1535 4.75171 12.2876C4.94931 12.4218 5.14853 12.4889 5.34936 12.4889C5.54857 12.4889 5.72248 12.4721 5.86839 12.4419C6.01377 12.4106 6.15107 12.3637 6.27759 12.3022C6.33359 11.884 6.48219 11.5631 6.72448 11.3389C6.37882 11.3009 6.06761 11.245 5.7914 11.17C5.51465 11.0957 5.22929 10.9744 4.93531 10.8055C4.63918 10.6378 4.39528 10.4298 4.20091 10.1794C4.00708 9.93227 3.84717 9.60353 3.72333 9.201C3.59949 8.79456 3.5365 8.32885 3.5365 7.79885C3.5365 7.04579 3.77556 6.40397 4.25583 5.87397C4.03077 5.30708 4.05123 4.67197 4.32044 3.96755C4.49542 3.91164 4.75656 3.95357 5.1033 4.09334C5.4495 4.2331 5.70256 4.3533 5.86355 4.45226C6.02507 4.55289 6.1543 4.63675 6.25121 4.70496C6.81494 4.54618 7.39751 4.46624 7.99946 4.46624C8.60142 4.46624 9.18399 4.5473 9.74933 4.70887L10.0955 4.48413C10.3324 4.33485 10.6124 4.199 10.9344 4.07321C11.2574 3.9491 11.504 3.91499 11.6742 3.9709C11.9472 4.67533 11.9719 5.30987 11.7469 5.87733C12.2261 6.40621 12.4662 7.04802 12.4662 7.80165C12.4662 8.33053 12.4037 8.79959 12.2794 9.20771C12.1555 9.61583 11.994 9.94457 11.7959 10.19C11.5988 10.436 11.3533 10.6428 11.0582 10.8117C10.7637 10.9794 10.4767 11.1007 10.2021 11.1751C9.92539 11.25 9.61526 11.3059 9.26744 11.3428C9.58349 11.6223 9.74125 12.064 9.74125 12.6695V14.6413C9.74125 14.7531 9.77894 14.8459 9.8554 14.9208C9.93077 14.9952 10.0508 15.0176 10.2151 14.9863C11.6257 14.5055 12.7747 13.6423 13.6647 12.3922C14.5547 11.1454 15 9.73827 15 8.17567C15 6.87415 14.6866 5.67383 14.061 4.57414C13.4348 3.475 12.5857 2.60453 11.5143 1.96328L11.5127 1.96272Z"></path></svg></a><a class="Footer_iconAnchor__YT5wz" href="https://linear.app/join-slack" target="_blank" rel="noopener noreferrer" aria-label="Community Slack"><svg class="" style="--icon-color:#8A8F98" width="18" height="18" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M3.95144 9.85024C3.95144 10.6639 3.29432 11.3216 2.48128 11.3216C1.66823 11.3216 1.01111 10.6639 1.01111 9.85024C1.01111 9.03655 1.66823 8.37891 2.48128 8.37891H3.95144V9.85024ZM4.68653 9.85024C4.68653 9.03655 5.34365 8.37891 6.15669 8.37891C6.96974 8.37891 7.62686 9.03655 7.62686 9.85024V13.5286C7.62686 14.3423 6.96974 14.9999 6.15669 14.9999C5.34365 14.9999 4.68653 14.3423 4.68653 13.5286V9.85024Z" fill="var(--icon-color, #E01E5A)"></path><path d="M9.84333 12.0573C10.6564 12.0573 11.3135 12.7149 11.3135 13.5286C11.3135 14.3423 10.6564 14.9999 9.84333 14.9999C9.03029 14.9999 8.37317 14.3423 8.37317 13.5286V12.0573H9.84333ZM9.84333 11.3216C9.03029 11.3216 8.37317 10.6639 8.37317 9.85024C8.37317 9.03655 9.03029 8.37891 9.84333 8.37891H13.5299C14.3429 8.37891 15.0001 9.03655 15.0001 9.85024C15.0001 10.6639 14.3429 11.3216 13.5299 11.3216H9.84333Z" fill="var(--icon-color, #ECB22E)"></path><path d="M12.0486 6.16083C12.0486 5.34713 12.7057 4.68949 13.5188 4.68949C14.3318 4.68949 14.9889 5.34713 14.9889 6.16083C14.9889 6.97452 14.3318 7.63217 13.5188 7.63217H12.0486V6.16083ZM11.3135 6.16083C11.3135 6.97452 10.6564 7.63217 9.84334 7.63217C9.03029 7.63217 8.37317 6.97452 8.37317 6.16083V2.47134C8.37317 1.65764 9.03029 1 9.84334 1C10.6564 1 11.3135 1.65764 11.3135 2.47134V6.16083Z" fill="var(--icon-color, #2EB67D)"></path><path d="M6.15672 3.94268C5.34368 3.94268 4.68655 3.28503 4.68655 2.47134C4.68655 1.65764 5.34368 1 6.15672 1C6.96977 1 7.62689 1.65764 7.62689 2.47134V3.94268H6.15672ZM6.15672 4.68949C6.96977 4.68949 7.62689 5.34713 7.62689 6.16083C7.62689 6.97452 6.96977 7.63217 6.15672 7.63217H2.47017C1.65712 7.63217 1 6.97452 1 6.16083C1 5.34713 1.65712 4.68949 2.47017 4.68949H6.15672Z" fill="var(--icon-color, #36C5F0)"></path></svg></a><a class="Footer_iconAnchor__YT5wz" href="https://www.youtube.com/@LinearApp" target="_blank" rel="noopener noreferrer" aria-label="YouTube"><svg class="" style="--icon-color:#8A8F98" width="18" height="18" viewBox="0 0 16 16" fill="#8A8F98" role="img" focusable="false" aria-hidden="true"><path d="M13.4527 3.40075C14.0558 3.56204 14.5262 4.03428 14.6893 4.63734C15.1159 6.35783 15.0881 9.61061 14.6983 11.358C14.537 11.961 14.0647 12.4315 13.4617 12.5946C11.7591 13.0157 4.13336 12.9638 2.52936 12.5946C1.92629 12.4333 1.45584 11.961 1.29275 11.358C0.890404 9.71814 0.918183 6.2503 1.28379 4.6463C1.44508 4.04324 1.91733 3.5728 2.5204 3.40971C4.79647 2.93478 12.6426 3.08801 13.4527 3.40075ZM6.64251 5.90007V10.0937L10.2986 7.9969L6.64251 5.90007Z"></path></svg></a></div></div><div class="Footer_section__VUdD6"><h3 class="Footer_sectionTitle__PrfeJ">Product</h3><ul class="Footer_sectionList__qZ85z"><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/features">Features</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/integrations">Integrations</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/pricing">Pricing</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/changelog">Changelog</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/docs">Docs</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/method">Linear Method</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/download">Download</a></li></ul></div><div class="Footer_section__VUdD6"><h3 class="Footer_sectionTitle__PrfeJ">Company</h3><ul class="Footer_sectionList__qZ85z"><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/about">About us</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" aria-current="page" href="/blog">Blog</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/careers">Careers</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/customers">Customers</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/brand">Brand</a></li></ul></div><div class="Footer_section__VUdD6"><h3 class="Footer_sectionTitle__PrfeJ">Resources</h3><ul class="Footer_sectionList__qZ85z"><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/startups">Startup Program</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" target="_blank" rel="noopener noreferrer" href="https://linear.app/join-slack">Community</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/contact">Contact</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/dpa">DPA</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/privacy">Privacy Policy</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/terms">Terms of service</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/security/vulnerability">Report a vulnerability</a></li></ul></div><div class="Footer_section__VUdD6"><h3 class="Footer_sectionTitle__PrfeJ">Developers</h3><ul class="Footer_sectionList__qZ85z"><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" target="_blank" rel="noopener noreferrer" href="https://developers.linear.app">API</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" target="_blank" rel="noopener noreferrer" href="https://linearstatus.com">Status</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" target="_blank" rel="noopener noreferrer" href="https://github.com/linear">GitHub</a></li><li class="Footer_sectionItem__lDqAK"><a class="Link_root__cNtak Link_kind-dimmed__QUE5Y" rel="noopener" href="/readme">README</a></li></ul></div></div></footer></div><script src="/_next/static/chunks/webpack-5d7fdfd3db0f75de.js" crossorigin="" async=""></script><script>(self.__next_f=self.__next_f||[]).push([0]);self.__next_f.push([2,null])</script><script>self.__next_f.push([1,"1:HL[\"/_next/static/css/697eabb2bbc445a1.css\",\"style\",{\"crossOrigin\":\"\"}]\n0:\"$L2\"\n"])</script><script>self.__next_f.push([1,"3:HL[\"/_next/static/css/66815da86973e6a3.css\",\"style\",{\"crossOrigin\":\"\"}]\n4:HL[\"/_next/static/css/c6672b3b5da5c45f.css\",\"style\",{\"crossOrigin\":\"\"}]\n"])</script><script>self.__next_f.push([1,"5:I[90463,[],\"\"]\n8:I[81665,[],\"\"]\na:I[19113,[],\"\"]\nb:I[79241,[\"2746\",\"static/chunks/597e48ed-804b33f585e7b335.js\",\"2853\",\"static/chunks/2853-b6ae42a21e872220.js\",\"4320\",\"static/chunks/4320-fecd6f2023ea247a.js\",\"1249\",\"static/chunks/1249-0b42965bd6dc6539.js\",\"4884\",\"static/chunks/4884-fc3796079191e457.js\",\"64\",\"static/chunks/64-79db7aac0ce71bb5.js\",\"705\",\"static/chunks/705-f288e190c72681e1.js\",\"3185\",\"static/chunks/app/layout-67c8bbf39822b51d.js\"],\"\"]\nc:I[28584,[\"2746\",\"static/chunks/597e48ed-804b33f585e7b33"])</script><script>self.__next_f.push([1,"5.js\",\"2853\",\"static/chunks/2853-b6ae42a21e872220.js\",\"4320\",\"static/chunks/4320-fecd6f2023ea247a.js\",\"1249\",\"static/chunks/1249-0b42965bd6dc6539.js\",\"4884\",\"static/chunks/4884-fc3796079191e457.js\",\"64\",\"static/chunks/64-79db7aac0ce71bb5.js\",\"705\",\"static/chunks/705-f288e190c72681e1.js\",\"3185\",\"static/chunks/app/layout-67c8bbf39822b51d.js\"],\"Providers\"]\nd:I[42766,[\"2853\",\"static/chunks/2853-b6ae42a21e872220.js\",\"4320\",\"static/chunks/4320-fecd6f2023ea247a.js\",\"4002\",\"static/chunks/4002-e41702bacedfdc74.js\",\""])</script><script>self.__next_f.push([1,"3831\",\"static/chunks/3831-f7c2823b994006f3.js\",\"1475\",\"static/chunks/1475-da89268f4995c903.js\",\"64\",\"static/chunks/64-79db7aac0ce71bb5.js\",\"1795\",\"static/chunks/1795-ee1cd5610d48545e.js\",\"7601\",\"static/chunks/app/error-2292dde8b1c24f73.js\"],\"\"]\ne:I[40093,[\"2853\",\"static/chunks/2853-b6ae42a21e872220.js\",\"4320\",\"static/chunks/4320-fecd6f2023ea247a.js\",\"4002\",\"static/chunks/4002-e41702bacedfdc74.js\",\"3831\",\"static/chunks/3831-f7c2823b994006f3.js\",\"1475\",\"static/chunks/1475-da89268f4995c903.js\",\"64\",\"static/chu"])</script><script>self.__next_f.push([1,"nks/64-79db7aac0ce71bb5.js\",\"1795\",\"static/chunks/1795-ee1cd5610d48545e.js\",\"9160\",\"static/chunks/app/not-found-d6645cca11b417f0.js\"],\"\"]\n10:I[58751,[\"2853\",\"static/chunks/2853-b6ae42a21e872220.js\",\"4320\",\"static/chunks/4320-fecd6f2023ea247a.js\",\"4002\",\"static/chunks/4002-e41702bacedfdc74.js\",\"3831\",\"static/chunks/3831-f7c2823b994006f3.js\",\"1475\",\"static/chunks/1475-da89268f4995c903.js\",\"64\",\"static/chunks/64-79db7aac0ce71bb5.js\",\"1795\",\"static/chunks/1795-ee1cd5610d48545e.js\",\"6470\",\"static/chunks/app/glob"])</script><script>self.__next_f.push([1,"al-error-b8704b25c3e7cbc6.js\"],\"\"]\n9:[\"slug\",\"linear-incident-on-jan-24th-2024\",\"d\"]\n11:[]\n"])</script><script>self.__next_f.push([1,"2:[[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/697eabb2bbc445a1.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}]],[\"$\",\"$L5\",null,{\"buildId\":\"Kpv1wlWxOMdFYmuneA1Eu\",\"assetPrefix\":\"\",\"initialCanonicalUrl\":\"/blog/linear-incident-on-jan-24th-2024\",\"initialTree\":[\"\",{\"children\":[\"(marketing)\",{\"children\":[\"blog\",{\"children\":[[\"slug\",\"linear-incident-on-jan-24th-2024\",\"d\"],{\"children\":[\"__PAGE__?{\\\"slug\\\":\\\"linear-incident-on-jan-24th-2024\\\"}\",{}]}]}]}]},\"$undefined\",\"$undefined\",true],\"initialSeedData\":[\"\",{\"children\":[\"(marketing)\",{\"children\":[\"blog\",{\"children\":[[\"slug\",\"linear-incident-on-jan-24th-2024\",\"d\"],{\"children\":[\"__PAGE__\",{},[\"$L6\",\"$L7\",null]]},[\"$\",\"$L8\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\",\"(marketing)\",\"children\",\"blog\",\"children\",\"$9\",\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$La\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"notFoundStyles\":\"$undefined\",\"styles\":[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/66815da86973e6a3.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}],[\"$\",\"link\",\"1\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/c6672b3b5da5c45f.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}]]}]]},[\"$\",\"$L8\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\",\"(marketing)\",\"children\",\"blog\",\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$La\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"notFoundStyles\":\"$undefined\",\"styles\":null}]]},[\"$\",\"$L8\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\",\"(marketing)\",\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$La\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"notFoundStyles\":\"$undefined\",\"styles\":null}]]},[null,[\"$\",\"html\",null,{\"lang\":\"en\",\"suppressHydrationWarning\":true,\"children\":[[\"$\",\"head\",null,{\"children\":[[\"$\",\"link\",\"preconnect-static\",{\"rel\":\"preconnect\",\"href\":\"https://static.linear.app\",\"crossOrigin\":\"anonymous\"}],[\"$\",\"link\",\"preconnect-sanity\",{\"rel\":\"preconnect\",\"href\":\"https://cdn.sanity.io\",\"crossOrigin\":\"anonymous\"}],[[\"$\",\"link\",\"/static/fonts/inter-var-subset-latin-opsz-sep27.woff2\",{\"rel\":\"preload\",\"href\":\"/static/fonts/inter-var-subset-latin-opsz-sep27.woff2\",\"as\":\"font\",\"type\":\"font/woff2\",\"crossOrigin\":\"anonymous\"}]],[\"$\",\"script\",null,{\"dangerouslySetInnerHTML\":{\"__html\":\"(()=\u003e{let e=document.documentElement,t=window.navigator;(function(){try{let r=8;void 0!==t\u0026\u0026\\\"hardwareConcurrency\\\"in t\u0026\u0026(r=t.hardwareConcurrency),e.classList.toggle(\\\"enhanced\\\",r\u003e4)}catch(e){}})(),function(){try{document.cookie\u0026\u0026document.cookie.indexOf(\\\"loggedIn=1\\\")\u003e-1\u0026\u0026e.classList.add(\\\"logged-in\\\")}catch(e){}}(),function(){try{e.classList.add(\\\"js\\\")}catch(e){}}()})();\"}}],[\"$\",\"script\",null,{\"dangerouslySetInnerHTML\":{\"__html\":\"/* \u003c!-- CLIENT_ENV --\u003e */\"}}]]}],[\"$\",\"body\",null,{\"children\":[\"$\",\"$Lb\",null,{\"children\":[\"$\",\"$Lc\",null,{\"children\":[\"$\",\"$L8\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\"],\"loading\":\"$undefined\",\"loadingStyles\":\"$undefined\",\"loadingScripts\":\"$undefined\",\"hasLoading\":false,\"error\":\"$d\",\"errorStyles\":[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/dc6646dc39149bbc.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}]],\"errorScripts\":[],\"template\":[\"$\",\"$La\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":[\"$\",\"$Le\",null,{}],\"notFoundStyles\":[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/dc6646dc39149bbc.css\",\"precedence\":\"next\",\"crossOrigin\":\"\"}]],\"styles\":null}]}]}]}]]}],null]],\"initialHead\":[false,\"$Lf\"],\"globalErrorComponent\":\"$10\",\"missingSlots\":\"$W11\"}]]\n"])</script><script>self.__next_f.push([1,"12:I[2575,[\"2853\",\"static/chunks/2853-b6ae42a21e872220.js\",\"4320\",\"static/chunks/4320-fecd6f2023ea247a.js\",\"4002\",\"static/chunks/4002-e41702bacedfdc74.js\",\"3831\",\"static/chunks/3831-f7c2823b994006f3.js\",\"1475\",\"static/chunks/1475-da89268f4995c903.js\",\"2065\",\"static/chunks/2065-1fba709f93bc0b20.js\",\"5984\",\"static/chunks/5984-fe6969300ebc9026.js\",\"1249\",\"static/chunks/1249-0b42965bd6dc6539.js\",\"2226\",\"static/chunks/2226-b8801cbd0f1f64c3.js\",\"3866\",\"static/chunks/3866-a9d8aacd2f46c894.js\",\"8400\",\"static/chunks/8400-34793ad87919c231.js\",\"9427\",\"static/chunks/9427-58a8df3ca7616cc2.js\",\"3938\",\"static/chunks/3938-6cae752823dd8243.js\",\"64\",\"static/chunks/64-79db7aac0ce71bb5.js\",\"1795\",\"static/chunks/1795-ee1cd5610d48545e.js\",\"705\",\"static/chunks/705-f288e190c72681e1.js\",\"3293\",\"static/chunks/3293-1d0c3dc323cbeb08.js\",\"2021\",\"static/chunks/2021-024b4f80edf8ab1b.js\",\"666\",\"static/chunks/app/(marketing)/blog/%5Bslug%5D/page-549e721e0d4ff138.js\"],\"\"]\n"])</script><script>self.__next_f.push([1,"7:[\"$\",\"$L12\",null,{\"post\":{\"slug\":{\"_type\":\"slug\",\"current\":\"linear-incident-on-jan-24th-2024\"},\"recommendedBlogPosts\":[{\"heroImage\":{\"_type\":\"accessibleImage\",\"alt\":\"Descript logo on dark gray background with tile cards representing the company's tips for using Linear. \",\"alternateImage\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-655567b1044177d9e9da88eeeae666d16796018c-4112x1888-png\",\"_type\":\"reference\"}},\"asset\":{\"_ref\":\"image-4a62063c5cf6d4252c0db260e1963780d99464f8-4112x1888-png\",\"_type\":\"reference\"}},\"category\":{\"name\":\"Linear community\"},\"title\":\"Descript’s internal guide for using Linear as your work operating system\",\"description\":[{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"2a5143bb37c0\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"em\"],\"text\":\"Linear is a critical internal tool for Descript’s EPD and business teams. Below is an excerpt of the best practices guide Descript CEO Andrew Mason wrote to help the teams at Descript set up Linear as their work operating system. The guide includes a breakdown of tips for mastering keyboard shortcuts, integrating Slack messages with Linear, using Raycast for quick issue capture, and customizing Linear for focused task management.\",\"_key\":\"90eb00403dc10\"}]}],\"slug\":{\"current\":\"descript-internal-guide-for-using-linear\",\"_type\":\"slug\"},\"date\":\"2024-01-31T22:04:03.942Z\",\"authors\":[{\"_id\":\"1ba59a06-ea61-445c-9c44-dbdff20a6696\",\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-62175a4eea5abef20b7338bcc6565876ce937416-467x492-jpg\",\"_type\":\"reference\"}},\"date\":null,\"twitterURL\":\"https://twitter.com/karrisaarinen/\",\"linkedinURL\":\"https://www.linkedin.com/in/karrisaarinen/\",\"name\":\"Karri Saarinen\",\"bio\":[{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"96270e5950f7\",\"markDefs\":[],\"children\":[{\"text\":\"Co-founder, CEO. \",\"_key\":\"160190d0244c\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_key\":\"112216a9a4c1\",\"_type\":\"span\",\"marks\":[],\"text\":\"\\nPast: Principal Designer at \"},{\"text\":\"Airbnb\",\"_key\":\"96270e5950f71\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"text\":\". Founding design at \",\"_key\":\"96270e5950f72\",\"_type\":\"span\",\"marks\":[]},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Coinbase\",\"_key\":\"96270e5950f73\"},{\"_type\":\"span\",\"marks\":[],\"text\":\". YC Alum. Co-founded \",\"_key\":\"96270e5950f74\"},{\"text\":\"RailsGirls\",\"_key\":\"96270e5950f75\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_key\":\"96270e5950f76\",\"_type\":\"span\",\"marks\":[],\"text\":\".\"}]}],\"slug\":{\"current\":\"karri-saarinen\",\"_type\":\"slug\"},\"department\":\"Co-founder\",\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/62175a4eea5abef20b7338bcc6565876ce937416-467x492.jpg\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/62175a4eea5abef20b7338bcc6565876ce937416-467x492.jpg\"}],\"_id\":\"2525cc92-d327-424b-85b7-38c0bb7993b5\",\"_createdAt\":\"2024-01-26T04:09:53Z\",\"_updatedAt\":\"2024-02-01T20:48:41Z\"},{\"title\":\"Why and how we do work trials at Linear\",\"date\":\"2023-12-13T18:00:00.000Z\",\"heroImage\":{\"alt\":\"List of work trial related issues in Linear\",\"alternateImage\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-7b5df7eb0d04a32baaced927af8ca5c5b7b9d7aa-4112x1888-png\",\"_type\":\"reference\"}},\"asset\":{\"_ref\":\"image-89ba8b2e3732e8a0f4f5fb2cb51318f6385b971a-4112x1888-png\",\"_type\":\"reference\"},\"_type\":\"accessibleImage\"},\"authors\":[{\"slug\":{\"current\":\"Nathalie\",\"_type\":\"slug\"},\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/3272368b508f6f06bd2af2f7251138c002234631-2964x3021.jpg\",\"twitterURL\":null,\"bio\":[{\"markDefs\":[],\"children\":[{\"_key\":\"f213493d6ede\",\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Talent. \\n\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"Past: Co-founder at \",\"_key\":\"245b43bbdba9\"},{\"text\":\"Techmate Talent\",\"_key\":\"a3ca22b47260\",\"_type\":\"span\",\"marks\":[\"strong\"]}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"0923bcb97941\"}],\"name\":\"Nathalie Alex\",\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-3272368b508f6f06bd2af2f7251138c002234631-2964x3021-jpg\",\"_type\":\"reference\"}},\"date\":null,\"department\":\"Talent\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/3272368b508f6f06bd2af2f7251138c002234631-2964x3021.jpg\",\"linkedinURL\":\"https://www.linkedin.com/in/nathalie-pretzer/\",\"_id\":\"0d23bc79-98eb-4d3a-b5ba-13db577c7406\"},{\"date\":null,\"department\":\"Chief of Staff\",\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/6b75a256fca3fee7d199421b80feecc45f4b7a8f-2521x2585.jpg\",\"_id\":\"3e6f06fa-eba6-4017-b6e1-1f73c0e54d2a\",\"name\":\"Zoe Bauer\",\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-6b75a256fca3fee7d199421b80feecc45f4b7a8f-2521x2585-jpg\",\"_type\":\"reference\"}},\"twitterURL\":\"https://twitter.com/ZoeBauer19\",\"linkedinURL\":\"https://www.linkedin.com/in/zoe-bauer-bb614692/\",\"bio\":[{\"style\":\"normal\",\"_key\":\"05580680ff93\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Operations. \",\"_key\":\"c90269f0dd89\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"\\nPast: Consulting. PjM at\",\"_key\":\"a16dae132be6\"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\" Stanford HAI.\",\"_key\":\"f69ca965ab1e\"},{\"text\":\" Technology and ops at \",\"_key\":\"ff023237533c\",\"_type\":\"span\",\"marks\":[]},{\"marks\":[\"strong\"],\"text\":\"Bridgewater Associates.\",\"_key\":\"34d18c6e45a2\",\"_type\":\"span\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"\\n\",\"_key\":\"29f61a52eb09\"}],\"_type\":\"block\"}],\"slug\":{\"current\":\"Zoe\",\"_type\":\"slug\"},\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/6b75a256fca3fee7d199421b80feecc45f4b7a8f-2521x2585.jpg\"},{\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-ec2a643386cf72d84140382e5d4e1c4ba5982f50-336x336-jpg\",\"_type\":\"reference\"}},\"twitterURL\":null,\"linkedinURL\":\"https://www.linkedin.com/in/marcosfiscal/\",\"slug\":{\"current\":\"Marcos\",\"_type\":\"slug\"},\"date\":null,\"department\":\"Talent\",\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/ec2a643386cf72d84140382e5d4e1c4ba5982f50-336x336.jpg\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/ec2a643386cf72d84140382e5d4e1c4ba5982f50-336x336.jpg\",\"_id\":\"fdc5ea03-d5f5-4a99-a04d-28e76654ebe0\",\"name\":\"Marcos Fiscal\",\"bio\":[{\"style\":\"normal\",\"_key\":\"d47c728631cf\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Talent. \\n\",\"_key\":\"91444763007b\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"Past: Lead Tech Recruiter at \",\"_key\":\"b41e685e7f1e\"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Techmate Talent\",\"_key\":\"7b0e61e7556c\"},{\"_type\":\"span\",\"marks\":[],\"text\":\",\",\"_key\":\"d7b0f070ca49\"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\" \",\"_key\":\"854bbefcc30a\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"Manager at\",\"_key\":\"f8f2066f7fb6\"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\" Premise Data\",\"_key\":\"5ed9303c682d\"},{\"_type\":\"span\",\"marks\":[],\"text\":\".\",\"_key\":\"c2814b101658\"}],\"_type\":\"block\"}]},{\"twitterURL\":null,\"name\":\"Alyssa Garrison\",\"bio\":[{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Talent. \\n\",\"_key\":\"388c8f897a69\"},{\"_key\":\"c3bdeb08e3f9\",\"_type\":\"span\",\"marks\":[],\"text\":\"Past: Co-founder at \"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Techmate Talent\",\"_key\":\"55784f671c6c\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"82898edaba8f\",\"markDefs\":[]}],\"department\":\"Talent\",\"date\":null,\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/7852bcab07a83726bb232277704caa943e39806c-1200x1200.png\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/7852bcab07a83726bb232277704caa943e39806c-1200x1200.png\",\"linkedinURL\":\"https://www.linkedin.com/in/alyssa-garrison/\",\"_id\":\"f5e5b6b2-5ca7-424f-a069-a771928ddab7\",\"slug\":{\"current\":\"Alyssa\",\"_type\":\"slug\"},\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-7852bcab07a83726bb232277704caa943e39806c-1200x1200-png\",\"_type\":\"reference\"}}},{\"bio\":[{\"markDefs\":[],\"children\":[{\"text\":\"People Ops.\\n\",\"_key\":\"f74fb4762dc2\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"text\":\"Past: Recruiting at \",\"_key\":\"c4ccffc8a871\",\"_type\":\"span\",\"marks\":[]},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Guidewheel\",\"_key\":\"64f39e9c0239\"},{\"_type\":\"span\",\"marks\":[],\"text\":\", HR at \",\"_key\":\"1b60a8dfc88d\"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Kindred Bravely\",\"_key\":\"f3b497fefa83\"},{\"_type\":\"span\",\"marks\":[],\"text\":\".\",\"_key\":\"f13bba88cce9\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"45d78eb01361\"}],\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-d7e6a02b0ef6ad6ecf637b5dc773b1822c659071-2145x2146-jpg\",\"_type\":\"reference\"}},\"department\":\"Operations\",\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/d7e6a02b0ef6ad6ecf637b5dc773b1822c659071-2145x2146.jpg\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/d7e6a02b0ef6ad6ecf637b5dc773b1822c659071-2145x2146.jpg\",\"_id\":\"8cefd562-885f-41ec-a674-484ec7c8dcfd\",\"slug\":{\"current\":\"katie\",\"_type\":\"slug\"},\"date\":null,\"twitterURL\":null,\"linkedinURL\":\"https://www.linkedin.com/in/katieroyer/\",\"name\":\"Katie Royer\"},{\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/62175a4eea5abef20b7338bcc6565876ce937416-467x492.jpg\",\"twitterURL\":\"https://twitter.com/karrisaarinen/\",\"name\":\"Karri Saarinen\",\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-62175a4eea5abef20b7338bcc6565876ce937416-467x492-jpg\",\"_type\":\"reference\"}},\"slug\":{\"current\":\"karri-saarinen\",\"_type\":\"slug\"},\"date\":null,\"department\":\"Co-founder\",\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/62175a4eea5abef20b7338bcc6565876ce937416-467x492.jpg\",\"linkedinURL\":\"https://www.linkedin.com/in/karrisaarinen/\",\"_id\":\"1ba59a06-ea61-445c-9c44-dbdff20a6696\",\"bio\":[{\"children\":[{\"text\":\"Co-founder, CEO. \",\"_key\":\"160190d0244c\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_key\":\"112216a9a4c1\",\"_type\":\"span\",\"marks\":[],\"text\":\"\\nPast: Principal Designer at \"},{\"text\":\"Airbnb\",\"_key\":\"96270e5950f71\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"text\":\". Founding design at \",\"_key\":\"96270e5950f72\",\"_type\":\"span\",\"marks\":[]},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Coinbase\",\"_key\":\"96270e5950f73\"},{\"_type\":\"span\",\"marks\":[],\"text\":\". YC Alum. Co-founded \",\"_key\":\"96270e5950f74\"},{\"text\":\"RailsGirls\",\"_key\":\"96270e5950f75\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_key\":\"96270e5950f76\",\"_type\":\"span\",\"marks\":[],\"text\":\".\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"96270e5950f7\",\"markDefs\":[]}]}],\"slug\":{\"current\":\"why-and-how-we-do-work-trials-at-linear\",\"_type\":\"slug\"},\"category\":{\"name\":\"Company Building\"},\"_id\":\"40fb76cc-4201-4c04-b6d3-886f002f081e\",\"_createdAt\":\"2023-12-12T19:52:16Z\",\"_updatedAt\":\"2023-12-13T20:40:58Z\",\"description\":[{\"style\":\"normal\",\"_key\":\"586cf6d144cc\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Building a quality product and business requires people we can trust to make good judgments. Linear uses paid work trials to have candidates complete real projects that test the skills and attributes essential for roles in our startup environment.\",\"_key\":\"be40a1c20a33\"}],\"_type\":\"block\"}]}],\"noIndex\":false,\"_id\":\"1af8c683-108c-45c2-8d9b-24f17662f604\",\"description\":null,\"date\":\"2024-01-30T15:15:09.574Z\",\"content\":[{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"fe53937dd7ea\",\"markDefs\":[{\"href\":\"https://linearstatus.com/incidents/01HMX98N26X50PZR72156MN1F4\",\"_key\":\"06f13547e9d9\",\"_type\":\"link\"}],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"On Wednesday January 24, Linear experienced a temporary \",\"_key\":\"1683ab15c6510\"},{\"_type\":\"span\",\"marks\":[\"06f13547e9d9\"],\"text\":\"data loss incident\",\"_key\":\"1683ab15c6511\"},{\"_type\":\"span\",\"marks\":[],\"text\":\" from 04:47 to 09:56 UTC (about five hours) due to restoration from backup. This affected workspaces that made changes during this window or updated via automated processes (e.g. cycle automation), especially those in the EU due to the timing.\",\"_key\":\"1683ab15c6512\"}]},{\"_key\":\"363bb79f866f\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"As part of the disaster recovery process, we took Linear offline for one hour, impacting all users. We restored over 99% of lost data within 36h of the start of the incident. But, in some cases we couldn't restore data due to conflicts.\",\"_key\":\"8415cd07f86e0\"}],\"_type\":\"block\",\"style\":\"normal\"},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"A database migration caused the incident, which accidentally deleted data from production servers. We put Linear into maintenance mode and reverted the database to a backup taken a few hours prior. We immediately began restoring missing data, which took two days to complete.\",\"_key\":\"853c358f3d5e0\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"cf9e28c2c5ba\",\"markDefs\":[]},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"b63c58f6268d\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"We’re deeply sorry for the inconvenience and frustration this incident caused for our customers. Reliability and security are top priorities for us at Linear and last week we fell short from our promise to our users and ourselves. In learning from this incident, we'll improve our tools and processes to prevent such mistakes and recover more quickly.\",\"_key\":\"6f3b72ce330a0\",\"_type\":\"span\"}]},{\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"Below is a timeline of the incident, and more details around its cause and the ultimate fix.\",\"_key\":\"2b20a52917410\",\"_type\":\"span\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"aee553f86855\"},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Incident timeline\",\"_key\":\"427fe87065330\"}],\"_type\":\"block\",\"style\":\"h2\",\"_key\":\"4d7463f45ee3\",\"markDefs\":[]},{\"children\":[{\"marks\":[\"em\"],\"text\":\"All times are in Coordinated Universal Time (UTC)\",\"_key\":\"73f270a68aa10\",\"_type\":\"span\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"5bf9cded914a\",\"markDefs\":[]},{\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"January 24\",\"_key\":\"2d957459f5dc0\"}],\"_type\":\"block\",\"style\":\"h3\",\"_key\":\"72e9613d213b\"},{\"_key\":\"a0c6fa067359\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"marks\":[\"strong\"],\"text\":\"04:47\",\"_key\":\"6c1f6849fa1c0\",\"_type\":\"span\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Full database backup completed (pre-incident).\",\"_key\":\"07c1fbf64cbd\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"_key\":\"320763b8c44c\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"marks\":[\"strong\"],\"text\":\"07:01\",\"_key\":\"b8093ad78dc50\",\"_type\":\"span\"},{\"text\":\": Merge of migration that caused data loss.\",\"_key\":\"f4c84e4eca0a\",\"_type\":\"span\",\"marks\":[]}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"_key\":\"670bb6e6acc1\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"07:20\",\"_key\":\"769b1ac95ddb0\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Migration completed.\",\"_key\":\"d12d1464ee6d\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"578577e0bf36\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_key\":\"7e6e84c166b10\",\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"07:52\"},{\"marks\":[],\"text\":\": Strange notifications noticed in the app, engineering and support notified to check other reports.\",\"_key\":\"0da5901d54d9\",\"_type\":\"span\"}],\"level\":1},{\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"marks\":[\"strong\"],\"text\":\"08:10\",\"_key\":\"b07aa5b9b1280\",\"_type\":\"span\"},{\"_key\":\"ba61e35161dc\",\"_type\":\"span\",\"marks\":[],\"text\":\": Critical incident initiated with Zoom call to investigate, additional engineers paged.\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"b780f532a1ad\"},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"08:36\",\"_key\":\"feaee1b2b39a0\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Update posted to Linear status page and shared on X: \\\"\",\"_key\":\"720fb0ea3af4\"},{\"_type\":\"span\",\"marks\":[\"706084883b67\"],\"text\":\"Investigating data access issues\",\"_key\":\"feaee1b2b39a1\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"\\\"\",\"_key\":\"feaee1b2b39a2\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"6d28f81cd0f5\",\"listItem\":\"bullet\",\"markDefs\":[{\"_type\":\"link\",\"href\":\"https://x.com/linear/status/1750074910456602761?s=20\",\"_key\":\"706084883b67\"}]},{\"markDefs\":[{\"href\":\"https://linearstatus.com/\",\"_key\":\"eb50c121ac20\",\"_type\":\"link\"}],\"children\":[{\"text\":\"09:20\",\"_key\":\"e9d43faf7d610\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_type\":\"span\",\"marks\":[],\"text\":\": Additional update posted on Linear \",\"_key\":\"f863fde2b7ba\"},{\"marks\":[\"eb50c121ac20\"],\"text\":\"status site\",\"_key\":\"e9d43faf7d611\",\"_type\":\"span\"},{\"_type\":\"span\",\"marks\":[],\"text\":\".\",\"_key\":\"e9d43faf7d612\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"6fd0c95443d0\",\"listItem\":\"bullet\"},{\"style\":\"normal\",\"_key\":\"a10b7cd4e502\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"text\":\"09:56\",\"_key\":\"638ec6f9fdd40\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_type\":\"span\",\"marks\":[],\"text\":\": Linear put in maintenance mode to prevent further changes and recover from backup.\",\"_key\":\"01818c6ceb4d\"}],\"level\":1,\"_type\":\"block\"},{\"_key\":\"a817f4af86ff\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"marks\":[\"strong\"],\"text\":\"10:48\",\"_key\":\"454dc067144c0\",\"_type\":\"span\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Linear access restored with the database backup from 04:47.\",\"_key\":\"015432b1a004\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"11:09\",\"_key\":\"b039fb2375a10\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Status site updated to monitoring.\",\"_key\":\"db62a71566e2\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"78bd5b410910\"},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"11:30\",\"_key\":\"662416ba37a70\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Restoration started for all data entered between 04:47 and 09:56.\",\"_key\":\"5c1e0083472d\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"fb9c02f5c701\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"_key\":\"9722e9635526\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"13:50\",\"_key\":\"fe722c821c6b0\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Emails sent to customers who had created workspaces between 04:47 and 09:56 asking them to recreate their workspace as we could not rebuild it.\",\"_key\":\"d9587dac3f23\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"eed1196e0bbf\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"15:35\",\"_key\":\"2f2b9544482a0\"},{\"marks\":[],\"text\":\": Emails sent to affected users and workspace administrators with information about the incident and restoration process.\",\"_key\":\"894a408a0d98\",\"_type\":\"span\"}],\"level\":1},{\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"January 25\",\"_key\":\"67246624608c0\"}],\"_type\":\"block\",\"style\":\"h3\",\"_key\":\"8d9a32538b8d\"},{\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"84b1b7d3b9ba\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_key\":\"016a4f1a14ba0\",\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"14:00\"},{\"text\":\": Data recovery page published to application settings and shared with admins via email.\",\"_key\":\"e673df8e57c0\",\"_type\":\"span\",\"marks\":[]}]},{\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"c1f3ddc5c956\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"14:25\",\"_key\":\"3735ff7c5e190\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Bug fixed with the data recovery page by forcing clients to refresh (which then overloaded our API and caused application loading issues).\",\"_key\":\"a718aa02d501\"}]},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"16:40\",\"_key\":\"55ba8fc670900\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Dry-runs of the restoration process started.\",\"_key\":\"751b09f53682\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"a45928d41125\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"style\":\"normal\",\"_key\":\"7103deef1620\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"text\":\"17:49\",\"_key\":\"035fca7800f30\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"marks\":[],\"text\":\": Workspace data restoration started.\",\"_key\":\"3290ebf78d0a\",\"_type\":\"span\"}],\"level\":1,\"_type\":\"block\"},{\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"19:48\",\"_key\":\"48bcbec2715a0\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": 98% of affected workspaces restored.\",\"_key\":\"b78bb835edd4\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"269b2e0b9d7f\",\"listItem\":\"bullet\"},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"23:20\",\"_key\":\"dbd18cb20e87\"},{\"marks\":[],\"text\":\": 99% of affected workspaces restored. \",\"_key\":\"331a5f61bd33\",\"_type\":\"span\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"882995892e38\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"_type\":\"block\",\"style\":\"h3\",\"_key\":\"57cc9409cfde\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"January 26\",\"_key\":\"a1aa2b7927350\"}]},{\"markDefs\":[],\"children\":[{\"_key\":\"285f35a158170\",\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"07:37\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Restoration for all workspaces except one completed.\",\"_key\":\"a84c3d8de6e3\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"964141c09d3e\",\"listItem\":\"bullet\"},{\"_key\":\"339989f7ccd9\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"08:39\",\"_key\":\"1a5206ab07d80\"},{\"_type\":\"span\",\"marks\":[],\"text\":\": Restoration of final workspace completed.\",\"_key\":\"031a6d14e2f0\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"markDefs\":[],\"children\":[{\"marks\":[\"strong\"],\"text\":\"What happened\",\"_key\":\"84f45c7a922c0\",\"_type\":\"span\"}],\"_type\":\"block\",\"style\":\"h2\",\"_key\":\"7acf1563f383\"},{\"children\":[{\"text\":\"Linear uses trunk-based development and changes land in production on an ongoing basis as part of new feature development. We gate execution using feature flags.\",\"_key\":\"a02e671f7c570\",\"_type\":\"span\",\"marks\":[]}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"7abe7ba9a45b\",\"markDefs\":[]},{\"children\":[{\"text\":\"While developing new features, we created two new tables in the production database. We also filled these tables with data from existing tables in preparation for rolling out the new functionality. We considered the data faulty and planned a new migration to update it. We had to drop the existing data in the new tables as part of that migration.\",\"_key\":\"0fd3b6ac33440\",\"_type\":\"span\",\"marks\":[]}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"7b6af91d8121\",\"markDefs\":[]},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"3c20f62bbedb\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"A pull request was created, reviewed and accepted that added a database migration that first dropped data in the new tables. Then, it copied data from existing tables to the new ones. The new tables were only used by engineers developing the feature and not used by actual users yet, so we deemed deletion safe. We used the following SQL statement for data deletion:\",\"_key\":\"dbe8116318470\",\"_type\":\"span\"}]},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"c85c74736450\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"code\"],\"text\":\"TRUNCATE TABLE \u003cnew_table\u003e CASCADE;\",\"_key\":\"0b3c4003fe31\"}]},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"The intention here was to delete any data in the new table as well as any rows of test data with foreign keys pointing to the new table. However, \",\"_key\":\"1de72b395ae8\"},{\"_type\":\"span\",\"marks\":[\"code\"],\"text\":\"CASCADE\",\"_key\":\"921ed4103a0e\"},{\"_type\":\"span\",\"marks\":[],\"text\":\" simply truncates the entire contents of any tables that have foreign keys to the table.\",\"_key\":\"844a2c02b12f\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"5a37986f930a\",\"markDefs\":[]},{\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"This caused the full deletion of production data for issue and document descriptions, comments, notifications, favorites, and reactions.\",\"_key\":\"ee2ce1e223890\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"82e4de5446c0\"},{\"children\":[{\"text\":\"When pull requests include migration, we do include CI warnings for dangerous migrations, like large indexing operations. The pull request was tested locally and reviewed by multiple engineers but the cascade operation was missed by both author and peer reviewers.\",\"_key\":\"b002661082c60\",\"_type\":\"span\",\"marks\":[]}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"7fbdbe5fe0fb\",\"markDefs\":[]},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Outage and investigation\",\"_key\":\"3631e62e845b0\"}],\"_type\":\"block\",\"style\":\"h2\",\"_key\":\"30d30167ae97\",\"markDefs\":[]},{\"children\":[{\"_key\":\"ffcb768dc0e70\",\"_type\":\"span\",\"marks\":[],\"text\":\"With the data deleted, symptoms weren't immediately visible to end-users, including our own team. Linear stores most workspace data in a local client cache, and there’s also a database cache which sits in front of the Postgres database. Our \"},{\"_key\":\"ffcb768dc0e71\",\"_type\":\"span\",\"marks\":[\"2c60f45ef2ea\"],\"text\":\"sync engine\"},{\"_type\":\"span\",\"marks\":[],\"text\":\" updates this database cache programmatically to add and remove \\\"sync\\\" packets describing database mutations.\",\"_key\":\"ffcb768dc0e72\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"647c391143d1\",\"markDefs\":[{\"_type\":\"link\",\"href\":\"https://linear.app/blog/scaling-the-linear-sync-engine\",\"_key\":\"2c60f45ef2ea\"}]},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"51179f82e179\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"As this is part of the application logic, deleting rows directly from the database won’t create sync packets and thus won’t update the cache or clients. This meant users would continue to see and receive data for the deleted tables in the Postgres database until they reloaded their local data and the backend cache was invalidated, which usually takes 24 hours. Data caching was also one of the primary reasons the problem was missed in local development.\",\"_key\":\"956588e177720\",\"_type\":\"span\"}]},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"The data loss went unnoticed for 30 minutes due to the multiple layers of cache involved. It was possible to reload the Linear client and not see any visible problems at all. Only mutations caused errors due to underlying database entities no longer existing.\",\"_key\":\"646b376d09580\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"7cb200c8d9e7\",\"markDefs\":[]},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"The outage first became apparent with notifications, as it is one of our most frequently updated tables, with notifications created and deleted based on many events. There is always a small lag in synchronization state between the backend and its clients. We often see the client trying to update deleted entities and see these as a \",\"_key\":\"a2b61368b7f60\"},{\"_type\":\"span\",\"marks\":[\"em\"],\"text\":\"synchronization lag \",\"_key\":\"a2b61368b7f61\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"warning. As an expected part of the system these did not count towards error metrics or trigger alert monitors.\",\"_key\":\"a2b61368b7f62\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"8109c050b597\",\"markDefs\":[]},{\"_key\":\"3297bff0efb7\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"An engineer started an internal incident to investigate when they saw problems while using their own notification inbox. User reports confirmed the synchronization issues that the problems pointed to. This happened around 30 minutes after the migration. This caused a critical incident to start and personnel to be paged.\",\"_key\":\"b1a2388d66170\"}],\"_type\":\"block\",\"style\":\"normal\"},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"The cause wasn’t clear at first due to the caching. But, a spike in warnings from clients to the \",\"_key\":\"5c1bbf5898c20\"},{\"_type\":\"span\",\"marks\":[\"em\"],\"text\":\"notificationUpdate\",\"_key\":\"5c1bbf5898c21\"},{\"_key\":\"5c1bbf5898c22\",\"_type\":\"span\",\"marks\":[],\"text\":\" GraphQL endpoint offered a clue about when the problem started. The faulty commit was quickly correlated. Our engineers verified that the commit did cause similar symptoms when run locally. Shortly thereafter, an engineer found all notifications had been deleted in their local database. At this point, production data was checked and we found multiple tables had been entirely deleted.\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"385632246517\",\"markDefs\":[]},{\"markDefs\":[],\"children\":[{\"_key\":\"c04ba7adc4870\",\"_type\":\"span\",\"marks\":[],\"text\":\"Linear was promptly taken into maintenance mode, which blocks any updates and presents a maintenance screen for all users. We created a new backup to capture all the sync packets created up until that point. Then the database was restored from the latest backup from before the faulty migration ran.\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"f81248d2d4ad\"},{\"_key\":\"cfb58bc26844\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"Linear takes daily full backups in addition to having point-in-time recovery. The last backup was taken at 04:47 UTC and the bad commit landed at 07:01 UTC. With point-in-time recovery engineering could have restored services to the state they were in at 07:01. However, the engineering team had never tested point-in-time recovery, nor had engineering developed tooling to quickly restore the database using point-in-time recovery, so that option wasn’t considered. Restoring from a full backup was a frequently tested procedure and thus executed to get the application up and running as quickly as possible, while initially losing more data than point-in-time recovery.\",\"_key\":\"45e5ec8f13170\",\"_type\":\"span\"}],\"_type\":\"block\",\"style\":\"normal\"},{\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"After backup restoration was completed, Linear was brought back online again with data as of the 04:47 UTC backup.\",\"_key\":\"942359166d310\",\"_type\":\"span\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"48ef713e58cb\"},{\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"The service recovered somewhat quickly, with intermittent problems affecting synchronization. The sync service uses a cursor to get changes not yet sent to clients and caches this cursor in Redis. However, the code did not account for the ID moving backwards, as was the case with a full database restore. Resetting the cache and restarting the sync service resolved these issues.\",\"_key\":\"b1d6fd0a36db0\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"461f10180f37\"},{\"_key\":\"9367c4368902\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Data restoration\",\"_key\":\"109120d457ea0\"}],\"_type\":\"block\",\"style\":\"h2\"},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"3f0721943324\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"After Linear was back online, our engineering team started restoring data that was affected by the rollback.\",\"_key\":\"5a31ec1ccd7c0\"}]},{\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"As mentioned earlier, Linear's realtime sync engine keeps a log of every change that affects entities and properties in a form of sync packet (\\\"action\\\") that is sent to clients. This includes the type of action taken (insert, update, archive, delete), a snapshot of the entire entity after the change, and a delta of properties changed in case of an update action. Each change also contains an actor, usually the user that made the change.\",\"_key\":\"c1ce7ff3362a0\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"9aa2d17c7d73\"},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"c25dbb87cbe2\",\"markDefs\":[],\"children\":[{\"_key\":\"1790a155311d0\",\"_type\":\"span\",\"marks\":[],\"text\":\"We used these actions to find all users affected by changes in the outage. We then contacted them to tell them about the problem and when data would be restored. We also sent an email to all administrators of the affected workspaces in the interest of transparency and awareness.\"}]},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"c62bd19a73b0\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Part of our team implemented a new settings page to show administrators the progress of the restoration procedure.This page also listed any lost data and errors during restoration so they could then fix their data, if needed.\",\"_key\":\"3f036392f6a50\"}]},{\"markDefs\":[],\"children\":[{\"_key\":\"c20cfffb6f410\",\"_type\":\"span\",\"marks\":[],\"text\":\"At the same time, we added a restoration script to run through the lost actions for each workspace. It reapplied the actions by users, integrations, API calls, and Linear automations. The script was first executed as a dry-run to expose and fix any edge cases in the replaying of these changes. When a workspace didn't have any errors in the dry-run, recovery was run for it. We investigated workspaces that had errors to see if we could fix them.\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"c279fd0a443a\"},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"89f5088e3dbe\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Due to the large number of affected workspaces, this investigation took the majority of the recovery time. Some actions were unrecoverable because of unresolvable conflicts and were listed in the individual workspace's recovery page. Most of the errors revolved around:\",\"_key\":\"108dcfc560f10\"}]},{\"markDefs\":[],\"children\":[{\"_key\":\"e52ec57d9b020\",\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Document content already existed:\"},{\"marks\":[],\"text\":\" This meant the description for an issue or document had been recently created. We did not want to override changes made by users.\",\"_key\":\"e52ec57d9b021\",\"_type\":\"span\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"08e49811b426\",\"listItem\":\"bullet\"},{\"_key\":\"a6c5ed6163cf\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"User created:\",\"_key\":\"30eac4e6c2b60\"},{\"_type\":\"span\",\"marks\":[],\"text\":\" Users created during the outage were not recovered as we had no way to tie them back to an authentication. Ultimately, all the actions involving a new user (e.g. being assigned to an issue) and issues created by these new users were skipped as well.\",\"_key\":\"30eac4e6c2b61\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Modifications to entities no longer existed:\",\"_key\":\"e23749f7348a0\"},{\"_type\":\"span\",\"marks\":[],\"text\":\" If an update for an entity was archived or deleted, those updates were skipped.\",\"_key\":\"e23749f7348a1\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"e139811550e3\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"markDefs\":[],\"children\":[{\"_key\":\"897b564198710\",\"_type\":\"span\",\"marks\":[],\"text\":\"We also excluded some objects from recovery entirely, such as integrations allowed to act in Linear on behalf of a workspace. We did not want to bring these back automatically after this delay. Instead, we chose to let users reconnect any missing integrations themselves. Similarly, we did not recover newly-created workspaces. Instead, we sent the workspace administrators an email telling them about the situation.\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"aaa4f7626bec\"},{\"_type\":\"block\",\"style\":\"h2\",\"_key\":\"2f2d8dd7579f\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Impact\",\"_key\":\"ec2fffdb16ab0\"}]},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"All users experienced one hour of platform downtime while the backup restored data.\",\"_key\":\"6b6291188db90\"}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"b71e82b68aac\",\"markDefs\":[]},{\"style\":\"normal\",\"_key\":\"b5b651f07469\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"In all, 12% of workspaces had data that was unavailable until the restoration finished. Another 7% had automated changes (like generated Cycles) that were also unavailable.\",\"_key\":\"9534aa6bc5900\"}],\"_type\":\"block\"},{\"_key\":\"9c40270e6cfc\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"We restored over 99% of data within 36 hours. The remaining 4,136 sync packets for unresolvable conflicts represent an average of 0.44 per workspace.\",\"_key\":\"5f1ab01f59df0\",\"_type\":\"span\"}],\"_type\":\"block\",\"style\":\"normal\"},{\"_type\":\"block\",\"style\":\"h2\",\"_key\":\"651af4ae30ff\",\"markDefs\":[],\"children\":[{\"marks\":[\"strong\"],\"text\":\"Next steps and improvements\",\"_key\":\"ac40ee94c7f20\",\"_type\":\"span\"}]},{\"children\":[{\"text\":\"This incident was the largest in Linear’s five-year history. We strive to build the fastest and most reliable product for our users. This incident revealed many areas where we can do better to find, stop, and recover from similar outages. We learned important lessons from the outage, which we’ll be making actionable in the coming weeks and months. We’ve listed some of the changes we’ll implement below, with more to come as we continue assessing our response:\",\"_key\":\"1daf0ecc5b930\",\"_type\":\"span\",\"marks\":[]}],\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"a044bd462e18\",\"markDefs\":[]},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"15b7999e7435\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"No user on the production databases should have \",\"_key\":\"e369908824740\"},{\"_type\":\"span\",\"marks\":[\"code\"],\"text\":\"TRUNCATE\",\"_key\":\"b986879e33f7\"},{\"_type\":\"span\",\"marks\":[],\"text\":\" privileges.\",\"_key\":\"63248cb6a73b\"}],\"level\":1},{\"children\":[{\"_key\":\"bd17f051df710\",\"_type\":\"span\",\"marks\":[],\"text\":\"Improve how migrations are created and applied to the database. This includes better review practices from database admins—separate from code reviews—and linting of dangerous operations.\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"15f801d3e266\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"_key\":\"54e6465244bb\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"Make testing of database migrations in a staging environment easier and automated to reduce friction.\",\"_key\":\"edbba72c706a0\",\"_type\":\"span\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Create and test tooling to quickly re-create the database from point-in-time backups.\",\"_key\":\"642377bbb5370\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"24493bb58ea0\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Various changes to internal tooling, addressing weaknesses or friction uncovered by the incident response.\",\"_key\":\"9e67f20181ba0\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"a68e04856765\",\"listItem\":\"bullet\",\"markDefs\":[]},{\"_key\":\"ac13a55cf6af\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Improved monitors for data integrity.\",\"_key\":\"d90b604864a70\"}],\"level\":1,\"_type\":\"block\",\"style\":\"normal\"},{\"_type\":\"block\",\"style\":\"normal\",\"_key\":\"4478bf1f0aa9\",\"listItem\":\"bullet\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[],\"text\":\"Implement the ability to turn on a read-only mode for Linear, so that clients have read access even when no changes are allowed to reduce the effects of downtime.\",\"_key\":\"ab628c90ba020\"}],\"level\":1},{\"style\":\"normal\",\"_key\":\"5545d8385fdd\",\"markDefs\":[],\"children\":[{\"marks\":[],\"text\":\"Again, we’re extremely sorry that this outage happened and it has involved most of our engineering team over the past week to resolve. We’ll keep working to improve as a team and to commit to a high level of transparency around our incident response, both during and after.\",\"_key\":\"1e063d775af90\",\"_type\":\"span\"}],\"_type\":\"block\"}],\"metadataTitle\":\"Post mortem on Linear incident from Jan 24th, 2024 | Linear\",\"metadataDescription\":\"Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages\",\"_createdAt\":\"2024-01-29T23:06:43Z\",\"title\":\"Post mortem on Linear incident from Jan 24th, 2024\",\"ogImage\":{\"_type\":\"image\",\"asset\":{\"_type\":\"reference\",\"_ref\":\"image-b35aa350e5f84242286a65f1464c4b25462947a5-2400x1256-jpg\"}},\"authors\":[{\"bio\":[{\"style\":\"normal\",\"_key\":\"df185af2adb4\",\"markDefs\":[],\"children\":[{\"_key\":\"4b1b057a108c\",\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Co-founder. \"},{\"marks\":[],\"text\":\"\\nPast: Early \",\"_key\":\"b834dcc04d52\",\"_type\":\"span\"},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Coinbase\",\"_key\":\"df185af2adb41\"},{\"text\":\" engineer. Created the API and frontend architecture. YC Alum.\",\"_key\":\"df185af2adb42\",\"_type\":\"span\",\"marks\":[]}],\"_type\":\"block\"}],\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-dff9dd98ea36ac93a38ef6405abc866dda659f68-492x492-jpg\",\"_type\":\"reference\"}},\"date\":null,\"department\":\"Co-founder\",\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/dff9dd98ea36ac93a38ef6405abc866dda659f68-492x492.jpg\",\"twitterURL\":\"https://twitter.com/jorilallo/\",\"linkedinURL\":null,\"_id\":\"65933c92-7f6e-46ea-9614-dd4ff58a5ce3\",\"name\":\"Jori Lallo\",\"slug\":{\"current\":\"jorilallo\",\"_type\":\"slug\"},\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/dff9dd98ea36ac93a38ef6405abc866dda659f68-492x492.jpg\"},{\"_id\":\"55d798df-7026-4bf7-832c-d57e02b1c9a7\",\"name\":\"Tuomas Artman\",\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-4d081a7a9ee2575014ab150f2d74cc3e2266a937-300x300-jpg\",\"_type\":\"reference\"}},\"date\":null,\"department\":\"Co-founder\",\"linkedinURL\":\"https://www.linkedin.com/in/tuomasartman/\",\"bio\":[{\"style\":\"normal\",\"_key\":\"60c65cd42c9f\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Co-founder. \",\"_key\":\"f6c2612db84f\"},{\"text\":\"\\nPast: Senior Staff Engineer and Engineering Manager at \",\"_key\":\"9cd9940d418f\",\"_type\":\"span\",\"marks\":[]},{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Uber\",\"_key\":\"924c8a5e15d91\"},{\"marks\":[],\"text\":\".\",\"_key\":\"1a5e81d311f4\",\"_type\":\"span\"}],\"_type\":\"block\"}],\"slug\":{\"_type\":\"slug\",\"current\":\"tuomas-artman\"},\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/4d081a7a9ee2575014ab150f2d74cc3e2266a937-300x300.jpg\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/4d081a7a9ee2575014ab150f2d74cc3e2266a937-300x300.jpg\",\"twitterURL\":\"https://twitter.com/artman/\"},{\"imageURL\":\"https://cdn.sanity.io/images/ornj730p/production/c5229881e58cfb3a30c31b036e2965f9f327dcc8-512x512.png\",\"twitterURL\":\"https://twitter.com/tommoor\",\"linkedinURL\":null,\"name\":\"Tom Moor\",\"bio\":[{\"style\":\"normal\",\"_key\":\"da218ed66fd1\",\"markDefs\":[],\"children\":[{\"_type\":\"span\",\"marks\":[\"strong\"],\"text\":\"Engineering. \",\"_key\":\"a13217621633\"},{\"_type\":\"span\",\"marks\":[],\"text\":\"\\nPast: Principal Engineer at \",\"_key\":\"3a7a2637b30c\"},{\"text\":\"Abstract\",\"_key\":\"94fbd950ca39\",\"_type\":\"span\",\"marks\":[\"strong\"]},{\"_type\":\"span\",\"marks\":[],\"text\":\", Co-Founder at \",\"_key\":\"23e47aaee057\"},{\"marks\":[\"strong\"],\"text\":\"Buffer\",\"_key\":\"de9d2dae2748\",\"_type\":\"span\"},{\"_type\":\"span\",\"marks\":[],\"text\":\".\",\"_key\":\"914baf99b364\"}],\"_type\":\"block\"}],\"image\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-c5229881e58cfb3a30c31b036e2965f9f327dcc8-512x512-png\",\"_type\":\"reference\"}},\"date\":null,\"department\":\"Engineering\",\"avatarUrl\":\"https://cdn.sanity.io/images/ornj730p/production/c5229881e58cfb3a30c31b036e2965f9f327dcc8-512x512.png\",\"_id\":\"46a80bff-064b-46a8-9d0f-21ebc62f1ac1\",\"slug\":{\"current\":\"tom-moor\",\"_type\":\"slug\"}}],\"_updatedAt\":\"2024-01-30T15:30:26Z\",\"heroImage\":{\"alternateImage\":{\"_type\":\"image\",\"asset\":{\"_ref\":\"image-2e281ae34028ff175a63af027de95bca66b9bcb1-2056x944-png\",\"_type\":\"reference\"}},\"asset\":{\"_ref\":\"image-e078697b78690170e26ec6ed13fd77483d5269c4-2056x944-png\",\"_type\":\"reference\"},\"_type\":\"accessibleImage\",\"alt\":\"The linear logo with a warning sign\"},\"category\":{\"name\":\"News\"}}}]\n"])</script><script>self.__next_f.push([1,"f:[[\"$\",\"meta\",\"0\",{\"name\":\"viewport\",\"content\":\"width=device-width, initial-scale=1, viewport-fit=cover\"}],[\"$\",\"meta\",\"1\",{\"charSet\":\"utf-8\"}],[\"$\",\"title\",\"2\",{\"children\":\"Post mortem on Linear incident from Jan 24th, 2024 | Linear\"}],[\"$\",\"meta\",\"3\",{\"name\":\"description\",\"content\":\"Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages\"}],[\"$\",\"meta\",\"4\",{\"name\":\"application-name\",\"content\":\"Linear\"}],[\"$\",\"link\",\"5\",{\"rel\":\"manifest\",\"href\":\"https://linear.app/static/pwa.webmanifest?v=3\"}],[\"$\",\"meta\",\"6\",{\"name\":\"format-detection\",\"content\":\"telephone=no, date=no, address=no, email=no\"}],[\"$\",\"meta\",\"7\",{\"name\":\"apple-mobile-web-app-capable\",\"content\":\"yes\"}],[\"$\",\"meta\",\"8\",{\"name\":\"apple-mobile-web-app-status-bar-style\",\"content\":\"black\"}],[\"$\",\"meta\",\"9\",{\"property\":\"og:title\",\"content\":\"Post mortem on Linear incident from Jan 24th, 2024 | Linear\"}],[\"$\",\"meta\",\"10\",{\"property\":\"og:description\",\"content\":\"Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages\"}],[\"$\",\"meta\",\"11\",{\"property\":\"og:image\",\"content\":\"https://cdn.sanity.io/images/ornj730p/production/b35aa350e5f84242286a65f1464c4b25462947a5-2400x1256.jpg?q=95\u0026auto=format\u0026dpr=2\"}],[\"$\",\"meta\",\"12\",{\"name\":\"twitter:card\",\"content\":\"summary\"}],[\"$\",\"meta\",\"13\",{\"name\":\"twitter:site\",\"content\":\"@linear\"}],[\"$\",\"meta\",\"14\",{\"name\":\"twitter:creator\",\"content\":\"@linear\"}],[\"$\",\"meta\",\"15\",{\"name\":\"twitter:title\",\"content\":\"Post mortem on Linear incident from Jan 24th, 2024 | Linear\"}],[\"$\",\"meta\",\"16\",{\"name\":\"twitter:description\",\"content\":\"Details on the cause of our January 24 temporary data loss incident and learnings to prevent future outages\"}],[\"$\",\"meta\",\"17\",{\"name\":\"twitter:image\",\"content\":\"https://cdn.sanity.io/images/ornj730p/production/b35aa350e5f84242286a65f1464c4b25462947a5-2400x1256.jpg?q=95\u0026auto=format\u0026dpr=2\"}],[\"$\",\"link\",\"18\",{\"rel\":\"icon\",\"href\":\"/favicon.ico\",\"sizes\":\"any\"}],[\"$\",\"link\",\"19\",{\"rel\":\"icon\",\"href\":\"/static/favicon.svg\",\"type\":\"image/svg+xml\"}],[\"$\",\"link\",\"20\",{\"rel\":\"apple-touch-icon\",\"href\":\"/static/apple-touch-icon.png\",\"sizes\":\"180x180\"}]]\n"])</script><script>self.__next_f.push([1,"6:null\n"])</script><script>self.__next_f.push([1,""])</script></body></html><style data-styled="" data-styled-version="5.3.6"></style>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment