The Lighthouse performance audit tool recommends lazy-loading offscreen images (and other content)
They go a step further and recommend using the new IntersectionObserver API to manage offscreen content. Of course, you need to use a polyfill to target old browsers (core-js doesn't offer a polyfill).
The documentation provided by the Lighthouse team warns against over-applying the IntersectionObserver. The have a whole section devoted to explaining why you don't want to intersect all the things. Basically, you want to add "sentinels" to your long lists so that you can track fewer items.
Thankfully there's an interesting React component, helpfully named react-intersection-observer, that makes it dead-simple to implement the IntersectionObserver API in a declarative way. And, to address the need for sentinels in long lists, they even offer a react-intersection-list component.
You can read some more about how the basics on using IntersectionObsever for infinite scoll (on scotch.io; a case-study on LiveNation; on Smashing Magazine).