So you're dropping frames, and you don't know why. You think "Hmm, I don't understand why this is happening, I see other people reporting it, so I suspect my streaming program is at fault!"
So, let's examine what frame drops are and why they happen. TL;DR at the bottom, but please don't skip to the TL;DR because I spent way too much time writing this when I probably should have been coding.
Your streaming program creates video/audio frames, compresses them, and sends them out right after they're compressed, at whatever framerate you set your streaming program to. To send them, it calls a single function called "send" for each compressed audio/video packet as soon as they're created. This "send" function is socket function implemented by the operating system itself (it's a Microsoft/Apple/Linux function -- not a function created by your streaming program). If your connection has no problems, that "send" function will finish right away, with almost no delay, and you will not get frame drops.
However, if that "send" function takes too long (due to network congestion of some sort), excess audio/video packets that you're trying to send out will start to back up in a queue so to speak, sitting there waiting to be sent off. Meanwhile, you're encoding data at however many frames per second, and they can't just sit there and wait for that function to finish on end forever until your memory runs out, eventually something's got to give. When they get to a specific point (about 700ish milliseconds worth of backed up data), it will trigger the phenomena you know as "dropping frames", where it's forced to dump video frames for a bit to clear up that backed up packet queue. This is why your stream will experience what appears to be video stuttering when that happens.
In the case of some random spontaneous network congestion, dropping frames after a specific amount of congestion is both necessary and important to ensure stream stability and minimize stream latency.
Why is it important for stability?
- In the case of spontaneous congestion, if it did not do this, it would cause bursting as all those backed up packets try to be send out as soon as the congestion stops, which could worsen the network stability, possibly causing more network instability.
- In the case of consistent congestion (i.e. you set your bitrate higher than your upload can maintain at a stable rate), it would continually queue audio/video packets until you simply ran out of memory, which is also something that must be avoided.
Why is it important for latency?
- In the case of spontaneous congestion, let's say it backs up for 5 seconds. If there were no frame drops, all your viewers will then be delayed by those five seconds or more. It would happen pretty much every time you had congestion as well.
- In the case of consistent congestion, what your viewers would see would be continually delayed and delayed, making it not so much a live stream anymore. That and the whole running out of memory thing because the packet queue with compressed audio/video data would just keep growing.
So to ensure network stability and minimize latency, after about 700ish milliseconds of backed up audio/video packets, it decides to start dropping frames. If it were lower, it would make it a bit too sensitive to congestion. If it were higher, it could cause stream delay or latency issues, so it tries to keep it at a reasonable number.
If it is happening consistently, it's typically a sign that your bitrate is too high for you to be able to maintain with stability.
So frame drops do not happen because of some bug or error in the program, or some problem with the program -- they literally happen only because the the program calls the "send" function, the send function takes too long, and queued audio/video packets start piling up waiting to be sent out, until they reach a critical mass in which they need to be chopped out to maintain stream latency and stability.
In other words, it only happens because of network congestion. Streaming programs do not drop frames for any other reason than network congestion.
What causes that network congestion?
Unstable connection to the server, poor/unstable ISP or connection to ISP are most common, in more rare cases it can be bad routers or modems, in very rare cases it can be other system issues or certain very uncommon network devices or drivers.
Note the key words above: "maintain with stability". The rate in which you send out audio/video packets must be consistent and stable to ensure both latency and further stability, as stated above. This does not necessarily mean that your internet connection is unable to upload at a higher rate, it only means that it's unable to upload at the set rate with stability. You could hypothetically have like 30mb/s up and only be able to maintain 3mb/s at a fully stable rate. Streaming programs need to be able to upload their data at a consistent, stable rate, otherwise there will be congestion, which can cause frame drops.
- In at least the case of OBS, there's a specific test we use to ensure that frame drop handling is working as expected. It's currently working properly and as expected. Were it not, it would fail that built-in test.
- Frame dropping is not exactly rocket science -- the packet queue gets too large, it dumps video data to ensure stream latency/stability.
- For why you see other people report frame drops: Because network congestion is not an uncommon issue! Many people get network congestion, from small streamers to big streamers. You live in some location in the United States where you only have one ISP monopolizing the entire area with dated infrastructure and like 5-20mb/s upload max, where that they have no incentive to fix stability issues because of that monopoly? You're likely to have frame drops. You live out in the boonies with your internet set up by some crazy local named billy bob using terrible wiring that ends up being chewed on by racoons every night? You're likely to have frame drops. You're on wifi? You're likely to have frame drops. You just have terrible upload speed? You're likely to have frame drops.
Again, there are many reasons why you can have network congestion, which is specifically why we created an entire section in our help for it:
https://github.com/jp9000/obs-studio/wiki/Dropped-Frames-and-General-Connection-Issues
So, if you've actually managed to read this far, I hope you've been enlightened as to what frame drops are, why they happen, why you hear other people report them, and why, despite that fact, they are very likely not the fault of your streaming program.
TL;DR? A single function of your operating system, "send" is called with your audio/video packets, and if it takes too long (due to network congestion and only network congestion), audio/video packets pile up (get queued), and after a certain point the program is forced to dump data to maintain stream latency and stability. It is quite literally not the streaming program's fault. It means you have network congestion for whatever reason.
I wrote all this for you. Yep, you on twitter there. For some reason I guess I just sort of snapped and wrote this entire thing just now. I did it because I literally cannot control the send function, because network congestion is not my fault, because there are a million and one reasons why network congestion can occur, and of course because I love you.
Hi Jim
Ich möchte mich ganz herzlich bei Dir Bedanken!!!!! Deine Auffassungsgabe und Dein Vermögen , Dropped Frames , zu Verstehen und dann auch noch klar in Physikalischen sehr Detaillierten Wissenschaftlichen Fakten zu Erläutern.
Danke Dir Jim für das Teilen deines Wissen ,in dessen ich auch absolut gleicher Auffassung bin , und dies in allen Aspekten die du Analysiert und als freies Feedback verfasst hast. jeder Hinsicht dieses trotzt allem als Phänomen betitelten Themas,
Liebe Grüsse Lenox M