Why is video not playing on recording of remote stream?

gistfile1.txt

https://stackoverflow.com/questions/61022341/why-is-video-not-playing-on-recording-of-remote-stream

This works without a problem. If I replace the local stream with a remote stream, same code, only received over webRTC, I see the first frame and nothing more... No errors... jsut one Frame.

for some reason mediaSource can only be there once?!

Not certain what is being described?

And there semms to be a problem if someone joins midstream, I need to find a way around that (or restart the recording?)

Joined how? Video and audio? You can use OfflineAudioContext or ChannelMerger and ChannelSplitter to result in multiple audio streams that are all recorded https://stackoverflow.com/questions/40570114/is-it-possible-to-mix-multiple-audio-files-on-top-of-each-other-preferably-with.

If a user exists and another user enters the communication, one approach could be to have one MediaStream that is recorded and use RTCRtpSender.replaceTrack() to replace audio and video tracks in "midstream".

WebRTC, the standard way (Client->Server->Client), is all working fine (Video and Audio). Only problem is the server load on video on many people.
So to solve this I record the video on the server with mediaRecorder,send the chunks back to the clients via websockets, (this way I avoid webRTC video transcoding), and render it via MediaSource. That is working but with two little problems atm:

1. If someone joins the session while the video is already running, he is not able to render the stream (because he is not starting on a keyframe or so)
2. If I start the video (everything is woking), stop it, and start again I'll get somthing like "MediaStream is closed" on the client even if I did a "new MediaStream" on every new stream start. (but will change that to the canvas solution anyway I think).