Skip to content

Instantly share code, notes, and snippets.

@kdxu
Last active March 14, 2022 05:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kdxu/47095e64aa3a0643fdcb60264fca4493 to your computer and use it in GitHub Desktop.
Save kdxu/47095e64aa3a0643fdcb60264fca4493 to your computer and use it in GitHub Desktop.

RNKit DataChannel 対応について

考える or 調べる必要がある点

  • JS 側のインタフェース設計をどうするか?
  • iOS/Android の DataChannel のインタフェースを調査する
    • そもそも libwebrtc の機能的に iOS/Android の間で差異があるか?

MDN の RTCDataChannel の仕様

https://developer.mozilla.org/ja/docs/Web/API/RTCDataChannel

RTCPeerConnection の DataChannel 関連のメソッド

RTCPeerConnection の DataChannel 関連のイベント

RTCDataChannel のプロパティ

  • label
  • ordered
  • protocol
  • id
  • readyState
  • bufferedAmount
  • binaryType
  • maxPacketLifeType
  • maxRetransmits
  • negotiated
  • reliable (dublicated)
  • stream (duplicated)

RTCDataChannel のイベントハンドラ

https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannelEvent

  • onopen
  • onmessage
  • onclose
  • onerror

RTCDataChannel のメソッド

  • close()
  • send()
    • DOMString (だいたい String), Blob, ArrayBuffer, ArrayBufferView を送信することができる

libwebrtc 側の調査

Android

https://chromium.googlesource.com/external/webrtc/+/refs/heads/master/sdk/android/api/org/webrtc/PeerConnection.java#869

  public DataChannel createDataChannel(String label, DataChannel.Init init) {
    return nativeCreateDataChannel(label, init);
  }

createDataChannel 相当のメソッドが実装されている

https://chromium.googlesource.com/external/webrtc/+/refs/heads/master/sdk/android/api/org/webrtc/DataChannel.java

public class DataChannel {
  /** Java wrapper for WebIDL RTCDataChannel. */
  public static class Init {
    public boolean ordered = true;
    // Optional unsigned short in WebIDL, -1 means unspecified.
    public int maxRetransmitTimeMs = -1;
    // Optional unsigned short in WebIDL, -1 means unspecified.
    public int maxRetransmits = -1;
    public String protocol = "";
    public boolean negotiated;
    // Optional unsigned short in WebIDL, -1 means unspecified.
    public int id = -1;
    @CalledByNative("Init")
    boolean getOrdered() {
      return ordered;
    }
    @CalledByNative("Init")
    int getMaxRetransmitTimeMs() {
      return maxRetransmitTimeMs;
    }
    @CalledByNative("Init")
    int getMaxRetransmits() {
      return maxRetransmits;
    }
    @CalledByNative("Init")
    String getProtocol() {
      return protocol;
    }
    @CalledByNative("Init")
    boolean getNegotiated() {
      return negotiated;
    }
    @CalledByNative("Init")
    int getId() {
      return id;
    }
  }

DataChannel.Init クラス。こちらを configuration 的にcreateDataChannel に渡して datachannel を作成できる。

  /** Java version of C++ DataChannelObserver. */
  public interface Observer {
    /** The data channel's bufferedAmount has changed. */
    @CalledByNative("Observer") public void onBufferedAmountChange(long previousAmount);
    /** The data channel state has changed. */
    @CalledByNative("Observer") public void onStateChange();
    /**
     * A data buffer was successfully received.  NOTE: |buffer.data| will be
     * freed once this function returns so callers who want to use the data
     * asynchronously must make sure to copy it first.
     */
    @CalledByNative("Observer") public void onMessage(Buffer buffer);
  }
....
  /** Send |data| to the remote peer; return success. */
  public boolean send(Buffer buffer) {
    checkDataChannelExists();
    // TODO(fischman): this could be cleverer about avoiding copies if the
    // ByteBuffer is direct and/or is backed by an array.
    byte[] data = new byte[buffer.data.remaining()];
    buffer.data.get(data);
    return nativeSend(data, buffer.binary);
  }

こちらの send 関数の Buffer に boolean binary プロパティが付与されていて、こちらで

binaryType については binary パラメータによって UTF-8 or binary data が選択できる模様。

iOS

https://chromium.googlesource.com/external/webrtc/+/refs/heads/master/sdk/objc/api/peerconnection/RTCPeerConnection+DataChannel.mm

@implementation RTCPeerConnection (DataChannel)
- (nullable RTCDataChannel *)dataChannelForLabel:(NSString *)label
                                   configuration:(RTCDataChannelConfiguration *)configuration {
  std::string labelString = [NSString stdStringForString:label];
  const webrtc::DataChannelInit nativeInit =
      configuration.nativeDataChannelInit;
  rtc::scoped_refptr<webrtc::DataChannelInterface> dataChannel =
      self.nativePeerConnection->CreateDataChannel(labelString,
                                                   &nativeInit);
  if (!dataChannel) {
    return nil;
  }
  return [[RTCDataChannel alloc] initWithFactory:self.factory nativeDataChannel:dataChannel];
}
@end

dataChannelForLabelpc.createDataChannel() 相当のメソッドとなる。

https://chromium.googlesource.com/external/webrtc/+/refs/heads/master/sdk/objc/api/peerconnection/RTCDataChannel.mm

class DataChannelDelegateAdapter : public DataChannelObserver {
 public:
  DataChannelDelegateAdapter(RTCDataChannel *channel) { channel_ = channel; }
  void OnStateChange() override {
    [channel_.delegate dataChannelDidChangeState:channel_];
  }
  void OnMessage(const DataBuffer& buffer) override {
    RTCDataBuffer *data_buffer =
        [[RTCDataBuffer alloc] initWithNativeBuffer:buffer];
    [channel_.delegate dataChannel:channel_
       didReceiveMessageWithBuffer:data_buffer];
  }
  void OnBufferedAmountChange(uint64_t previousAmount) override {
    id<RTCDataChannelDelegate> delegate = channel_.delegate;
    SEL sel = @selector(dataChannel:didChangeBufferedAmount:);
    if ([delegate respondsToSelector:sel]) {
      [delegate dataChannel:channel_ didChangeBufferedAmount:previousAmount];
    }
  }

onmessage, onstatechange(close やら open やら error の状態変更), onbufferedamountchange などのイベントを受け取れそう

https://chromium.googlesource.com/external/webrtc/+/refs/heads/master/sdk/objc/api/peerconnection/RTCDataChannelConfiguration.mm

各種 configration(isOrdered) などは↑の RTCDataChannelConfiguration クラスで定義してある。

こちらも binaryType については isBinary パラメータによって UTF-8 or binary data が選択できる模様

react-native-webrtc-kit 側の調査

Android

react-native-webrtc-kit の WebRTCPeerConnectionObserver.java に以下のようなコードがある。

https://github.com/shiguredo/react-native-webrtc-kit/blob/develop/android/src/main/java/jp/shiguredo/react/webrtckit/WebRTCPeerConnectionObserver.java#L249-L252

    @Override
    public void onDataChannel(DataChannel dataChannel) {
        // DataChannel は現在対応しない
    }

ここで peer.ondatachannel 相当のイベントを受け取ることができる。

iOS

RTCPeerConnection.m に以下のようなコードがある

- (void)peerConnection:(RTCPeerConnection*)peerConnection didOpenDataChannel:(RTCDataChannel*)dataChannel {
    // DataChannel は現在対応しない
}

https://github.com/shiguredo/react-native-webrtc-kit/blob/9333129eb6411bb36122278460a215cd276c91a6/ios/WebRTCModule%2BRTCPeerConnection.m#L722-L724

ここで peer.ondatachannel 相当のイベントを受け取ることができる。


以上を踏まえて、JS / Native 側に必要な実装事項をまとめると

  • RTCDataChannelクラスを追加する

  • datachannel.onstatechange, datachannel.onmessage datachannel.onbufferedamountchange コールバックを実装する

    • datachannel のイベントハンドラを実装する
  • peerConnection.createDataChannel() メソッドを追加する

    • 新たに datachannel を作成する関数
      • 引数は label, options にすると仮定すると
        • options で isBinary, isOrderd, MaxPacketLifeTime, MaxRetransmits などを設定できるようにする必要がある
  • peerConnection.ondatachannel() イベントハンドラを追加する

    • リモートに datachannel が新規に作成されたときのコールバック

ネイティブ から ondatachannel を JS に受け渡す例として

- (void)peerConnection:(RTCPeerConnection*)peerConnection didOpenDataChannel:(RTCDataChannel*)dataChannel {
    // DataChannel は現在対応しない
}

- (void)peerConnection:(RTCPeerConnection*)peerConnection didOpenDataChannel:(RTCDataChannel*)dataChannel {
    [self.bridge.eventDispatcher sendDeviceEventWithName:@"peerConnectionGotDataChannel"
                                                    body:@{@"valueTag": peerConnection.valueTag,
                                                           @"datachannel": @{
                                                                   @"label": ... // 色々情報を付与する
		}
	}
}

のような実装が考えられる。

@kgrvaidya
Copy link

Hi @kdxu
This info is really helpful, Thank you!
I also need one more suggestion, currently I'm working on cross platform VideoChat application and I need to connect ios client with JS master (ios user will be viewer and master will be opening the app inn browser). All other events are triggering properly like didDataChannelOpen and all, but onMessage implementation on ios is not triggering, irrespective of what dataType I pass. In the library, the listener for onmessage expects the data to be in RTCBufferData but I'm unable to send a message of that type from website. Can you please help me with this?

@kdxu
Copy link
Author

kdxu commented Mar 14, 2022

@kgrvaidya Please give me more information about your app. e.g.

  • libraries
  • node version
  • code snippets

@kgrvaidya
Copy link

@kdxu I'm making use aws kinesis sdk for video chat, here are the repo links.
for master view, this repo is used -> https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-js and this file is used -> https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-js/blob/master/examples/master.js

for viewer, this ios sdk is used -> https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-ios

However, in the given ios sdk, createDataChannel is not implemented, which I added here.

` required init(iceServers: [RTCIceServer], isAudioOn: Bool) {
let config = RTCConfiguration()
config.iceServers = iceServers
config.sdpSemantics = .unifiedPlan
config.continualGatheringPolicy = .gatherContinually
config.bundlePolicy = .maxBundle
config.keyType = .ECDSA
config.rtcpMuxPolicy = .require
config.tcpCandidatePolicy = .enabled

    let constraints = RTCMediaConstraints(mandatoryConstraints: nil, optionalConstraints: nil)
    peerConnection = WebRTCClient.factory.peerConnection(with: config, constraints: constraints, delegate: nil)

    super.init()
    configureAudioSession()

    if (isAudioOn) {
    createLocalAudioStream()
    }
    createLocalVideoStream()
    peerConnection.delegate = self
    
    self.dataChannel = **self.setupDataChannel()**
    self.dataChannel?.delegate = self
}

private func setupDataChannel() -> RTCDataChannel{
let dataChannelConfig = RTCDataChannelConfiguration()
let _dataChannel = self.peerConnection.dataChannel(forLabel: "kvsDataChannel", configuration: dataChannelConfig)
return _dataChannel!
}
`

I have added the deletegate method for onMessage implementation like this ->

func dataChannel(_: RTCDataChannel, didReceiveMessageWith buffer: RTCDataBuffer) { delegate?.webRTCClient(self, didReceiveData: buffer.data) }

And in controller, I 'm just printing the value coming in buffer.data

Here's my JS master views send message implementation which is exactly this file -> https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-js/blob/master/examples/master.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment