Notes March 2024

notes-march.markdown

Notes March 2024

1. Reply to issue mrousavy/react-native-vision-camera#2614
2. Code review for PR Fix single line text not fully rendered on old architecture

2. For the Frame Processor runtime I need access to the jsi::Value/jsi::Function that the user passes to the view directly, instead of converting it to a callback within the TurboModules/Native Modules system because the Frame Processor function is a Worklet. This is how this works currently:
   JS: Camera.tsx (uses findNodeHandle(..) + a global setFrameProcessor(..) func)
   iOS: VisionCameraProxy.mm (uses UIManager::viewForReactTag to find the view)
   Android: VisionCameraProxy.kt (uses UIManagerHelper.getUIManager to find the View)

I believe there is an example of implementation in TextInput.js which uses the codegen command setTextAndSelection facebook/react-native#42701 (comment) instead of using UIManager + findNodeHandle

The codegen command calls FabricRenderer dispatchCommand which uses createFromHostFunction to register a function on the native Android/iOS API and calling it from JavaScript.

1. Function::createFromHostFunction() registers a new function with methodNam is dispatchCommand

/// A function which has this type can be registered as a function
/// callable from JavaScript using Function::createFromHostFunction().
/// When the function is called, args will point to the arguments, and
/// count will indicate how many arguments are passed.  The function
/// can return a Value to the caller, or throw an exception.  If a C++
/// exception is thrown, a JS Error will be created and thrown into
/// JS; if the C++ exception extends std::exception, the Error's
/// message will be whatever what() returns. Note that it is undefined whether
/// HostFunctions may or may not be called in strict mode; that is `thisVal`
/// can be any value - it will not necessarily be coerced to an object or
/// or set to the global object.

2. The HostFunction for the methodName dispatchCommand is passed as the last argument of createFromHostFunction

https://github.com/facebook/react-native/blob/8317325fb2bf6563a9314431c42c90ff68fb35fb/packages/react-native/ReactCommon/jsi/jsi/jsi.h#L100-L111

  /// Create a function which, when invoked, calls C++ code. If the
  /// function throws an exception, a JS Error will be created and
  /// thrown.
  /// \param name the name property for the function.
  /// \param paramCount the length property for the function, which
  /// may not be the number of arguments the function is passed.

https://github.com/facebook/react-native/blob/8ff05b5a18db85ab699323d1745a5f17cdc8bf6c/packages/react-native/ReactCommon/react/renderer/uimanager/UIManagerBinding.cpp#L603-L618

[uiManager, methodName, paramCount](
    jsi::Runtime& runtime,
    const jsi::Value& /*thisValue*/,
    const jsi::Value* arguments,
    size_t count) -> jsi::Value {
  validateArgumentCount(runtime, methodName, paramCount, count);


  auto shadowNode = shadowNodeFromValue(runtime, arguments[0]);
  if (shadowNode) {
    uiManager->dispatchCommand(
        shadowNode,
        stringFromValue(runtime, arguments[1]),
        commandArgsFromValue(runtime, arguments[2]));
  }
  return jsi::Value::undefined();
});

3. The dispatchCommand HostFunction calls FabricMountingManager::dispatchCommand:

For the Frame Processor runtime I need access to the jsi::Value/jsi::Function that the user passes to the view directly, instead of converting it to a callback within the TurboModules/Native Modules system because the Frame Processor function is a Worklet.

https://react-native-vision-camera.com/docs/guides/frame-processors

If I try to solve the issue without using worklet (we can try to add it later).

function App() {
  const userDefinedFunction = (frame) => {
    const objects = detectObjects(frame)
    console.log(`Detected ${objects.length} objects.`)
  }
  const frameProcessor = useFrameProcessor(userDefinedFunction)

  return (
    <Camera
      {...cameraProps}
      frameProcessor={frameProcessor}
    />
  )
}

1. Register userDefinedFunction using createFromHostFunction
2. Get the host function from cpp

  /// Returns the underlying HostFunctionType iff isHostFunction returns true
  /// and asserts otherwise. You can use this to use std::function<>::target
  /// to get the object that was passed to create the HostFunctionType.
  ///
  /// Note: The reference returned is borrowed from the JS object underlying
  ///       \c this, and thus only lasts as long as the object underlying
  ///       \c this does.

https://github.com/facebook/react-native/blob/8317325fb2bf6563a9314431c42c90ff68fb35fb/packages/react-native/ReactCommon/jsi/jsi/jsi.h#L1073-L1079

3. Call the host function with parameter the current frame

  /// Calls the function with \c count \c args.  The \c this value of the JS
  /// function will not be set by the C++ caller, similar to calling
  /// Function.prototype.apply(undefined, args) in JS.
  /// \b Note: as with Function.prototype.apply, \c this may not always be
  /// \c undefined in the function itself.  If the function is non-strict,
  /// \c this will be set to the global object.
  Value call(Runtime& runtime, const Value* args, size_t count) const;

https://github.com/facebook/react-native/blob/8317325fb2bf6563a9314431c42c90ff68fb35fb/packages/react-native/ReactCommon/jsi/jsi/jsi.h#L1002-L1008

Text length higher than parent view width

facebook/react-native#41770

CLICK TO OPEN EXAMPLE

import * as React from 'react';
import {StyleSheet, Text, TextInput, View} from 'react-native';

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com  From vincenzoddrlxagon+five@gmail.com';
  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            style={styles.parentText}
            numberOfLines={1}
          >
            {email}
          </Text>
        </View>
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Before	OLD ARCH	NEW ARCH

Text width lower than parent view (no ellipsis)

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {StyleSheet, Text, TextInput, View} from 'react-native';

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com';
  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            style={styles.parentText}
            numberOfLines={1}
          >
            {email}
          </Text>
        </View>
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Nested Text with different combination of styles (ellipsisMode head)

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {StyleSheet, Text, TextInput, View} from 'react-native';

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com From vincenzoddragon+five@gmail.com';
  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            android_hyphenationFrequency="full"
            ellipsizeMode="head"
            textBreakStrategy="highQuality"
            style={styles.parentText}
            numberOfLines={1}>
            <Text style={{fontSize: 50}}>
              From vincenzoddragon+five@gmail.com
            </Text>{' '}
            From vincenzoddragon+five@gmail.com
          </Text>
        </View>
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Nested rtl short Text with inline image (image shows to the right)

Adding an inline Image adds a TextInlineViewPlaceholderSpan which is responsible for layout calculation. The placeholder does not appear in the Text, but is only used to compute the image dimension and position in the text.

Detailed explanation of this API: react-native-community/discussions-and-proposals#528 (reply in thread)

fabOnReact/react-native@a2285b1 and facebook/react-native@91b3f5d

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {Image, StyleSheet, Text, TextInput, View} from 'react-native';
const fullImage: ImageSource = {
  uri: 'https://www.facebook.com/ads/pics/successstories.png',
};

const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};
export default function App() {
  const email = 'مرحبا بالعالم من فابريزيو';
  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            writingDirection="rtl"
            style={styles.parentText}
            numberOfLines={1}>
            <Text>{email}</Text>
            <Image source={smallImage} style={{height:  50, width:  50}}/>
          </Text>
        </View>
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    direction: 'rtl',
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Before	After

Before (changes from this PR): The image placeholderLeftPosition is 345
After (changes from this PR): The image placeholderLeftPosition is 345

Very long rtl text (image does not show and there is no ellipsis)

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const fullImage: ImageSource = {
  uri: 'https://www.facebook.com/ads/pics/successstories.png',
};

const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email =
    'مرحبا بالعالم من فابريزيومرحبا بالعالم من فابريزيو مرحبا بالعالم من فابريزيومرحبا بالعالم من فابري زيو';
  let textRef = React.useRef(null);
  const measureCallback = (x, y, w, h, pageX, pageY) => {
    console.warn(
      'x: ' +
        parseInt(x) +
        ' y: ' +
        parseInt(y) +
        ' w: ' +
        parseInt(w) +
        ' h: ' +
        parseInt(h),
    );
  };

  const measureTextView = () => {
    if (textRef) {
      textRef.measure(measureCallback);
    }
  };

  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            ref={ref => (textRef = ref)}
            writingDirection="rtl"
            style={styles.parentText}
            numberOfLines={1}>
            <Text>{email}</Text>
            <Image source={smallImage} style={{height: 50, width: 50}} />
          </Text>
        </View>
        <Button title="measure" onPress={measureTextView} />
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    direction: 'rtl',
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Before	After

Text with nested image is shorter than parent view (the image shows and there is no ellipsis)

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const fullImage: ImageSource = {
  uri: 'https://www.facebook.com/ads/pics/successstories.png',
};

const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email =
    'مرحبا بالعالم من فابريزيومرحبا بالعالم من فابريزيو مرحبا  مرحبا';
  let textRef = React.useRef(null);
  const measureCallback = (x, y, w, h, pageX, pageY) => {
    console.warn(
      'x: ' +
        parseInt(x) +
        ' y: ' +
        parseInt(y) +
        ' w: ' +
        parseInt(w) +
        ' h: ' +
        parseInt(h),
    );
  };

  const measureTextView = () => {
    if (textRef) {
      textRef.measure(measureCallback);
    }
  };

  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            ref={ref => (textRef = ref)}
            writingDirection="rtl"
            style={styles.parentText}
            numberOfLines={1}>
            <Text>{email}</Text>
            <Image source={smallImage} style={{height: 50, width: 50}} />
          </Text>
        </View>
        <Button title="measure" onPress={measureTextView} />
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    direction: 'rtl',
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Before	After

  const email =
    'مرحبا بالعالم من فابريزيومرحبا بالعال';

Before	After

Correctly calculates the placeholderLeftPosition for inline images.

  const email =
    'مرحبا بالعالم من ';

Before	After

  const email =   'مر';

Before	After

  const email =   'ر';

Before	After

BoringLayout support for margin and padding with single line text

CLICK TO OPEN TESTS SOURCE CODE

import * as React from 'react';
import {
  Button,
  Pressable,
  SafeAreaView,
  StyleSheet,
  Text,
  View,
} from 'react-native';

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com From vincenzoddragon+five@gmail.com';
  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            textBreakStrategy="simple"
            numberOfLines={1}
            style={styles.input}>
            {email}
          </Text>
        </View>
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
    backgroundColor: 'red',
  },
  input: {
    backgroundColor: 'pink',
    marginLeft: 50,
    paddingLeft: 50,
    marginRight: 50,
    paddingRight: 50,
  },
});

Paper and Fabric behave the same.

Before	After

The text wraps correctly without leaving white spaces

CLICK TO OPEN SOURCECODE

import * as React from 'react';
import {
  Button,
  Text,
  SafeAreaView,
  StyleSheet,
  Pressable,
  View,
} from 'react-native';

const RNTesterApp = () => {
  const [text, setText] = React.useState('From vincenzoddragon+five@gmail.com');
  var things = ['short', 'verylong', 'superlongword'];
  var thing = things[Math.floor(Math.random() * things.length)];
  const addText = () => {
    setText(prevText => prevText + ' ' + thing);
  };

  const email =
    'From vincenzoddragon+five@gmail.com From vincenzoddragon+five@gmail.com \n';
  return (
    <>
      <View style={styles.container}>
        <View style={styles.flexBrokenStyle}>
          <Text
            textBreakStrategy="simple"
            adjustFontSizeToFit={true}
            style={{backgroundColor: 'pink'}}>
            {text}
          </Text>
        </View>
        <Button title="add text" onPress={addText} />
      </View>
    </>
  );
};

export default RNTesterApp;

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
    maxWidth: 300,
  },
});

Before	After commit 85308	After

Text width is larger than parent

Text width is higher of parent (YogaMeasureMode.EXACTLY)

CLICK TO OPEN SOURCECODE

import * as React from 'react';
import {
  Button,
  Pressable,
  SafeAreaView,
  StyleSheet,
  Text,
  View,
} from 'react-native';

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com From vincenzoddragon+five@gmail.com';
  return (
    <>
      <View style={styles.flexBrokenStyle}>
        <Text textBreakStrategy="simple" numberOfLines={1} style={styles.input}>
          {email}
        </Text>
      </View>
    </>
  );
}

const styles = StyleSheet.create({
  container: {
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    paddingTop: 50,
    paddingRight: 0,
    width: 300,
    height: 300,
    backgroundColor: 'red',
  },
  input: {
    backgroundColor: 'pink',
    width: 400,
  },
});

Before	After

Paper

Before	After

TextView takes the entire line

• TextView onMeasure calculatedWidth is 1080
• Text lineWidth is 406.0
• placeholderLeftPosition is 1080 - 406

CLICK TO OPEN SOURCECODE

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email = 'مرحبا بالعالم من ف';
  return (
    <>
      <Text writingDirection="rtl" style={styles.parentText} numberOfLines={1}>
        <Text>{email}</Text>
        <Image source={smallImage} style={{height: 50, width: 50}} />
      </Text>
    </>
  );
}

const styles = StyleSheet.create({
  parentText: {
    backgroundColor: 'red',
  },
});

Fabric

Before	After

Paper

Before	After

Image does not fix in TextView

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com From vincenzoddragon+five@gmail.com';
  return (
    <>
      <Text style={styles.parentText} numberOfLines={1}>
        <Text>{email}</Text>
        <Image source={smallImage} style={{height: 50, width: 50}} />
      </Text>
    </>
  );
}

const styles = StyleSheet.create({
  parentText: {
    backgroundColor: 'red',
  },
});

Fabric

Before	After

Paper

Before	After

Images do not fit in the line (parent applies flex style)

This is a separate issue caused by the logic that calculates placehoderTop and placeholderLeftPosition, which does not handle scenarios where text is single line. The image is positioned on the second line as if text would wrap on the second line.

CLICK TO OPEN SOURCECODE

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com  From vincenzoddrlxagon+five@gmail.com';
  return (
    <View style={styles.container}>
      <View style={styles.flexBrokenStyle}>
        <Text style={styles.parentText} numberOfLines={1}>
          <Text>{email}</Text>
          <Image source={smallImage} style={{height: 50, width: 50}} />
        </Text>
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Fabric

Before	After

Paper

Before	After

Text with images wraps on the second line

CLICK TO OPEN SOURCECODE

Relevant commit 00c318cb742 and https://gist.github.com/assets/24992535/65240c44-ba57-4d66-8dd2-e168aeb98da2

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com ddddddddd dddd dd dd dd dd d';
  return (
    <View style={styles.container}>
      <View style={styles.flexBrokenStyle}>
        <Text style={styles.parentText}>
          <Text>{email}</Text>
          <Image source={smallImage} style={{height: 50, width: 50}} />
        </Text>
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Fabric:

Before	After

Paper:

Before	After

Text includes emoji (not boring text)

CLICK TO OPEN TESTS RESULTS

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email = 'From vinesdfsd 👍 😊 🐚 😆';
  const email2 =
    'مرحبا بالعالم من فابريزيومرحبا بالعالم من فابريزيو مرحبا  مرحبامن فابريزيو مرحبا  مرحبا';
  return (
    <View style={styles.container}>
      <View style={styles.flexBrokenStyle}>
        <Text numberOfLines={1}>
          <Text style={styles.parentText}>{email}</Text>
        </Text>
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
  },
  flexBrokenStyle: {
    // direction: 'rtl',
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
    fontSize: 30,
  },
});

After

Veryfing the result of the onLayout callback

CLICK TO OPEN SOURCECODE

import * as React from 'react';
import {Button, Image, StyleSheet, Text, TextInput, View} from 'react-native';
const smallImage = {
  uri: 'https://www.facebook.com/favicon.ico',
};

export default function App() {
  const email =
    'From vincenzoddragon+five@gmail.com  From vincenzoddrlxagon+five@gmail.com';
  const onLayoutCallback = e => {
    const {width, height, x, y} = e.nativeEvent.layout;
    console.warn(
      'w: ' +
        parseInt(width) +
        ' h:' +
        parseInt(height) +
        ' x: ' +
        parseInt(x) +
        ' y: ' +
        parseInt(y),
    );
  };
  return (
    <View style={styles.container}>
      <View style={styles.flexBrokenStyle}>
        <Text
          onLayout={onLayoutCallback}
          style={styles.parentText}
          numberOfLines={1}>
          <Text>{email}</Text>
          <Image source={smallImage} style={{height: 50, width: 50}} />
        </Text>
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    padding: 8,
    backgroundColor: 'yellow',
  },
  flexBrokenStyle: {
    flexDirection: 'row',
  },
  parentText: {
    backgroundColor: 'red',
  },
});

Before (Fabric)	After (Fabric)

Before (Paper)	After (Paper)