- Name: Om Doiphode
- Organization: Unicode
- Mentors: Frank Yung-Fong Tang, Shane Carr
- Project: Unicode Spoof Detection
In orthography and typography, a homoglyph is one of two or more characters with shapes that appear identical or very similar. Characters that look the same can be swapped in strings to make strings that look similar. These strings can trick users into clicking on fake websites that look real or avoid spam and plagiarism detectors. While people have known about the security risks of these look-alike characters for a long time, it has become a bigger issue as Unicode has become the main way text is processed. Unlike ASCII, which has only 128 characters, the newest Unicode standard has over 140 thousand characters. Thus, it becomes imperative to develop an AI tool that can automatically detect confusable characters.
- Build an AI model that can detect similarity between two characters. Run the developed model over the Unicode codepoints to generate confusability data.
- Submit the new data to the UCD.
- Setup, Onboarding, and Data Generation
- Model training and iteration
- Publish the results
- There has been work done on homoglpyh detection for the Han script previously by an intern at Google. Hence, my task was to extend the homoglyph detection for other Unicode scripts. I decided to go with the left-to-right alphabetic scripts.
- The dataset for the model was created using the UnicodeData.txt and Blocks.txt from the Unicode Database. The dataset contains the characters from left-to-right scripts. The total number of characters amounted to 5095.
- At the recommendation of Roozbeh, only those characters were taken that do not have a canonical decomposition. This reduced the number of characters to 4287. Fonts used for rendering the characters were as follows:
- A JSON file was created, which gives the mapping of codepoints to the fonts that can render them.
- The dataset is organized at the codepoint level, where each Unicode character (codepoint) has its dedicated folder containing images representing that character in various fonts or styles. The structure is as follows:
dataset/
├── U+0041/ # Folder for codepoint 'A'
│ ├── img1.png
│ ├── img2.png
│ └── …
├── U+0430/ # Folder for Cyrillic 'а'
│ ├── img1.png
│ ├── img2.png
│ └── …
├── U+0391/ # Folder for Greek 'Α'
│ ├── img1.png
│ ├── img2.png
│ └── ...
- Dataset split: train, test, and validation (80:10:10)
- Preprocessing:
- Images were resized to a standard shape (e.g., 224×224)
- Normalized to a [0, 1] intensity range
- Converted to RGB
- Augmentations used were RandomTranslation, RandomZoom, RandomZoom and RandomContrast.
- Batch Sampling Strategy:
- For training the model using triplet loss, a custom sampling strategy was implemented to ensure balanced batch composition. Each training batch consists of P × K images, where:
- P = Number of unique codepoints sampled in the batch
- K = Number of images per codepoint
- This strategy guarantees that every batch contains K examples from each of the P selected codepoints, enabling efficient triplet generation (anchor, positive, negative) within a batch.
- The sampling function ensures that:
- If a codepoint has fewer than K images, its images are repeated to meet the requirement.
- Codepoints are randomly selected per batch to promote generalization.
- For training the model using triplet loss, a custom sampling strategy was implemented to ensure balanced batch composition. Each training batch consists of P × K images, where:
Dataset Generation Notebook: https://github.com/unicode-org/ml-confusables-generator/blob/main/deep2025/source/dataset_generation.ipynb
Homoglyph detection is similar to face verification, wherein the similarity between two images is calculated. The FaceNet paper introduced triplet loss to train a CNN for face verification or recognition. The triplet loss is defined as follows:
L = max [d(a,p) — d(a,n) + margin, 0]
where,
- d(a,p) = distance between the anchor image and the positive image
- d(a,n) = distance between the anchor image and the negative image
Thus, the goal of the triplet loss is to minimize the distance between the anchor and the positive images while maximizing the distance between the anchor and the negative images. Thus, two images with the same labels (anchor-positive pair) should have their embeddings close together, while the two images with different labels (anchor-negative pair) should have their embeddings far apart.
Based on the definition of the loss, there are three categories of triplets:
- Easy triplets: Triplets that have a loss of 0 because d(a,p) + margin < d(a,n)
- Semi-hard triplets: Triplets where the negative is not closer to the anchor than the positive, but still have a positive loss: d(a,p) < d(a,n) < d(a,p) + margin
- Hard triplets: Triplets where the negative is closer to the anchor than the positive. d(a,n) < d(a,p)
For embedding learning through triplet loss, we need to generate triplets of images for model training. There are two approaches to triplet generation:
- Offline Triplet Mining: The first way to produce triplets is to find them offline, at the beginning of each epoch, for instance. We compute all the embeddings on the training set, and then only select hard or semi-hard triplets. We can then train one epoch on these triplets.
- Online Triplet Mining: Online triplet mining was introduced in FaceNet. The idea here is to compute useful triplets on the fly, for each batch of inputs. This is much more efficient than offline mining.
I decided to implement the online triplet mining. TensorFlow addons consist of different loss functions like triplet, metric learning, etc., but it is not compatible with TensorFlow ≥2.1.5, hence I adapted it by:
- Referring to the source code of the triplet loss from the TensorFlow Addons GitHub repository.
- Extracting and integrating the loss function directly into my codebase.
- Making necessary modifications to ensure compatibility with the rest of my training pipeline and the current TensorFlow version.
Semi-Hard Triplet Loss is used with a margin of 0.2, and the model is trained for 10 epochs (1000 steps per epoch) with a batch size of 32. After training the model, for a codepoint, its embedding is calculated from the model, and its distance is calculated with every other codepoint’s embedding, and then the top 10 closest codepoints are chosen. The distance metric can be Manhattan, Euclidean, or cosine. This is the first level of filtering. For the second level of filtering, sum sum-squared difference is calculated between the selected codepoint and the candidate closest codepoints for more robust filtering.
Training Notebook: https://github.com/unicode-org/ml-confusables-generator/blob/main/deep2025/source/training_triplets.ipynb
Model training history: https://docs.google.com/spreadsheets/d/1a3iT9RfvdFCJfLykyEdtTInLBm16SwtDMc1WqCAQbWY/edit?usp=sharing
- The evaluation of the homoglyph detection model is based on ground truth clusters derived from the official confusables.txt file provided by the Unicode Character Database (UCD). This file lists known homoglyph mappings and serves as a reliable reference for assessing the correctness of the predicted clusters.
- The primary evaluation metric is Recall, computed for each codepoint as:
Where:
- Predicted Cluster: The set of characters that the model identifies as visually similar to the given codepoint
- Actual Cluster: The set of confusable characters listed for that codepoint in confusables.txt
- Initially, with a batch size of 32 and k = 4, the recall was 0.52, i.e., it could cover only 52% of the confusables.txt space for LTR scripts.
- When I increased the batch size to 64 and kept the same value for k, then the recall increased to nearly 70%.
- Shane Carr for giving access to the Google Cloud.
- Roozbeh for guiding on the fonts to be used.
- Frank, for mentoring me.
unicode-org/ml-confusables-generator#31
- Extend the project for the Brahmic scripts
- Contribute to other Unicode projects
- Tensorflow
- Embedding learning and triplet loss
- Unicode terminologies
- Hyperparameter tuning
- Google Cloud Platform
I am truly grateful for the knowledge and skills I've gained, which have significantly contributed to my development as a developer.