Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Removing Vowels from Hebrew Unicode Text
/*
* One of the questions that recently came up is how to remove vowels from Hebrew characters in Unicode
* (or any other similar language). A quick look at Hebrew Unicode chart shows that the vowels are all
* located between 0x0591 (1425) and 0x05C7 (1479). With this and Javascript's charCodeAt function, it
* is trivial to strip them out with Javascript as follows
*
* Live demo is available here:
* https://jsfiddle.net/js0ge7gn/
*/
function stripVowels(rawString)
{
var newString = '';
for(j=0; j<rawString.length; j++) {
if(rawString.charCodeAt(j)<1425
|| rawString.charCodeAt(j)>1479)
{ newString = newString + rawString.charAt(j); }
}
return(newString);
}
/* @shimondoodkin suggested even a much shorter way to do this */
function stripVowels2(rawString) {
return rawString.replace(/[\u0591-\u05C7]/g,"")
}
@yakovsh

This comment has been minimized.

Copy link
Owner Author

@yakovsh yakovsh commented Jan 17, 2016

Dovid Harrison suggested the following algorithm in Ruby:

def stripVowels(rawString)
newString = ”
rawString.mb_chars.each_char do |c|
newString << c if c.ord1479
end
return newString
end
@yakovsh

This comment has been minimized.

Copy link
Owner Author

@yakovsh yakovsh commented Jan 17, 2016

Yarix suggested a Google Sheets function to do the same:

=REGEXREPLACE(B1,”[\x{0591}-\x{05C7}]”,””)
@yakovsh

This comment has been minimized.

Copy link
Owner Author

@yakovsh yakovsh commented Jan 17, 2016

Gnarlodious suggested the following in Python:

import unicodedata
nikudnik=”הַבַּיְתָה”
normalized=unicodedata.normalize(‘NFKD’, nikudnik) # Reduce hebrew vowel ניקוד marks
flattened=”.join([c for c in normalized if not unicodedata.combining(c)])
flattened
@flamholz

This comment has been minimized.

Copy link

@flamholz flamholz commented Sep 19, 2016

FYI: the jsfiddle seems to unintentionally strip dashes and non-vowel punctuation. For example, I put in Bereshit 1:12

"וַתּוֹצֵ֨א הָאָ֜רֶץ דֶּ֠שֶׁא עֵ֣שֶׂב מַזְרִ֤יעַ זֶ֨רַע֙ לְמִינֵ֔הוּ וְעֵ֧ץ עֹֽשֶׂה־פְּרִ֛י אֲשֶׁ֥ר זַרְעוֹ־ב֖וֹ לְמִינֵ֑הוּ וַיַּ֥רְא אֱלֹהִ֖ים כִּי־טֽוֹב"

and got

"ותוצא הארץ דשא עשב מזריע זרע למינהו ועץ עשהפרי אשר זרעובו למינהו וירא אלהים כיטוב"

@altoP

This comment has been minimized.

Copy link

@altoP altoP commented Dec 20, 2017

And remove vowels in Java:

private String removeVowels(String hebString){
    String newString = "";
    for(int j=0; j<hebString.length() ; j++) {
        char c = hebString.charAt(j);
        if(hebString.charAt(j)<1425 || hebString.charAt(j)>1479)
            newString = newString + hebString.charAt(j);
    }
    return newString;
}
@avrtau

This comment has been minimized.

Copy link

@avrtau avrtau commented Jan 30, 2018

@flamholz You can just exclude the "Maqaf" (the "dash" connecting two words in the hebrew text, unicode u05BE) in the RegEx: return rawString.replace(/[\u0591-\u05BD\u05BF-\u05C7]/g,"");

@bsesic

This comment has been minimized.

Copy link

@bsesic bsesic commented May 24, 2018

Thanks for the python script @yakovsh! 👍
Based on that script I created a more advanced script in Python. 🐍

🎉 It has a GUI, using Tkinter!

It can read and write text files and also removes only Taamei haMikra characters if yo wish.

🔥 The script is part of an open source bible software I am writing, called Emmeth.

Check it our here: https://github.com/Emmeth/tools/blob/master/scripts/remove_nikkud.py

@canjecricketer

This comment has been minimized.

Copy link

@canjecricketer canjecricketer commented Jun 8, 2018

So this thread has been super helpful. I've been using the Google Sheets function suggested by yakovsh, with avrtau's modification to keep the maqqef and sof pasuq. Here's what I'm using: =REGEXREPLACE(B1,"[(\x{0591}-\x{05BD})OR(\x{05BF}-\x{05C2})OR(\x{05C4}-\x{05C7})]","")

But what I want to know is whether and how I could incorporate similar code into a macro for Word, since that is ultimately what I am using this for--to remove nikkud from Hebrew words and text extracts in a Word document. Currently I copy the vocalized text from the Word doc, paste it into my Google sheet, and then copy/paste that output over the vocalized text back in Word. It would be faster and more efficient if I could just select the vocalized text and run a macro on it with a shortcut key. I'm fairly sure this is possible, but I don't which if any of the code listed above would work. I'm pretty new to Word macros; I'm familiar with creating and editing them, and I understand the basics, but I don't know the syntax myself. Thanks for any suggestions!

@orish1

This comment has been minimized.

Copy link

@orish1 orish1 commented Jan 1, 2020

Thanks for the ideas above. I imported all the data in an Excel spreadsheet and couldn't find a way to do it elegantly using a regexreplace function, say.

This is a bit brute-force, but it worked for me and only took about 10 seconds to convert around 10,000 words and sentences:

Function stripVowels(rawString)
Dim stripped As String
stripped = rawString
For H = 1425 To 1479
stripped = Replace(stripped, ChrW(H), "")
Next
stripVowels = stripped
End Function

And then just type "=stripVowels(A2)" in the cell(s) where you want nikkud-less Hebrew text (obviously replace "A2" with whatever the cell is of the original text).

:)

@orish1

This comment has been minimized.

Copy link

@orish1 orish1 commented Jan 1, 2020

Here's the variation required for a Word Find-and-Replace macro:

Sub StripNkudot()
Application.DisplayAlerts = False
For H = 1425 To 1479
Selection.Find.ClearFormatting
Selection.Find.Replacement.ClearFormatting
With Selection.Find
.Text = ChrW(H)
.Replacement.Text = ""
.Forward = True
.Wrap = wdFindStop
.Format = False
.MatchCase = False
.MatchWholeWord = False
.MatchKashida = False
.MatchDiacritics = False
.MatchAlefHamza = False
.MatchControl = False
.MatchWildcards = False
.MatchSoundsLike = False
.MatchAllWordForms = False
End With
Selection.Find.Execute replace:=wdReplaceAll
Next
Application.DisplayAlerts = True
End Sub

(You might want to comment out DisplayAlerts=False if you want to see what's going on, but if you have a lot of text then switching off the display will speed up the process considerably.)

@dir01

This comment has been minimized.

Copy link

@dir01 dir01 commented Apr 10, 2020

Golang example based on other comments on this gist: https://play.golang.org/p/B9VRe-L5lS3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.