Skip to content

Instantly share code, notes, and snippets.

@datfoosteve
Last active January 31, 2022 23:16
Show Gist options
  • Save datfoosteve/a2f80d1e7ff0092243351c841f9d765c to your computer and use it in GitHub Desktop.
Save datfoosteve/a2f80d1e7ff0092243351c841f9d765c to your computer and use it in GitHub Desktop.
Another Regex Tutorial but includes Fundamentals of Regular Language.

Relavent XKCD

Regex is based on set theory logic. Since all computation uses this, not just search engines, There are ties to regex from all the way of understanding how computers work , because regular expressions are used in the theory of computation itself. On how to teach a computer to understand human language. Regex at first looks like computer language, Regex is a very powerful skill but can be at times very fustrating. The power involves being able to do functions without having to type out long code lines in IDE's.

Regular expressions are used in search engines, search and replace dialogs of word processors and text editors, in text processing utilities such as sed and AWK and in lexical analysis. Many programming languages provide regex capabilities either built-in or via libraries, as it has uses in many situations.

Summary

This gist will explain most of the syntax and commands that regex use's, hope to help people gain a better understanding of this concept, and at the same time, searing the information into own my brain; I usually write stuff as informative as possible, trying to take the best out of it, and making it that I am able to go back to it later, and since this is an assignment as well, ill put in extra effort.

For this assignment requirement, the gist will include a Hexdecimal regular expression

^#?([a-f0-9]{6}|[a-f0-9]{3})$/

The assigment requires I set out and explain how this reads. Ill go through every character in this expression to show how its doing the logic. The rest of Gist shows the rules, the syntax, the what and why's and hows of Regex. Ill include some things that can further augment these logical understandings.

What is a (RegEx) Regular Expression?

These are Expressions, just like mathamatical expressions, that denotes the regular language.

Where does Regular expressions stand among all the other types in the hierarchy

[Where it stands, or sits, or flys

What are the characters in regular expressions called?

They are called "Metacharacters"

I included some tables. They will appear here.

And the rest will be explained below

Table of Contents

Regex Components

Anchors

The up arrow indicates beginning of the string being evaluated. While the dollar sign indicates the end of the string being evaluated. For example, ^Let's Go!$, displays the exact string match which starts and ends with Let's Go!

The anchors allow to catch more similiar types of matches even though theres variability the text where errors can occur.

Quantifiers

  • *: 0 or more
  • +: 1 or more
  • ?: 0 or 1
  • {3: Exactly 3
  • {3,}: 3 or more
  • {3,5}: 3, 4 or 5

Note: Quantifiers are greedy - they match as many times as possible. Add a ? after the quantifier to make it ungreedy. By using the ? we then provide a conditon only if it resolves true, then the quanifier will run

"+" allows for us to repeat a number of times as long as they match accepted characters. "{2,6}" this means we need to two characters for the capturing group, but no more than 6.

Grouping Constructs

  • .: Any character except newline (\n)
  • (a|b): a or b
  • (…): Group
  • (?:…): Passive (non-capturing) group
  • [abc]: a, b or c
  • [^abc]: Not a, b or c
  • [a-z]: Letters from a to z
  • [A-Z]: Uppercase letters from A to Z
  • [0-9]: Digits from 0 to 9

Note: Ranges are inclusive.

Grouping allows for checks or matches that are contained within each others bounds depending on what type they are groups as.

Bracket Expressions

  • [abc]: a, b or c
  • [^abc]: Not a, b or c
  • [a-z]: Letters from a to z
  • [A-Z]: Uppercase letters from A to Z
  • [0-9]: Digits from 0 to 9

Note: Ranges are inclusive.

\d indicates any single character that is a digit. This can also be represented by 0-9. This is also known ask character set. It is one of the most ocmmonly used features of regular expression. It allows for you to locate a word even if it is misspelled. It does not matter the order of the characters inside the character class, the results will be identicial. You can also include together a range and single character. For example,[0-9a-fxA-FX] is considered a hexadecimal digit match.

Character Classes

  • \s: Whitespace
  • \S: Not whitespace
  • \w: Word
  • \W: Not word
  • \d: Digit
  • \D: Not digit
  • \x: Hexadecimal digit
  • \O: Octal digit

Flags signify what option the search with take or what the search is defined as. These are part of regular expressions, and can be used seperatly or combined .

The OR Operator

  • |: Alternation

Also called the Union Operator , it is identified with | . Similiar to the condtional "||" it allows for checks instead of having to write out more code for other conditions.

Flags

The Most common flags are Denoted as
- 1. i : the search is case-insensitive
- 2. g : Look at the entire medium until it finds its first match then returns
- 3. m : this is called enabling multiline code where regex is now able to use lines that are not just part of the first line
- 4. s : Mode "dotall" is now enabled, allows for a multiline process that allows "dot" => (.) operator to match a newline
- 5. u : allows processing of surrogate type pairs
- 6. y : sticky mode is now enabled, allows precision control within mediums

The options allow for variability and results that can easily adapt to many different sitations. THis allows even code reusability to different substraits.

Character Escapes

  • \:Escape following character. Used to escape any of the following metacharacters: {}^$.|*+?.
  • \Q: Begin literal sequence
  • \E: End literal sequence

This escape is initated only when you start off with "". Also allows for options included above. With options we can vary the text to give choice on the escape. Must always be used specifically when it involes a template literal \ chararcter

POSIX

  • [:upper:]: Uppercase letters
  • [:lower:]: Lowercase letters
  • [:alpha:]: All letters
  • [:alnum:]: Digits and letters
  • [:digit:]: Digits
  • [:xdigit:]: Hexade­cimal digits
  • [:punct:]: Punctu­ation
  • [:blank:]: Space and tab
  • [:space:]: Blank characters
  • [:cntrl:]: Control characters
  • [:graph:]: Printed characters
  • [:print:]: Printed characters and spaces
  • [:word:]: Digits, letters and underscore

Pattern Modifiers

  • g: Global match
  • i: Case-insensitive
  • m: Multi-line mode. Causes ^ and $ to also match the start/end of lines.
  • s: Single-line mode. Causes . to match all, including line breaks.
  • x: Allow comments and whitespace in pattern
  • e: Evaluate replacement
  • U: Ungreedy mode

8 REGULAR EXPRESSIONS YOU SHOULD KNOW:

  • Matching a Username: /^[a-z0-9_-]{3,16}$/
  • Matching a Password: /^[a-z0-9_-]{6,18}$/
  • Matching a Hex Value: /^#?([a-f0-9]{6}|[a-f0-9]{3})$/
  • Matching a Slug: /^[a-z0-9-]+$/
  • Matching an Email: /^([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})$/
  • Matching a URL: /^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$/
  • Matching an IP Address: /^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$/
  • Matching an HTML Tag: /^<([a-z]+)([^<]+)*(?:>(.*)<\/\1>|\s+\/>)$/

Solution To Assignment

^#?([a-f0-9]{6}|[a-f0-9]{3})$


[^] = >Matches the beginning of the string, or the beginning of a line if the multiline flag (m) is enabled. This matches a position, not a character.


[#?] = Matches 0 or 1 of the preceding token, effectively making it optional.


([a-f0-9]{6}|[a-f0-9]{3}) = Groups multiple tokens together and creates a capture group for extracting a substring or using a backreference.


([a-f0-9]{6}|[a-f0-9]{3})$

  • Within Capturing Group 1 = Then character set [a-f0-9]
  • [a-f0-9] = Natches a character in the range of a to f. case sensitive

{6} = This is the quantifier, and is stating must match 6 of the preceding token, which is the character set


[|] = Acts like a boolean OR. Matches the expression before or after the |. It can operate within a group, or on a whole expression. The patterns will be tested in order. In between both character sets


Within Capturing Group 1 = Then character set [a-f0-9] [a-f0-9] = Natches a character in the range of a to f. case sensitive


{3} = This is the quantifier, and is stating must match 6 of the preceding token, which is the character set


last: {$} = Matches the end of the string, or the end of a line if the multiline flag (m) is enabled. This matches a position, not a character.

Resources

Interactive Game that is themed XKCD!

Compiler

Stephen Puthenpurackal Information obtained from the internet from all various sources.

Email me if you have any questions or concerns!

font-family: 'Open Sans', sans-serif;
font-family: 'Orbitron', sans-serif;
font-family: 'Roboto Mono', monospace;
Description POSIX Non-standard Perl/Tcl Vim Java ASCII
ASCII characters [:ascii:][31] \p{ASCII} [\x00-\x7F]
Alphanumeric characters [:alnum:] \p{Alnum} [A-Za-z0-9]
Alphanumeric characters plus "_" [:word:][31] \w \w \w [A-Za-z0-9_]
Non-word characters \W \W \W [^A-Za-z0-9_]
Alphabetic characters [:alpha:] \a \p{Alpha} [A-Za-z]
Space and tab [:blank:] \s \p{Blank} [ \t]
Word boundaries \b \< \> \b (?<=\W)(?=\w)|(?<=\w)(?=\W)
Non-word boundaries \B (?<=\W)(?=\W)|(?<=\w)(?=\w)
Control characters [:cntrl:] \p{Cntrl} [\x00-\x1F\x7F]
Digits [:digit:] \d \d \p{Digit} or \d [0-9]
Non-digits \D \D \D [^0-9]
Visible characters [:graph:] \p{Graph} [\x21-\x7E]
Lowercase letters [:lower:] \l \p{Lower} [a-z]
Visible characters and the space character [:print:] \p \p{Print} [\x20-\x7E]
Punctuation characters [:punct:] \p{Punct} [][!"#$%&'()*+,./:;<=>?@\^_`{|}~-]
Whitespace characters [:space:] \s \_s \p{Space} or \s [ \t\r\n\v\f]
Non-whitespace characters \S \S \S [^ \t\r\n\v\f]
Uppercase letters [:upper:] \u \p{Upper} [A-Z]
Hexadecimal digits [:xdigit:] \x \p{XDigit} [A-Fa-f0-9]
Metacharacter Description
^ Matches the starting position within the string. In line-based tools, it matches the starting position of any line.
. Matches any single character (many applications exclude newlines, and exactly which characters are considered newlines is flavor-, character-encoding-, and platform-specific, but it is safe to assume that the line feed character is included). Within POSIX bracket expressions, the dot character matches a literal dot. For example, a.c matches "abc", etc., but [a.c] matches only "a", ".", or "c".
[ ] A bracket expression. Matches a single character that is contained within the brackets. For example, [abc] matches "a", "b", or "c". [a-z] specifies a range which matches any lowercase letter from "a" to "z". These forms can be mixed: [abcx-z] matches "a", "b", "c", "x", "y", or "z", as does [a-cx-z].

The - character is treated as a literal character if it is the last or the first (after the ^, if present) character within the brackets: [abc-], [-abc]. Note that backslash escapes are not allowed. The ] character can be included in a bracket expression if it is the first (after the ^) character: []abc].

[^ ] Matches a single character that is not contained within the brackets. For example, [^abc] matches any character other than "a", "b", or "c". [^a-z] matches any single character that is not a lowercase letter from "a" to "z". Likewise, literal characters and ranges can be mixed.
$ Matches the ending position of the string or the position just before a string-ending newline. In line-based tools, it matches the ending position of any line.
( ) Defines a marked subexpression. The string matched within the parentheses can be recalled later (see the next entry, \n). A marked subexpression is also called a block or capturing group. BRE mode requires \( \).
\n Matches what the nth marked subexpression matched, where n is a digit from 1 to 9. This construct is vaguely defined in the POSIX.2 standard. Some tools allow referencing more than nine capturing groups. Also known as a backreference. backreferences are only supported in BRE mode
* Matches the preceding element zero or more times. For example, ab*c matches "ac", "abc", "abbbc", etc. [xyz]* matches "", "x", "y", "z", "zx", "zyx", "xyzzy", and so on. (ab)* matches "", "ab", "abab", "ababab", and so on.
{m,n} Matches the preceding element at least m and not more than n times. For example, a{3,5} matches only "aaa", "aaaa", and "aaaaa". This is not found in a few older instances of regexes. BRE mode requires \{m,n\}.
Metacharacter Description
? Matches the preceding element zero or one time. For example, ab?c matches only "ac" or "abc".
+ Matches the preceding element one or more times. For example, ab+c matches "abc", "abbc", "abbbc", and so on, but not "ac".
| The choice (also known as alternation or set union) operator matches either the expression before or the expression after the operator. For example, abc|def matches "abc" or "def".
Meta­character(s) Description Example
. Normally matches any character except a newline.
Within square brackets the dot is literal.
$string1 = "Hello World\n";
if ($string1 =~ m/...../) {
  print "$string1 has length >= 5.\n";
}

Output:

Hello World
 has length >= 5.
( ) Groups a series of pattern elements to a single element.
When you match a pattern within parentheses, you can use any of $1, $2, ... later to refer to the previously matched pattern. Some implementations may use a backslash notation instead, like \1, \2.
$string1 = "Hello World\n";
if ($string1 =~ m/(H..).(o..)/) {
  print "We matched '$1' and '$2'.\n";
}

Output:

We matched 'Hel' and 'o W'.
+ Matches the preceding pattern element one or more times.
$string1 = "Hello World\n";
if ($string1 =~ m/l+/) {
  print "There are one or more consecutive letter \"l\"'s in $string1.\n";
}

Output:

There are one or more consecutive letter "l"'s in Hello World.
? Matches the preceding pattern element zero or one time.
$string1 = "Hello World\n";
if ($string1 =~ m/H.?e/) {
  print "There is an 'H' and a 'e' separated by ";
  print "0-1 characters (e.g., He Hue Hee).\n";
}

Output:

There is an 'H' and a 'e' separated by 0-1 characters (e.g., He Hue Hee).
? Modifies the *, +, ? or {M,N}'d regex that comes before to match as few times as possible.
$string1 = "Hello World\n";
if ($string1 =~ m/(l.+?o)/) {
  print "The non-greedy match with 'l' followed by one or ";
  print "more characters is 'llo' rather than 'llo Wo'.\n";
}

Output:

The non-greedy match with 'l' followed by one or more characters is 'llo' rather than 'llo Wo'.
* Matches the preceding pattern element zero or more times.
$string1 = "Hello World\n";
if ($string1 =~ m/el*o/) {
  print "There is an 'e' followed by zero to many ";
  print "'l' followed by 'o' (e.g., eo, elo, ello, elllo).\n";
}

Output:

There is an 'e' followed by zero to many 'l' followed by 'o' (e.g., eo, elo, ello, elllo).
{M,N} Denotes the minimum M and the maximum N match count.
N can be omitted and M can be 0: {M} matches "exactly" M times; {M,} matches "at least" M times; {0,N} matches "at most" N times.
x* y+ z? is thus equivalent to x{0,} y{1,} z{0,1}.
$string1 = "Hello World\n";
if ($string1 =~ m/l{1,2}/) {
  print "There exists a substring with at least 1 ";
  print "and at most 2 l's in $string1\n";
}

Output:

There exists a substring with at least 1 and at most 2 l's in Hello World
[…] Denotes a set of possible character matches.
$string1 = "Hello World\n";
if ($string1 =~ m/[aeiou]+/) {
  print "$string1 contains one or more vowels.\n";
}

Output:

Hello World
 contains one or more vowels.
| Separates alternate possibilities.
$string1 = "Hello World\n";
if ($string1 =~ m/(Hello|Hi|Pogo)/) {
  print "$string1 contains at least one of Hello, Hi, or Pogo.";
}

Output:

Hello World
 contains at least one of Hello, Hi, or Pogo.
\b Matches a zero-width boundary between a word-class character (see next) and either a non-word class character or an edge; same as

(^\w|\w$|\W\w|\w\W).

$string1 = "Hello World\n";
if ($string1 =~ m/llo\b/) {
  print "There is a word that ends with 'llo'.\n";
}

Output:

There is a word that ends with 'llo'.
\w Matches an alphanumeric character, including "_";
same as [A-Za-z0-9_] in ASCII, and
[\p{Alphabetic}\p{GC=Mark}\p{GC=Decimal_Number}\p{GC=Connector_Punctuation}]

in Unicode,[47] where the Alphabetic property contains more than Latin letters, and the Decimal_Number property contains more than Arab digits.

$string1 = "Hello World\n";
if ($string1 =~ m/\w/) {
  print "There is at least one alphanumeric ";
  print "character in $string1 (A-Z, a-z, 0-9, _).\n";
}

Output:

There is at least one alphanumeric character in Hello World
 (A-Z, a-z, 0-9, _).
\W Matches a non-alphanumeric character, excluding "_";
same as [^A-Za-z0-9_] in ASCII, and
[^\p{Alphabetic}\p{GC=Mark}\p{GC=Decimal_Number}\p{GC=Connector_Punctuation}]

in Unicode.

$string1 = "Hello World\n";
if ($string1 =~ m/\W/) {
  print "The space between Hello and ";
  print "World is not alphanumeric.\n";
}

Output:

The space between Hello and World is not alphanumeric.
\s Matches a whitespace character,
which in ASCII are tab, line feed, form feed, carriage return, and space;
in Unicode, also matches no-break spaces, next line, and the variable-width spaces (amongst others).
$string1 = "Hello World\n";
if ($string1 =~ m/\s.*\s/) {
  print "In $string1 there are TWO whitespace characters, which may";
  print " be separated by other characters.\n";
}

Output:

In Hello World
 there are TWO whitespace characters, which may be separated by other characters.
\S Matches anything but a whitespace.
$string1 = "Hello World\n";
if ($string1 =~ m/\S.*\S/) {
  print "In $string1 there are TWO non-whitespace characters, which";
  print " may be separated by other characters.\n";
}

Output:

In Hello World
 there are TWO non-whitespace characters, which may be separated by other characters.
\d Matches a digit;
same as [0-9] in ASCII;
in Unicode, same as the \p{Digit} or \p{GC=Decimal_Number} property, which itself the same as the \p{Numeric_Type=Decimal} property.
$string1 = "99 bottles of beer on the wall.";
if ($string1 =~ m/(\d+)/) {
  print "$1 is the first number in '$string1'\n";
}

Output:

99 is the first number in '99 bottles of beer on the wall.'
\D Matches a non-digit;
same as [^0-9] in ASCII or \P{Digit} in Unicode.
$string1 = "Hello World\n";
if ($string1 =~ m/\D/) {
  print "There is at least one character in $string1";
  print " that is not a digit.\n";
}

Output:

There is at least one character in Hello World
 that is not a digit.
^ Matches the beginning of a line or string.
$string1 = "Hello World\n";
if ($string1 =~ m/^He/) {
  print "$string1 starts with the characters 'He'.\n";
}

Output:

Hello World
 starts with the characters 'He'.
$ Matches the end of a line or string.
$string1 = "Hello World\n";
if ($string1 =~ m/rld$/) {
  print "$string1 is a line or string ";
  print "that ends with 'rld'.\n";
}

Output:

Hello World
 is a line or string that ends with 'rld'.
\A Matches the beginning of a string (but not an internal line).
$string1 = "Hello\nWorld\n";
if ($string1 =~ m/\AH/) {
  print "$string1 is a string ";
  print "that starts with 'H'.\n";
}

Output:

Hello
World
 is a string that starts with 'H'.
\z Matches the end of a string (but not an internal line).[52]
$string1 = "Hello\nWorld\n";
if ($string1 =~ m/d\n\z/) {
  print "$string1 is a string ";
  print "that ends with 'd\\n'.\n";
}

Output:

Hello
World
 is a string that ends with 'd\n'.
[^…] Matches every character except the ones inside brackets.
$string1 = "Hello World\n";
if ($string1 =~ m/[^abc]/) {
 print "$string1 contains a character other than ";
 print "a, b, and c.\n";
}

Output:

Hello World
 contains a character other than a, b, and c.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment