regular expression or Regex (beginner level)
Table of Content
- What is regex
- Background
- Application
- Re module
1- What is Regex
What is Regex: A regular expression is sometimes called a relational expression or RegEx, which is used for character or string matching and, in many cases, find and replace the characters or strings [3]. Regular expressions, or regex, are a powerful tool used for searching and manipulating text strings. With regex, we can define a pattern that matches specific characters, words, or patterns within a text. These patterns can be used to match, search, replace, and manipulate textual data in a variety of ways. Regular expressions are often referred to as regex and can be used easily with python via the `re` library.
2- Background:
Regular expressions are strings you can use that have a special syntax, which allows you to match patterns and find other strings. A pattern is a series of letters or symbols which can map to an actual text or words or punctuation. They can save a lot of time in tasks like searching and replacing text in files, validating text input like passwords and email addresses, and renaming multiple files using a regex pattern to change their extensions [2].
3- Application
- Find Webpages parse email addresses, and remove unwanted strings or characters: You can use regular expressions to do things like find links in a webpage, parse email addresses, and remove unwanted strings or characters.
- Validate passwords: we can create regex patterns to validate passwords based on certain criteria, such as requiring at least one special character, digit, and uppercase letter.
- Manipulate text strings: They are particularly useful in extracting information from text, such as log files, spreadsheets, or even textual documents [2]
- This can be especially helpful in tasks like searching and replacing text in files [2]
- In essence, regular expressions provide a way to define a pattern that matches specific characters, words, or patterns within a text. [2]
4-Common regex patterns
There are hundreds of characters and patterns you can learn and memorize with regular expressions, but to get started, I want to share a few common patterns. The first pattern \w we already saw, it is used to match words. The \d pattern allows us to match digits, which can be useful when you need to find them and separate them in a string. The \s pattern matches spaces, the period is a wildcard character. The wildcard will match ANY letter or symbol. The + and * characters allow things to become greedy, grabbing repeats of single letters or whole patterns. For example, to match a full word rather than one character, we need to add the + symbol after the \w. Using these character classes as capital letters negates them so the \S matches anything that is not a space. You can also create a group of characters you want by putting them inside square brackets, like our lowercase group.
4- Re-Module
What is re-module: This built-in Python module provides all the necessary functionality required for handling patterns and regex [2].
Background: To start using the re module, simply type “import re” at the beginning of your Python file. This will allow you to access the module’s methods and special characters [2].To learn more about the functionality, methods, and attributes of the RE module, you can use the “help” function. With the passing the module’s name as an argument, like “print(help(re)) [2],”
Methods:
1- Search and Match: re.search(pattern, str), re.match(pattern, str). match an entire string or substring based on a pattern or To find a match at the beginning of a string, use a regex pattern that matches the start of the string. Only a match at the beginning of the string will be returned: Search for a pattern, Search for occurrences of the regex pattern inside the target string and return only the first match. We can match a substring by using the re.match method which matches a pattern with a string. It takes the pattern as the first argument, the string as the second and returns a match object, here we see it matched exactly what we expected: abc.We can also use special patterns that regex understands, like the \w+ which will match a word. We can see here via the match object representation that it has matched the first word it found.Using the regex module's re.match and re.search are pretty essential tools for Python string processing. Learning when to use search versus match can be challenging, so let's take a look at how they are different. When we use search and match with the same pattern and string with the pattern is at the beginning of the string, we see we find identical matches. That is the case with matching and searching abcde with the pattern abc. When we use search for a pattern that appears later in the string we get a result, but we don't get the same result using match. This is because match will try and match a string from the beginning until it cannot match any longer. Search will go through the ENTIRE string to look for match options. If you need to find a pattern that might not be at the beginning of the string, you should use search. If you want to be specific about the composition of the entire string, or at least the initial pattern, then you should use match [1].
2- Split:re.split(pattern, str)
Split a string on regex. The syntax for the regex library is always to pass the pattern first, and the string second. Depending on the method, it may return an iterator, a new string or a match object. Here we see the re.split method will take a pattern for spaces and a string with some spaces and return a list object with the results of splitting on spaces. This can be used for tokenization, so you can preprocess text using regex while doing natural language processing.
3-findall: re.findall(pattern, str)
This function searches for all non-overlapping occurrences of the regex pattern within the string, and returns them as a list of strings [2].What if we want to find a word which comes multiple times, for that we use the findall operation.
-
Imported the RE module into our program
-
to match any digit between 0 to 9, we created a regex pattern “\d”
-
Used the re.findall() method to match our pattern
- hat “\d” is a special sequence that matches any digit between 0 and 9.
-
Obtained the output of the method, which was two digits 6 and 5
The above operation just finds the prints a string that occurs multiple times in a string. But if we want to know the span of the words in a sentence so that we can get an idea of the placement of the word for that, we use an iteration method finditer operation [3].
4- re.compile('pattern'):
To create a re. Pattern object simply compiles the provided regular expression pattern into a string.
5- Group: Returns the part of the string that matches the pattern we're looking for. If the pattern contains capturing groups, we can use the group method with an argument to get the matched string for a specific group [2].
6- groups(): Returns a tuple of matched strings for all the capturing groups in the pattern [2].
7-finditer:re.finditer(pattern, str): This function also searches for non-overlapping occurrences of **pattern**within str, but instead of returning a list of strings, it returns an iterator that yields match objects. This allows for more efficient use of memory and processing, especially for large strings or patterns [2].
8- Re-Sub: re.sub(pattern, replacement, str): Replace one or more occurrences of a pattern in the string with a replacement [2].
9- Start: start(): Returns the starting position of the matched substring in the original string [2].
10- end(): Returns the ending position of the matched substring in the original string.
11- span(): Returns a tuple of the starting and ending positions of the matched substring in the original string.