|
How to Use Regular Expressions (1) |
|
|
|
||
You are visitor No.
|
||
|
1. Introduction Whenever I came across something interesting in a mail that was created with TheBat! like "cleaned" Subject-strings or automagically deleted PGP-lines, I would ask in one of the mailing lists: "How did you do that?". Quite often I would receive a reply like "You will need a regex for that!" And sometimes the result was something like: %QUOTES="%SETPATTREGEXP=""(?is)(-----BEGIN PGP SIGNED.*?\n(Hash:.*?\n)?\s*)?(.*?)(^(- --|--\n|-----BEGIN PGP SIGNATURE)|\z)""%REGEXPBLINDMATCH=""%text""%SUBPATT=""3"""This is only a simple example of those cryptic looking combinations of TB!-Macros and regular expressions which are simply called "regex" by the TB-experts. To me it seemed a random sequence of characters; as if a cat walked across my keyboard. Awkward, arbitrary and cryptic, that at least was my impression until Januk Aggarwal (special thanks to him) gave me a short introduction to regex at TBTECH and my workmate Alfred Rübartsch gave me a copy of Jeffrey Friedls excellent book "Mastering Regular Expressions". Although I entered the fascinating world of Regular Expression with the help of these two, I am still not an expert in the "regexian" language. Anyway, as an advanced beginner, I have dared to write this tutorial to hopefully explain some things and give a good start in "Regular Expressions" to other beginners. This tutorial is meant to bring you into closer contact with the regex topic. Well, let's see how it works; let's see whether we will be able to explain the "regex"-example above by the time we come to the end of this tutorial. Back to top 2. Regular Expressions 2.1. What does "Regular Expression" mean? Regex are not only used in TB! You can find them in quite a lot of different UNIX-tools (e.g. grep), in some programming languages like PERL (Practical Extraction and Report Language, sometimes called 'Pathologically Eclectic Rubbish Lister' <bg>) and even my editor UltraEdit uses them. Laura Lemay wrote in her book "PERL in 21 days" that the term "Regular Expression" makes no sense at first sight (to be honest: even at second sight it still makes no sense to me), because these are not real expressions and furthermore no one really can explain why they are "regular"! Well, let's ignore this; let's simply accept that the term "Regular Expression" has its origin in formal algebra and that they are indeed part of Mathematics. The easiest and most convenient way to define "Regular Expression" is to say: "They are search patterns to match characters in strings." Those of you who have tried to find files using the DOS command line or the search function in the Explorer may have used patterns like: dir *.doc copy *.??t c:\tempThese examples show patterns that consist of letters, stars, question marks and other characters to define which files should be listed or copied. In the first example only files that have the suffix "doc" should be listed. In the second example only files that have a three-letter suffix and a "t" as last character in the suffix should be copied. But these regex are merely wildcards! In no way as mighty as "Regular Expressions". One can't compare them to real regex, which offer much more than wildcards for characters. Back to top 3. Simple Patterns 3.1 Simple Known Characters 3.2 Search Patterns for Metacharacters 3.3 Simple Unknown Characters 3.4 Groups of Characters and Character-Classes 3.5 Overview and Summary To explain some regular expressions and to understand the examples given in this tutorial we have to define how the regex will appear. I will envelope the regular expression in quotation marks ("). If you want to test the regex you will have to copy the part between the "-characters. Testing regular expressions? Yes, sure, this is possible. You have to download a DLL written by Dirk Heiser (RegExTest.zip) and copy it into your TB-directory. Then, when you open the TB-help, you will find a tab called RegEx. Or, if you are using the CHM-Version of the help (this probably applies only to the German version), you can use this DLL by creating a link on your desktop which opens the DLL: "%windir%\system32\rundll32.exe <your_path>regextest.dll, Run" Please, I really recommend that you download this utility. It will make it so much easier to follow the tutorial. Back to top of section Back to top 3.1 Simple Known Characters Ok, let's start with simple search patterns: "give or take" Yes, you won't believe it, this is already a regular expression: it matches the string 'give or take' in a text. Exactly these characters! And no, this does not mean that this pattern matches either 'give' or 'take'. The regular expression only matches if the characters in quotation marks appear somewhere in the text! Regular expressions are stubborn and stupid: they will look for exactly what they are told to search for. They are case sensitive and they are not interested in word boundaries unless told to be so. For example, our first regex will find the characters in the following string: 'You have to forgive or take the consequences!' Back to top of section Back to top 3.2 Search Patterns for Metacharacters Regular expressions can search for any character - alphanumeric, hexadecimal, binary numbers, etc.. A small but important exception are those characters that have a special meaning in regular expressions, the metacharacters. Metacharacters are: * + ? . ( ) [ ] { } \ / | ^ $ (Hi experts: Yes, you are right! I stretched the truth!! These are not actually all metacharacters. But trust me, just assume that I am right for now. We will see later why I prefer to define the above as metacharacters). I will explain these metacharacters later in the tutorial, step by step, as many of them as necessary. Just one thing for now. If you want to search for those characters as they stand you have to tell the regex that you want to do so. The regular expression has to be told that you don't mean to use a metacharacter but want to search for it literally. So you have to "escape" or "mask" the character with another character (which of course is a metacharacter in itself <g>): it is the backslash "\" If you want to match a question mark the regex has to be "\?". If it is a slash you're after, you have to enter "\/". And, although it looks queer, if you want to find a backslash you need to type two of them "\\" Back to top of section Back to top 3.3 Simple Unknown Characters The first metacharacter we are going to learn is a dot "." It represents exactly one unknown character we want to match, no matter what this character might be (Hello experts: let's come to exceptions later. Ok?) "M.ller" will match 'Miller', 'Meller' or 'Millerton' but not 'Milton Keynes'. "h..s" matches 'hips' or 'hers'. And within the word 'house', the same regex will match 'hous'. Later we will learn about some more metacharacters; ones that will allow us to look for more than one unknown character without repeating the dot over and over. Back to top of section Back to top 3.4 Groups of Characters and Character-Classes Some metacharacters define groups of characters, making a very powerful tool. There is a wide variety of these groups. Let's start with the easy ones: "\d" symbolizes a digit. "\d\d" searches for any sequence of two digits. "\w" stands for any letter or the underscore character. This group is called 'alphanumeric characters'. With what we already know we can create our first more complicated looking regex: "Re \[\d\]:" searches for the string 'Re' followed by a space, an opening square bracket, any digit, a closing square bracket and finally a colon in a text. Ooops, that looks like a Subject-line which was created by someone who forgot the %SINGLERE in his reply template ;-) There are -of course- metacharacters which have the opposite meaning: "\W" and "\D" \W is stands for any non-alphanumeric character and \D means any character that is not a digit. Another elegant method to define your own group of characters is to use the square brackets [ ] which stands for 'character classes'. With square brackets, the regex will search for exactly one character, no matter how many characters are in between these brackets: "[AEX]". This combination will match any one-letter string that must be one of A, E or X. You may even define ranges of characters. You don't have to type in every character of the range, no; regexian makes it easy for you: just enter the first character of the range, a hyphen "-" and the last letter: "[e-z]" means that all letters from e to z should be matched. "[AEXe-z]" is a combination of both: a one-letter string with one of A,E,X or any letter within the range e to z. This is a powerful tool in regexian: "[0-1][0-9]\/[0-3][0-9]\/" will match only a MM/DD/ formatted date. Other combinations which are not a date (e.g. 35/47/) won't be found. (Yeah, you're right! My regex will match 19/39/ which isn't a terrestrial date at all. We will get this one later once we have learned some more elements....) You can negate character classes with one keystroke. Just add a "^" after the opening square bracket and that's it. 'Find any character as long as it isn't 1,2,3 or 4!' in regexian is: "[^1-4]". Oh, we should remember this one for later. This funny '^' character has a totally different meaning when not in square brackets! Back to top of section Back to top 3.5 Overview and Summary What did we learn in this chapter?
In the second example the regex searches for three alphanumeric characters followed by a comma, a space, two digits, another space. Next come three alphanumeric characters, a space and finally four digits. Phew, what could this be? Well, it looks like a format for dates again, but this time in an Anglo-American format: Tue, 19 Feb 2002. Well, like the first example, this regex is not perfect. It only matches dates with two-digit days. We will see later how we can modify the regex to find one- or two-digit days Third pattern: the regex looks for two characters and a space. The next character is a square bracket. Then a square bracket follows which isn't escaped by a preceding backslash: this defines group of characters! Any digit in the range 0 to 9 is going to match the string. A square bracket again and a colon. This combination would match 'Re [2]:' In the last example the regex looks for only one character. Any letter is allowed, even capital letters. Why isn't "\w" used? Well, that would include the underscore and perhaps the author doesn't want to match that character ;-) Back to top of section Back to top |
||