Regex basics in Python: match, search and validate one field with re

Most of us meet regular expressions the same way: a ticket says "reject bad file names" or "pull the order number out of this message", someone pastes a pattern from memory, and it works on the three examples in the ticket. A week later a user uploads report.mp4.exe and it passes. The pattern was not wrong in an exotic way. It was wrong in one of the four or five basic ways that every regex beginner trips over.

This guide covers the small core of regex that handles most day-to-day work in Python: the shorthand tokens, character sets, quantifiers, anchors, and the functions in the re module that decide how a pattern is applied. We finish by building one pattern that validates one field, an uploaded video filename, and testing it properly. Combining many patterns and extracting several values at once is a separate topic; here we stay with one pattern, one input, one yes-or-no answer.

What a regex actually does

A regular expression describes the shape of text. It does not know what an email or a date is; it only knows sequences of characters. When you hand a pattern and a string to the engine, it walks through the string trying to line the pattern up against it, character by character, and reports where (or whether) it succeeded.

That is the right mental model for everything below: a pattern is a sequence of small rules, and each rule consumes some characters of the input.

The re module in two minutes

Python ships regex support in the standard library, so there is nothing to install:

import re

Always write patterns as raw strings (r"..."). In a normal string, "\d" happens to survive, but "\b" becomes a backspace character, and you will spend an afternoon wondering why your word boundary never matches. Raw strings hand the backslash to the regex engine untouched.

The functions differ in where the pattern is allowed to match:

import re

text = "invoice 2041 paid"

print(re.search(r"\d+", text))      # <re.Match object; span=(8, 12), match='2041'>
print(re.match(r"\d+", text))       # None, the string does not start with a digit
print(re.fullmatch(r"\d+", "2041")) # <re.Match object; span=(0, 4), match='2041'>
print(re.findall(r"\d+", "3 boxes, 12 units, 40 kg"))  # ['3', '12', '40']

For validation, fullmatch is almost always what you want. We come back to why in the anchors section.

Shorthand tokens

These are the building blocks you will type most often:

import re

log = "user_7 retried 3 times\tstatus=fail"

print(re.findall(r"\d", log))   # ['7', '3']
print(re.findall(r"\w+", log))  # ['user_7', 'retried', '3', 'times', 'status', 'fail']
print(re.findall(r"\s", log))   # [' ', ' ', ' ', '\t']

Two details matter in real code.

First, on str patterns Python's \d and \w are Unicode-aware. \d matches the Arabic-Indic digit "٣" and \w matches "ñ". That is often fine for names, but if a field must be ASCII digits only, write [0-9] or pass re.ASCII:

import re

print(bool(re.fullmatch(r"\d+", "١٢٣")))            # True
print(bool(re.fullmatch(r"\d+", "١٢٣", re.ASCII)))  # False
print(bool(re.fullmatch(r"[0-9]+", "١٢٣")))         # False

Second, . is a wildcard, not a period. If you mean a literal dot, escape it as \.. Forgetting this is the single most common bug in filename and domain patterns: r"report.pdf" also matches reportXpdf.

Character sets and ranges

Square brackets define a set: one character from the listed options.

import re

print(re.findall(r"[aeiou]", "regular"))       # ['e', 'u', 'a']
print(re.findall(r"[A-Z]", "Build 42 OK"))     # ['B', 'O', 'K']
print(re.findall(r"[0-9a-f]", "id: 3fZ9"))     # ['d', '3', 'f', '9']

Rules worth remembering:

• A hyphen between two characters is a range (a-z). A hyphen at the start or end of the set is a literal hyphen ([a-z-]).
• A caret right after the opening bracket negates the set: [^0-9] is "any character that is not a digit".
• Most special characters lose their meaning inside a set. [.] is a literal dot, [+*] is a plus or a star. The exceptions are \, ], ^ at the start and - in the middle.

Quantifiers

A quantifier says how many times the previous unit may repeat. The previous unit is one character, one set, one escape, or one parenthesised group.

import re

samples = "ht, hot, hoot, hooot"

print(re.findall(r"ho*t", samples))     # ['ht', 'hot', 'hoot', 'hooot']
print(re.findall(r"ho+t", samples))     # ['hot', 'hoot', 'hooot']
print(re.findall(r"ho?t", samples))     # ['ht', 'hot']
print(re.findall(r"ho{2}t", samples))   # ['hoot']
print(re.findall(r"ho{1,2}t", samples)) # ['hot', 'hoot']

"Previous unit" is where beginners get bitten. ab+ means "a, then one or more b", not "one or more ab". To repeat a sequence you need a group, (ab)+ or the non-capturing (?:ab)+.

Quantifiers are also greedy by default: they take as much as they can and only give characters back if the rest of the pattern fails. Adding ? after a quantifier (*?, +?) makes it lazy. Greedy versus lazy rarely changes a yes-or-no validation, but it changes what findall and search return, so keep it in mind when you start extracting values.

Anchors

Anchors match a position, not a character.

Read the $ row twice. In Python, $ also matches just before a trailing \n. That makes ^...$ a leaky validator:

import re

pattern = r"^[a-z]+$"
print(bool(re.search(pattern, "admin")))     # True
print(bool(re.search(pattern, "admin\n")))   # True, the newline slipped through
print(bool(re.fullmatch(r"[a-z]+", "admin\n")))  # False

A value with a trailing newline that passes validation and then gets written into a header, a log line or a filename is a classic source of injection bugs. re.fullmatch requires the pattern to consume the whole string, newline included, so it has no such gap. If you must use search, anchor with \A and \Z.

Worked example: validate an uploaded video filename

Requirement from the ticket: accept a filename for a video upload. The base name may contain letters, digits, underscores, hyphens and dots, must not be empty, and must be at most 100 characters. The extension must be one of mp4, mov, mkv or webm, case-insensitive.

We build it one piece at a time.

1. Base name: [A-Za-z0-9_.-]{1,100}. The dot inside the set is literal, the hyphen sits at the end so it is literal too.
2. The separator dot: \., escaped because we want a real dot.
3. The extension: (?:mp4|mov|mkv|webm). The group keeps the alternation contained; without it, \.mp4|mov would mean ".mp4" or "mov anywhere".
4. The whole string must match, so we use fullmatch and the re.IGNORECASE flag.

import re

VIDEO_NAME = re.compile(r"[A-Za-z0-9_.-]{1,100}\.(?:mp4|mov|mkv|webm)", re.IGNORECASE)


def is_valid_video_name(name: str) -> bool:
    return VIDEO_NAME.fullmatch(name) is not None

Compiling once with re.compile keeps the pattern next to its name and avoids repeating flags. Python caches recently used patterns internally, so the speed difference is small; readability is the real gain.

Now the test, which matters more than the pattern. Good regex tests list both what must pass and what must fail, and the failing list should come from the bugs you are trying to prevent:

import pytest

from uploads import is_valid_video_name


@pytest.mark.parametrize("name", [
    "intro.mp4",
    "Q3-review_final.MOV",
    "talk.v2.webm",
    "a.mkv",
])
def test_accepts(name):
    assert is_valid_video_name(name)


@pytest.mark.parametrize("name", [
    "",                      # empty
    ".mp4",                  # no base name
    "clip.mp4.exe",          # wrong final extension
    "clip.mp4\n",            # trailing newline
    "clipXmp4",              # dot must be literal
    "../../etc/passwd.mp4",  # slash is not allowed
    "clip.mp3",              # extension not on the list
    "a" * 101 + ".mp4",      # too long
])
def test_rejects(name):
    assert not is_valid_video_name(name)

python -m pytest -q test_uploads.py

Every rejected case maps to a rule in the pattern. If one of them starts passing after a refactor, the test name tells you which rule broke.

A regex only checks shape. This one stops path separators, but the server should still generate its own storage name and check the file content type, because a file called intro.mp4 can contain anything.

Common mistakes

Key takeaways

• Think of a pattern as a sequence of small rules, each consuming part of the input.
• For one-field validation, use re.fullmatch; it removes the trailing-newline gap of $.
• Quantifiers apply to the previous unit only; group a sequence before repeating it.
• Escape literal dots and wrap alternations in a group.
• \d and \w are Unicode-aware on str; use explicit ranges when the field must be ASCII.
• Write the reject cases first; they are where regex bugs live.

References

• Python documentation, re module: https://docs.python.org/3/library/re.html
• Python Regular Expression HOWTO: https://docs.python.org/3/howto/regex.html
• pytest parametrize documentation: https://docs.pytest.org/en/stable/how-to/parametrize.html