String and Text Processing

Text data is everywhere – user reviews, emails, logs, social media posts, medical records. Cleaning and transforming text is one of the most common tasks in data science. This lesson covers the tools you need.

String Fundamentals

Strings in Python are immutable sequences of Unicode characters. Once created, they cannot be changed – every operation produces a new string.

String length: counts characters.

Slicing: extracts characters from index to . Negative indices count from the end: accesses the -th character from the right.

Concatenation: joins two strings. Repeating: repeats exactly times.

Membership: tests if character appears in string .

Essential String Methods

Case and Whitespace

Searching and Testing

Splitting and Joining

String Formatting

f-Strings (Python 3.6+)

f-strings are the preferred way to format strings. They are fast, readable, and support expressions.

Other Formatting Methods

Regular Expressions

The re module provides pattern matching for complex text operations.

Basic Patterns

Common Regex Patterns

Regex Quick Reference

Pattern matching: matches either or . Quantifiers: (zero or more), (one or more), (zero or one).

Character classes: matches digits, matches word characters, matches whitespace. Negations: , , .

Anchoring: matches the start, $s$$ matches the end of the string.

.        Any character (except newline)
\d       Digit [0-9]
\w       Word character [a-zA-Z0-9_]
\s       Whitespace
\b       Word boundary
^        Start of string
$        End of string
*        0 or more
+        1 or more
?        0 or 1
{n}      Exactly n times
{n,m}    Between n and m times
[abc]    Character set
[^abc]   Negated set
(abc)    Capture group
(?:abc)  Non-capturing group
a|b      Alternation (a or b)

String Processing Pipeline

Text Cleaning for Data Science

Common Cleaning Pipeline

Unicode Normalization

Tokenization

Stopword Removal

Practical Examples

Extracting Features from Text

Batch Text Processing

Key Takeaways

• Strings are immutable – every operation creates a new string.
• f-strings are the best way to format strings in modern Python.
• Regex is powerful but start simple; use re.findall() and re.sub() most often.
• Text cleaning (lowercasing, removing noise, tokenizing) is essential before analysis.
• Always match your cleaning approach to your specific data and problem.