Python Script for Normalizing Multi-Source Property Listing Feeds
Generate a Python script to deduplicate and standardize fragmented residential real estate listings.
Use this template when ingesting raw real estate listings from multiple MLS feeds or broker portals. It builds a robust pandas-based cleaning script that fixes address formatting, standardizes property types, and imputes missing metrics.
Role: Senior Real Estate Data Engineer specializing in MLS aggregation and listing normalization.
Context
- Raw Listing Feed Source: {{raw_listing_file_path}}
- Target Schema Mapping: {{target_schema_mapping}}
- Deduplication Similarity Threshold: {{duplicate_threshold}}
- Metric Imputation Strategy: {{imputation_strategy}}
- Clean Output File Path: {{output_cleaned_path}}
Task
Write a fully executable Python data cleaning script using pandas and regex to ingest unstandardized real estate listing data from {{raw_listing_file_path}}, rectify malformed addresses, standardize inconsistent property categories against {{target_schema_mapping}}, resolve duplicates using {{duplicate_threshold}}, and save the sanitized dataset to {{output_cleaned_path}}.
Method
- Ingest the raw data from {{raw_listing_file_path}} into a pandas DataFrame, inferring and logging data types.
- Parse and normalize mixed address strings into standard fields (street number, street name, unit, city, state, postal code) using regular expressions.
- Map heterogeneous property type strings (e.g., 'TH', 'Townhouse', 'Row Home') to standardized schema values defined in {{target_schema_mapping}}.
- Sanitize numerical attributes including price, lot acreage, and interior square footage by removing non-numeric characters and converting to numeric floats.
- Handle missing numerical attributes using {{imputation_strategy}} grouped by geographic zip code and property category.
- Identify and merge cross-listed duplicate records by applying string similarity matching on normalized address and price using {{duplicate_threshold}}.
- Format and export the final validated DataFrame to {{output_cleaned_path}} with comprehensive processing statistics printed to stdout.
Constraints
- The script MUST be written in clean, PEP 8 compliant Python 3.10+ using only standard library, pandas, and re.
- The script MUST NOT drop records with missing metrics without logging the exact reason and record ID to a quarantine log.
- Type hints and docstrings MUST be provided for every data cleaning function.
- Avoid hardcoded file paths; use the defined variables.
- Include inline defensive assertions for critical non-null fields.
Output format
Provide a single runnable Python script structured into the following sections:
- Module docstring and library imports.
- Configurable constants block mapping to {{raw_listing_file_path}}, {{target_schema_mapping}}, {{duplicate_threshold}}, {{imputation_strategy}}, and {{output_cleaned_path}}.
- Core modular cleaning functions.
- A main execution block (
if __name__ == '__main__':) demonstrating end-to-end execution with console summary metrics.
Self-review
- Confirm that all 5 variables are used in the generated script logic.
- Verify that regex parsing handles edge cases like suite numbers and directional prefixes.
- Ensure execution handles empty dataframes or schema mismatches gracefully without unhandled exceptions.
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.