Real Estate Lead Automation: 100% Traceability & Zero Infra Cost

The Win

Reduced prospecting time from days of manual labor to seconds of automated processing. By building a custom multi-source mining engine, I delivered a 100% auditable lead pipeline for a law firm with zero infrastructure costs.

• 97+ unique qualified leads delivered in the first cycle.
• 10 banks and 7 aggregator portals integrated into a single schema.
• Zero data loss through canonical hash deduplication.

Situation

The prospecting process was an operational bottleneck. An intern manually scanned auctioneer portals, resulting in data loss and a lack of standardization. The scale involved ten banks (BB, Bradesco, Santander, Itaú, etc.) and multiple aggregator portals with no available APIs.

The business constraint was clear: weekly delivery of qualified leads in a spreadsheet, with zero external infrastructure costs (no paid proxies, no cloud).

Task

As the solo engineer and PO, my role was to design the architecture, implement the scrapers, normalize the data, and manage client expectations through short delivery cycles. The success criterion was the delivery of unique leads, traceable back to the original URL, in an auditable format.

Action

Decoupled Pipeline Architecture

I implemented the system as a four-stage pipeline with disk persistence: data/raw/ → data/parsed/ → data/normalized/ → data/output/.

graph LR
    subgraph Fontes ["Heterogeneous Sources"]
        A[Portal Zuk]
        B[Seu Imóvel BB]
        C[Bradesco Vitrine]
        D[Viacredi/Ailos]
    end
 
    E[Collector Layer] --> F[/data/raw/]
    F --> G[Parser/Normalizer]
    G --> H[/data/normalized/]
    H --> I[Deduper Hash]
    I --> J[[XLSX Delivery]]
 
    A & B & C & D --> E

Key Technical Decisions

1. Single Schema: I translated the varied vocabulary of 7+ portals into a single business schema, simplifying final consolidation.
2. Hash Deduplication: I used a hash of invariant fields (address + bank + value) to ensure the client never received duplicate leads, regardless of URL changes on the portals.
3. Mission Control: I implemented a rigorous cycle log (Status/Delivered/Pending) that saved the project from scope creep after Cycle 1.

Result

Impact Metrics

Qualitative Impact

The project didn't just automate the process; it transformed the client's operation. The --build-delivery-banks command generates the contact-ready spreadsheet, reducing manual work from days to seconds. The pluggable architecture allowed adding new sources in less than 24 hours as business needs evolved.

Compliance Note: All names of debtors and sensitive data were masked or removed. This case focuses on data engineering and the effectiveness of the public collection pipeline.