Real Estate Platform with ETL & Scraping
Backend / Data Engineering / Django
A Django application built on aggregated property data from multiple websites. Data was collected through scraping and APIs, put through an ETL process, and then surfaced through the Django site.
- Year
- 2024
- Role
- Backend & data engineer
- Source
- Public repository
Technologies
- Python
- Django
- ETL
- Web Scraping
- APIs
- SQL
The problem
Property listings live on many sites in many shapes. Comparing them requires collecting them first, and then reconciling formats that agree on nothing.
The solution
A collection layer using scraping and APIs, an ETL stage that normalises everything into one schema, and a Django application that serves the unified result.
Architecture
How it is put together.
Collection
Listings are gathered from multiple sources via scraping and APIs.
Transform
ETL normalises differing formats into a single schema.
Storage
Clean records are persisted in the application database.
Application
Django serves browsing and search over the unified dataset.
Key features
What it actually does.
- Multi-source data collection via scraping and APIs
- ETL normalisation into a single schema
- Django application over the resulting dataset
Engineering challenges
The parts that were hard.
Every source disagrees
Field names, units and completeness vary per site. The transform stage carries most of the complexity.
Outcomes
What came out of it.
Qualitative outcomes only — no invented benchmarks or metrics.
One browsable dataset assembled from many sources
A repeatable collection-to-application pipeline
Next project
Secure File Sharing System
Client-side encryption so the server never sees an unencrypted file.
Let’s build
Want something like this for your business?
Tell me the problem and I will come back with an architecture, a scope and a timeline.