Launching a Budget Flight Search Engine in UAE with Wego & Rehlat Data
Overview
With UAE’s rising demand for low-cost airline travel, especially among millennials, one startup aimed to launch a flight price comparison engine focused solely on budget fares. Travel Scrape was brought in to scrape real-time flight listings and fare insights from Wego and Rehlat, powering the platform’s engine with the latest, cheapest, and most flexible options across MENA routes.
Objectives
- Scrape and aggregate budget airline fares across UAE and GCC countries.
- Build a real-time price comparison engine using Wego and Rehlat data.
- Focus on low-cost carriers like Air Arabia, flydubai, SalamAir, Jazeera Airways.
- Enable filters for flexibility, such as refundable, carry-on only, flexible dates.
- Deliver fare alerts, deal-of-the-day features, and predictive price recommendations.
Why Wego & Rehlat?
Both Wego and Rehlat dominate the budget and mobile-first travel segment in the Middle East. These platforms list:
- Dynamic daily fares from regional low-cost carriers
- City-to-city flight matrices with flexible pricing
- Custom discounts, wallet offers, and last-minute deals
Scraping them provides a low-cost fare intelligence layer that’s hyper-relevant to the UAE audience.
Travel Scrape’s Data Strategy
1. Target Market & Route Identification
- Top UAE Budget Routes Extracted:
- Dubai ↔ Muscat
- Abu Dhabi ↔ Riyadh
- Sharjah ↔ Jeddah
- Dubai ↔ Cairo
- Abu Dhabi ↔ Kochi
- Airlines Focused:
- o Air Arabia, flydubai, Jazeera Airways, Air India Express, SalamAir
2. Fields Scraped from Wego & Rehlat
| Field | Sample Value |
|---|---|
| Departure City | Abu Dhabi |
| Arrival City | Riyadh |
| Airline | flydubai |
| Price (One-way) | AED 423 |
| Travel Date | 2025-07-15 |
| Stops | Non-stop |
| Flight Time | 11:30 AM – 1:45 PM |
| Refundable Option | No |
| Fare Trend | “Drop of 12% in last 48 hrs” |
| Booking Source | Wego / Rehlat |
Sample JSON
3. Scraping Architecture
- Used Scrapy + Puppeteer to extract JavaScript-rendered listings.
- Implemented rotating proxy pool across UAE, Oman, and India nodes.
- Scraped twice daily for each airline-route-date combination.
- Aggregated results into Elasticsearch for fast querying and suggestions.
Platform Features Powered by Scraped Data
| Feature | Description |
|---|---|
| Budget Fare Comparator | Shows lowest fare across both platforms in real-time |
| Flex Date Suggestion Engine | Suggests cheapest 3-day fare range before/after query |
| Airline Scorecard | Ranks airlines by fare consistency & cancellation policy |
| Mobile Push Deal Alerts | Triggered by price drops > 15% on watched routes |
| Monthly Fare Graph | Trends for each route and airline for user planning |
Implementation Phases
Phase 1: MVP Launch
- Extracted 20,000+ fare records across 50 popular routes.
- Created basic REST API with filters for price, stops, and carrier.
- Integrated results into a React-based frontend within 30 days.
Phase 2: Dynamic Alerting System
- Introduced real-time fare volatility tracking.
- Enabled “Smart Booking Window” suggestions: e.g., “Book now – fare expected to rise in 3 days.”
Phase 3: API Monetization
- Offered fare trend APIs to travel blogs, comparison sites, and financial apps.
- Added affiliate redirect integration to both Rehlat and Wego for commission-based earnings.
Results Achieved (Within 60 Days)
| Metric | Before Travel Scrape | After Integration |
|---|---|---|
| Average User Session Duration | 1 min 02 sec | 2 min 21 sec |
| Booking Redirection Conversion | 4.8% | 8.3% |
| Daily Active Users (DAU) | 6,300 | 11,500+ |
| Flights Tracked by Users | – | 40,000+ routes |
| External API Usage (partners) | – | 680K+ calls/month |
Data Insights
Top 5 Most Price-Volatile Budget Routes
| Route | Avg. Fluctuation | Airline Focus |
|---|---|---|
| Dubai – Muscat | AED 120 | SalamAir, flydubai |
| Abu Dhabi – Kochi | AED 170 | Air India Express |
| Sharjah – Jeddah | AED 140 | Air Arabia |
| Dubai – Cairo | AED 180 | flynas, Jazeera |
| Abu Dhabi – Riyadh | AED 90 | flydubai, SalamAir |
Client Testimonial
"Travel Scrape gave us the tech backbone we needed. Their data pipelines from Wego and Rehlat helped us get to market fast and stand out in the budget travel space."
Tech Stack Used
- Scraping Layer: Scrapy, Puppeteer, Python
- Storage: Elasticsearch, MongoDB, AWS S3
- Frontend: ReactJS (Client’s Dev Team)
- API Framework: FastAPI
- Monitoring & Alerts: Prometheus + Grafana
Key Challenges & Solutions
| Challenge | Solution Implemented |
|---|---|
| Dynamic JS flight listings | Headless browser scraping (Puppeteer) |
| IP blocks on Wego/Rehlat | UAE-based proxy rotation, delay simulation |
| Inconsistent carrier names | Standardized airline mapping |
| Real-time search window alignment | Synced hourly scrapes with user local time |
| Affiliate link maintenance | Added tracking templates and fallback logic |
Conclusion
Travel Scrape’s scraped data from Wego and Rehlat helped a UAE startup rapidly launch and scale a real-time budget flight comparison platform. The insights allowed the startup to:
- Provide price transparency in a cluttered OTA market
- Drive higher conversions via smart nudges and alerts
- Monetize traffic via affiliate links and APIs
In regions like the UAE, where cost-sensitive travelers are actively seeking deals across multiple platforms, scraping flight fare data isn’t just competitive intelligence—it’s the core of a scalable business model.