Python Developer (Web Scrapping)

We are seeking a Python Developer who specializes in data processing, automation, and web scraping. This role will work closely with the CTO and engineering team to extract, process, and optimize real-time travel data. If you are proficient working with Python, data pipelines, and web scraping, this is an opportunity to have a direct impact on a product that sits at the intersection of travel, rewards, and fintech.

Key Responsibilities:

1. Develop and maintain Python-based automation for data extraction, processing, and analysis.
2. Build and optimize web scraping solutions to collect real-time travel pricing and availability data.
3. Work with third-party APIs and integrate data from multiple sources.
4. Design and maintain data pipelines for processing large datasets efficiently.
5. Implement ETL (Extract, Transform, Load) processes to structure travel and financial data.
6. Collaborate with backend engineers to support API integrations and data workflows.
7. Optimize data storage and retrieval using NoSQL (MongoDB) and relational databases.
8. Use cloud services (AWS, Azure) to deploy and manage automation scripts.

Required Skills and Qualifications:

1. 5+ years of professional experience as a Python developer.
2. Expertise in web scraping frameworks such as Scrapy, Selenium, or PlayWright.
3. Strong background in data processing and automation using Python (Pandas, NumPy, Airflow).
4. Experience with API integration and working with structured/unstructured data.
5. Familiarity with ETL workflows and data transformation techniques.
6. Hands-on experience with NoSQL databases (MongoDB) and data storage optimization.
7. Comfortable with cloud-based scripting and deployment (AWS Lambda, EC2, etc.).
8. Strong problem-solving skills and ability to work independently on data challenges.
9. Bonus: Experience in the travel or financial industries is a plus.

Web Scraping Requirements:

1. API Reverse Engineering – Strong ability to inspect and reverse engineer APIs, including:
2. Analyzing browser network traffic (using Chrome DevTools, Burp Suite, Fiddler, or similar tools).
3. Understanding encrypted or obfuscated API responses.
4. Mimicking authenticated API calls and bypassing security restrictions.
5. Handling Web Security Measures – Experience in bypassing CAPTCHAs, bot protections (Cloudflare, PerimeterX, DataDome, Akamai), and anti-scraping techniques.
6. Browser Automation – Deep understanding of headless browser automation tools such as Playwright, Puppeteer, or Selenium.
7. Data Extraction & Parsing – Ability to extract, clean, and structure data from HTML, JSON, XML, and other formats.
8. Proxy & IP Rotation Management – Experience working with residential proxies, rotating IPs, user-agent spoofing, and fingerprint evasion.
9. Performance Optimization – Ability to optimize scraping scripts for speed, concurrency, and scalability while minimizing detection risks.