Remote, CA, US
Summary
Led the development of a fully automated and scalable Lead Generation System for the Growth Engineering team, optimizing data scraping, processing, and filtering to support marketing initiatives.
Highlights
Engineered a full-stack web application using Django for the backend and React for the frontend, powering a scalable lead generation system that enhanced data accessibility for marketing.
Optimized data processing workflows with Pandas and managed high-volume background tasks using Celery, significantly enhancing system efficiency and throughput for large datasets.
Implemented advanced web scraping solutions using Scrapy, increasing the volume and quality of contact and job information extracted from diverse online sources by an estimated 30%.
Designed and maintained robust data infrastructure utilizing GCS for storage, MySQL for relational data, Redis for high-speed caching, and BigQuery for analytical data warehousing, ensuring data integrity and rapid access for analytics.
Integrated RabbitMQ for efficient message queuing, enabling seamless communication and processing across distributed system components and improving system reliability.