Scrapy Expert for Public Data API Integration

Nico Shafiq

1000 - 2000

1 week

We are in search of a skilled developer proficient in Scrapy to assist with a project involving the extraction and integration of data from public APIs. The ideal candidate will have a strong background in web scraping, data parsing, and API interactions, ensuring efficient and accurate data collection.

Responsibilities:

  • Develop and maintain Scrapy spiders to extract data from various public APIs.
  • Ensure data integrity and accuracy during the extraction process.
  • Handle data cleaning and preprocessing as required.
  • Collaborate with our team to integrate the extracted data into our existing systems.
  • Troubleshoot and resolve any issues related to data extraction and integration.

Requirements:

  • Proven experience with Scrapy framework and Python programming.
  • Familiarity with RESTful APIs and JSON data format.
  • Strong understanding of web scraping best practices and data handling.
  • Ability to write clean, efficient, and maintainable code.
  • Excellent problem-solving skills and attention to detail.
  • Good communication skills and ability to work collaboratively.

Required services

Cloud

Databases

Web apps

Required tech stack

BeautifulSoup

JSON

Python

RESTful APIs

Scrapy