ETL Pentaho Lead (e-Commerce)

About the role:

As a ETL Pentaho Lead, you will be responsible for extracting and ingesting data from websites using web crawling tools. In this role you will own the creation process of these tools, services, and workflows to improve crawl/scrape analysis, reports, and data management.

We will rely on you to test the data and the scrape to ensure accuracy and quality. You will own the process to identify and rectify any issues with breaks as well as scale scrapes as needed.

Job Responsibilities:

• Perform data extraction and data loading, solving errors and checking filter criteria.
• Understand the steps - input, output, and connection to database.
• Inspect data and understand job/transformation processes.
• Have a basic understanding of Salesforce - fields, objects, and lookups.
• Knowledge of schema, database, and views.
• Provide technical knowledge/expertise to support the requirements definition, design, and development of data warehouse components/subsystems, specifically Extract-Transform Load or Extract-Load-Transform technologies and implementations and/or Change Data Capture technologies and implementations.
• Analyze fallout reports and understand how to get output files.
• Research, design, develop, and modify the ETL and related database functions, implementing changes/enhancements to the system.
• Design ETL processes and develop source to target transformations and load processes.

Requirements:

• Bachelor's degree in computer science or a related field or equivalent demonstrated experience.
• At least 3 - 5 years of experience in Pentaho.
• Experience running large scale web scrapes.
• Experience with Linux/UNIX, HTTP, HTML, JavaScript, and Networking.
• Experience with techniques and tools for crawling, extracting, and processing data.
• Great communication skills (written and spoken in English).

Good to have:

• Familiarity with system monitoring/administration tools.
• Familiarity with version control, open-source practices, and code review.
• Familiarity with Python and SQL.