Data Engineer

Dig Insights is a tech-enabled research company that helps our clients - global consumer brands - move beyond consumer-centric to decision-centric. Our clients can then go-to-market with innovations that shift consumer decisions in their favor.

Our work is supported by the technologies that we leverage and create. This includes Upsiide, our proprietary innovation insights platform. Upsiide is a SaaS platform that reinvents how enterprise companies screen, optimize, and build a business case for innovation.

Our clients span verticals including CPG, QSR, retail, technology, financial services, and telecommunications. Our work is led by a team of over 250 strategists, insights leaders and data scientists. We work for a global client base out of offices in Toronto, Chicago, and London.

Our success is due to our strong commitment to our clients, and the creativity and dedication of our entire team. Since the beginning, we have been focused on building the smartest consumer insights company and that means hiring people who are bright, creative, resourceful, and kind. People who succeed at Dig are curious, question established norms and are passionate about helping our clients to move their businesses forward. If you want to join a team that takes themselves just seriously enough to produce great work, we’d love to welcome you.

As we continue to grow, both geographically and in our expertise we are looking for people who want to join a high-growth and highly collaborative company.

Data Engineer – About this role

We are building the next generation of our market research data platform, a scalable, well-structured data lake that centralizes all survey, response, and behavioral data in a clean, analytics-ready format. As our new Data Engineer, you will play a foundational role in designing and implementing this platform.

You will work closely with engineering, product, and data science teams to define schema standards, implement data pipelines, strip away business logic from source systems, and ensure the data is performant, discoverable, and trustworthy.

This is a high-impact role where you’ll influence our long-term data architecture and enable new insights, AI capabilities, and product features.

Your primary responsibilities will include, but are not limited to:

Data Lake & Architecture

• Design and implement a scalable data lake to store structured and unstructured market research data.
• Develop and maintain data models, schema definitions, and metadata standards that remove embedded business logic.
• Establish best practices for data ingestion, orchestration, partitioning, and lifecycle management.

Data Pipelines & ETL/ELT

• Build robust, automated ETL/ELT pipelines to ingest data from internal applications (survey systems, analytics tools, custom platforms, etc.).
• Clean, normalize, and transform raw data into intuitive and performant queryable datasets.
• Optimize pipelines for reliability, performance, and cost (e.g., streaming vs batch, compression formats, columnar storage).

Quality, Governance, & Observability

• Implement data quality checks, validation rules, and reconciliation logic.
• Set up monitoring, testing, lineage tracking, and documentation to ensure trustworthy data.
• Collaborate with engineering teams to align data contracts and prevent breaking changes.

Cross-Team Collaboration

• Partner with backend engineers to define interfaces between operational systems and the data lake.
• Work with analysts, data scientists, and product managers to understand requirements and ensure the data lake supports analytical and AI use cases.
• Support building dashboards, BI layers, and APIs that sit on top of the lake.

Experience, Skills and Requirements:

• 3–5+ years of experience as a Data Engineer or similar role.
• Strong experience with modern data lake/storage technologies (e.g., S3, Iceberg, BigQuery, Snowflake, Athena, Lakehouse architectures, Parquet/Delta tables).
• Proficiency in ETL/ELT development using Python, SQL, and workflow/orchestration tools.
• Hands-on experience modeling large, complex datasets for analytics.
• Familiarity with distributed data processing frameworks
• Experience designing APIs or integrations between application databases and analytical layers.
• Understanding of schema evolution, partitioning, indexing, and file format optimization.

Nice to Have

• Experience with market research, survey logic, or data from research platforms.
• Background working with Node.js-based backends.
• Exposure to AI/ML pipelines and feature stores.
• Knowledge of event-driven architectures and real-time streaming data.

Soft Skills

• Strong systems-thinking and ability to design for scalability and long-term maintainability.
• Excellent communication skills — able to translate technical considerations for non-technical stakeholders.
• Proactive, self-directed, and comfortable owning critical data infrastructure.

Why Join Us

• Help architect the core data platform powering next-generation insights and AI products.
• Work with a modern tech stack and a team that values thoughtful design, scalability, and best practices.
• High autonomy, high impact: your work directly shapes how data flows across our entire company.
• Opportunities to influence standards, tooling, and engineering culture.

Work Perks:

• Hybrid working policy which gives the flexibility to employees to work remotely or in one of our office locations
• Unlimited vacation policy
• Reimbursement for health and wellness classes/memberships, and continuous learning
• Medical Insurance
• In-person and virtual social events such as poker night, paint night, trivia night and more!

To find out more about us visit us at www.diginsights.com & www.upsiide.com

Our culture is built on 5 core values: Energy, Excellence, Evolution, Equality and Empathy. We believe that our success is dependent on the diverse talents, skills, and ideas of its staff. We are committed to creating an inclusive work environment and encourage applications from all qualified candidates including those in the BIPOC and LGBTQ communities, and from people with disabilities.

We thank you for your interest in Dig Insights, however, only candidates who are chosen for an interview will be contacted.