Data Engineer Interview Questions

20,979 data engineer interview questions shared by candidates

Data integrity: Given transaction partition files (100 files), that are batch ingested with pipelines from storage (like S3) to a distributed DWH. What is the preferred data structure ingestion to allow data integrity? (each invoice is fixed or ingested only once). Details: - each invoice has its unique id, and each invoice contains a list of products to be added or fixed - the ingestion procedure upserts the data: update if the invoice already exists or insert if the invoice is new
avatar

Data Engineer

Interviewed at ciValue

4.2
Mar 16, 2023

Data integrity: Given transaction partition files (100 files), that are batch ingested with pipelines from storage (like S3) to a distributed DWH. What is the preferred data structure ingestion to allow data integrity? (each invoice is fixed or ingested only once). Details: - each invoice has its unique id, and each invoice contains a list of products to be added or fixed - the ingestion procedure upserts the data: update if the invoice already exists or insert if the invoice is new

Viewing 261 - 270 interview questions

Glassdoor has 20,979 interview questions and reports from Data engineer interviews. Prepare for your interview. Get hired. Love your job.