Senior Data Engineer at Everest Reinsurance
New York, United States
Shanmuka Siva Varma Chekuri
Council Member
Shanmuka Siva Varma Chekuri is an experienced Data Engineer with 6+ years specializing in designing and optimizing scalable ETL pipelines and cloud-based data solutions across Azure technologies. Currently serving as Senior Data Engineer at Everest Reinsurance since January 2023, he leads design and deployment of cloud data pipelines automating claims and ERP workflows across the U.S., U.K., EU, and APAC, with expertise in Azure Data Factory, Databricks, Synapse Analytics, Delta Lake, and integrating AI/ML solutions including RAG pipelines and NLP-based models.
At Everest Reinsurance, Shanmuka architects global-scale data infrastructure serving multi-region operations. Using Azure Databricks, ADF, and Delta Lake, he unified fragmented regional data flows and built automated validation, reconciliation, and control-table governance—making reconciliation cycles 40% faster with fully traceable audits and zero data loss. He developed ETL pipelines achieving 20% workflow efficiency gains, migrated critical on-premises systems to Azure Cloud reducing processing times by 30%, and built ELT pipelines from Azure Data Lake to Snowflake enhancing analytics efficiency by 30%. He developed a conversational AI chatbot powered by RAG principles reducing query resolution time by 25%, optimized NLP pipelines for document classification using Azure Cognitive Services, and achieved 40% reduction in Spark job execution times through advanced performance tuning.
As Core Data Engineering Lead at Unityware AI (2024-Present), Shanmuka architected a multi-region lakehouse powering an AI-driven payroll and compliance platform for SMEs across the U.S., Canada, India, Dubai, and Australia. He designed real-time ingestion, jurisdiction-specific tax models, and anomaly-detection engines—achieving 60% faster payroll runs, zero compliance errors, and securing investor-backed funding for global rollout. Investors highlighted his data platform design as a key differentiator driving funding decisions.
His technical innovation includes creating a Multi-File Set Validation Framework and Concurrency-Safe MERGE Pattern now used across multiple ASG projects as internal standards, eliminating ingestion errors and stabilizing concurrent jobs while establishing best-practice standards for enterprise data reliability. He is recognized as a technical authority in data engineering and cloud automation, frequently invited to review system designs, conduct governance audits, and mentor engineers. Senior leaders consistently cite his frameworks as "best-practice models" for financial data integrity, with engineers from the U.S., U.K., India, Singapore, and Australia seeking guidance on implementing similar architectures.
At Everest Reinsurance, Shanmuka architects global-scale data infrastructure serving multi-region operations. Using Azure Databricks, ADF, and Delta Lake, he unified fragmented regional data flows and built automated validation, reconciliation, and control-table governance—making reconciliation cycles 40% faster with fully traceable audits and zero data loss. He developed ETL pipelines achieving 20% workflow efficiency gains, migrated critical on-premises systems to Azure Cloud reducing processing times by 30%, and built ELT pipelines from Azure Data Lake to Snowflake enhancing analytics efficiency by 30%. He developed a conversational AI chatbot powered by RAG principles reducing query resolution time by 25%, optimized NLP pipelines for document classification using Azure Cognitive Services, and achieved 40% reduction in Spark job execution times through advanced performance tuning.
As Core Data Engineering Lead at Unityware AI (2024-Present), Shanmuka architected a multi-region lakehouse powering an AI-driven payroll and compliance platform for SMEs across the U.S., Canada, India, Dubai, and Australia. He designed real-time ingestion, jurisdiction-specific tax models, and anomaly-detection engines—achieving 60% faster payroll runs, zero compliance errors, and securing investor-backed funding for global rollout. Investors highlighted his data platform design as a key differentiator driving funding decisions.
His technical innovation includes creating a Multi-File Set Validation Framework and Concurrency-Safe MERGE Pattern now used across multiple ASG projects as internal standards, eliminating ingestion errors and stabilizing concurrent jobs while establishing best-practice standards for enterprise data reliability. He is recognized as a technical authority in data engineering and cloud automation, frequently invited to review system designs, conduct governance audits, and mentor engineers. Senior leaders consistently cite his frameworks as "best-practice models" for financial data integrity, with engineers from the U.S., U.K., India, Singapore, and Australia seeking guidance on implementing similar architectures.
At Steel and Metal Service Center (June 2021 - December 2022), he orchestrated seamless migration from on-premises SQL Server to Azure Synapse Analytics and Azure SQL DB, reducing data loading time by 20%. He leveraged Azure Databricks and Delta Lake achieving 30% processing time improvements, automated pipeline orchestration reducing manual intervention by 40%, and designed disaster recovery solutions ensuring 99.9% system uptime while reducing cluster costs by 20%.
At Innovation Systems (January 2019 - December 2020), he implemented data solutions using SSIS, SQL Server, and Power BI, reducing data loading times by 20% and processing time by 50% through scalable data pipelines. He improved project delivery time by 20% through Agile sprint implementation and maintained comprehensive documentation ensuring knowledge transfer and best practices.
His technical expertise spans Python, SQL, PySpark, Scala, Azure (ADF, Synapse, Databricks, Data Lake, Azure ML, DevOps), AI/ML (RAG pipelines, Vector Databases, NLP, TensorFlow, PyTorch), Snowflake, Power BI, and MLOps workflows with CI/CD pipelines. He holds Microsoft Certified Azure Data Engineer certification, demonstrating commitment to industry standards.
What distinguishes Shanmuka is his ability to architect enterprise-scale data systems that deliver measurable business impact while establishing technical standards adopted across organizations. His expertise in cloud migration, AI integration, and MLOps positions him as an ideal leader for organizations requiring senior data engineering expertise that bridges complex technical challenges with strategic business outcomes across global operations.
At Innovation Systems (January 2019 - December 2020), he implemented data solutions using SSIS, SQL Server, and Power BI, reducing data loading times by 20% and processing time by 50% through scalable data pipelines. He improved project delivery time by 20% through Agile sprint implementation and maintained comprehensive documentation ensuring knowledge transfer and best practices.
His technical expertise spans Python, SQL, PySpark, Scala, Azure (ADF, Synapse, Databricks, Data Lake, Azure ML, DevOps), AI/ML (RAG pipelines, Vector Databases, NLP, TensorFlow, PyTorch), Snowflake, Power BI, and MLOps workflows with CI/CD pipelines. He holds Microsoft Certified Azure Data Engineer certification, demonstrating commitment to industry standards.
What distinguishes Shanmuka is his ability to architect enterprise-scale data systems that deliver measurable business impact while establishing technical standards adopted across organizations. His expertise in cloud migration, AI integration, and MLOps positions him as an ideal leader for organizations requiring senior data engineering expertise that bridges complex technical challenges with strategic business outcomes across global operations.