Varnit Singh — Senior Analytics Engineer

About Me

I'm a Senior Analytics Engineer at Optum with nearly 4 years of experience designing and building production data systems. I specialize in creating modular, scalable pipelines and self-service frameworks that empower analyst teams.

My work spans framework development for API-to-warehouse ingestion, orchestration with Airflow and dbt, and deploying dashboards that serve 100+ monthly users. I also build and publish open-source tools — including a PyPI package for Adobe Analytics — and contribute to projects like Preswald.

Outside of work, I've completed 100+ freelance Python projects on Fiverr, ranging from web scraping bots to full web applications.

Experience

Senior Analytics Engineer

Optum · Full-time

Jun 2025 — Present · Gurugram, India (Hybrid)

Facilitating framework development for analyst teams — analysts pull required dashboard data through reusable frameworks.
Built frameworks from scratch: Adobe CJA API → Databricks, Adobe Workfront API → Databricks.
Deployed 8 dashboards with 100+ unique monthly views using Databricks, Power Automate & Power BI.

Analytics Engineer

Optum · Full-time

Jul 2023 — Jun 2025 · 2 yrs

Developed ingestion frameworks: Adobe 1.4 API → Snowflake, 2.0 API → Snowflake.
Set up alerts, logging & monitoring on ingestion/transformation pipelines through Airflow — missed SLAs reduced by 90%.
Created ETLs using Airflow, Cosmos, dbt & Snowflake stack with open-source technologies.
Built a self-service migration product (Python, FastAPI, Selenium) accelerating DOMO → Power BI migrations.

FastAPI Docker dbt Airflow Snowflake

Associate Data Engineer

Optum · Full-time

Jul 2022 — Jul 2023 · 1 yr 1 mo

Implemented modular design for Python ingestion code — reduced new ingestion setup from 2 hours to 15 minutes.
Upgraded the main dashboard from a 24-hour delay to near real-time (1 hour).
Automated compliance-related manual tasks, enhancing efficiency and accuracy.

Domo Power BI Python

Data Engineer Intern

DataGrokr · Internship

Jan 2022 — Jul 2022 · 7 mos (Remote)

Converted Teradata legacy scripts to Snowflake-compatible scripts.
Created POC to auto-load data from XML files on Amazon S3 into Snowflake tables.

Snowflake AWS

Python Developer

Fiverr · Freelance (Part-time)

Aug 2020 — Dec 2021 · 1 yr 5 mos (Remote)

Completed 100+ Python projects: web apps, scripts, automation bots, and web scraping tools.
Achieved Level 1 Seller status on Fiverr.

Python Web Scraping Automation

Featured Projects

Open-source PyPI package for pulling report data from the Adobe Analytics API. Simplifies authentication and data retrieval so teams can get started quickly.

Python PyPI Adobe Analytics

Streamlit app to scrape, analyze and visualize product data from any Shopify store — explore pricing, SEO, categories, stock status and export as CSV.

Streamlit Python E-commerce

Skills & Tools

Languages & Frameworks

Python
SQL
FastAPI
Selenium
Streamlit

Data Engineering

Apache Airflow
dbt (Data Build Tool)
Astronomer Cosmos
ETL / ELT Pipelines

Cloud & Warehouses

Snowflake
Databricks
AWS (S3)
MinIO
Azure

BI & Visualization

Power BI
Power Automate
Apache Superset
Domo

DevOps & Tools

Docker
Git
DuckDB
PostgreSQL

Domains

Adobe Analytics (CJA, 1.4 & 2.0 APIs)
E-commerce Analytics
Healthcare Data
Web Scraping & Automation

Education & Certifications

Bachelor's Degree — Information Technology

University Institute of Engineering and Technology (UIET)

2018 — 2022

Certifications

Tech On the Go: Ethics in AI LinkedIn · Feb 2024
AWS Partner: Accreditation (Technical) Amazon Web Services · Mar 2022

Get in Touch

I'm always open to discussing data engineering challenges, freelance projects, or collaboration opportunities. Feel free to reach out.

LinkedIn Medium Blog