**Please strictly adhere to the following resume naming convention:
ALL CAPS, NO SPACES BETWEEN UNDERSCORES
PTN_US_GBAMSREQID_CandidateBeelineID
Example: PTN_US_9999999_SKIPJOHNSON0413
: -
MSP Owner: Michelle Lee
Location: Remote
Duration: 6 months
skill id: 10892152
Role Overview
• We are seeking a highly skilled Databricks & AI Engineer to join our growing FinOps team.
• In this role, your primary focus will be engineering-driven: building robust data ingestion pipelines in Databricks, managing massive datasets across multi-cloud environments using Lakehouse architecture, and leading our transition toward Agentic AI Proof of Concepts (PoCs).
• While this role sits within the FinOps functional area, core FinOps domain knowledge is secondary.
• We are looking for a pure technical powerhouse who can leverage Python, Databricks, and advanced AI frameworks to optimize, track, and revolutionize how we manage cloud spend.
Key Responsibilities
Data Pipeline Engineering
• Design, build, and maintain scalable, automated data ingestion pipelines to collect cloud billing, usage, and performance data from various sources in Databricks.
Big Data Processing
• Leverage Databricks (Spark/PySpark) to process, transform, and clean large-scale unstructured and structured cloud financial data.
API Integration
• Develop and maintain API integrations (specifically utilizing FastAPI) to pull data from diverse vendor platforms and internal systems.
Multi-Cloud Data Management
• Query and analyze massive datasets across AWS (Athena), GCP (BigQuery), and Azure (Data Lake & Cosmos DB).
AI & Automation Innovations
• Architect and implement next-generation Agentic AI workflows and PoCs to automate cloud cost anomalies, forecasting, and recommendations.
Version Control & DevOps
• Manage codebase integrity and CI/CD alignment using Git and its associated ecosystem.
Required Skills
Data & Systems Engineering
• Hands-on experience building production-grade data pipelines.
• Experience working natively with large datasets using distributed computing (Spark/PySpark).
Big Data Ecosystem
• Strong working knowledge of Databricks.
• Familiarity with Data Lake architecture.
• Familiarity with Medallion Architecture.
Programming & Querying
• Advanced proficiency in Python.
• Advanced proficiency in SQL.
API Development
• Proven experience creating and consuming RESTful API calls to aggregate data.
Cloud Architecture
• Solid understanding of multi-cloud platforms, specifically:
o Azure Data Lake
o Cosmos DB
o AWS
o GCP
Version Control
• Proficient with Git.
• Standard branching and collaboration workflows.
Preferred Qualifications
Agentic AI
• Hands-on experience or deep conceptual understanding of Agentic AI frameworks, specifically:
o LangChain
o LangGraph
Web Frameworks
• Experience with FastAPI.
Cloud Data Warehousing
• Experience with:
o GCP BigQuery
o AWS Athena
FinOps Familiarity
• Basic understanding of cloud economics.
• Billing structures.
• FinOps framework principles.
Skills: Digital : Databricks ~ Data Build Tool
Experience Required: 8-10
ALL CAPS, NO SPACES BETWEEN UNDERSCORES
PTN_US_GBAMSREQID_CandidateBeelineID
Example: PTN_US_9999999_SKIPJOHNSON0413
: -
MSP Owner: Michelle Lee
Location: Remote
Duration: 6 months
skill id: 10892152
Role Overview
• We are seeking a highly skilled Databricks & AI Engineer to join our growing FinOps team.
• In this role, your primary focus will be engineering-driven: building robust data ingestion pipelines in Databricks, managing massive datasets across multi-cloud environments using Lakehouse architecture, and leading our transition toward Agentic AI Proof of Concepts (PoCs).
• While this role sits within the FinOps functional area, core FinOps domain knowledge is secondary.
• We are looking for a pure technical powerhouse who can leverage Python, Databricks, and advanced AI frameworks to optimize, track, and revolutionize how we manage cloud spend.
Key Responsibilities
Data Pipeline Engineering
• Design, build, and maintain scalable, automated data ingestion pipelines to collect cloud billing, usage, and performance data from various sources in Databricks.
Big Data Processing
• Leverage Databricks (Spark/PySpark) to process, transform, and clean large-scale unstructured and structured cloud financial data.
API Integration
• Develop and maintain API integrations (specifically utilizing FastAPI) to pull data from diverse vendor platforms and internal systems.
Multi-Cloud Data Management
• Query and analyze massive datasets across AWS (Athena), GCP (BigQuery), and Azure (Data Lake & Cosmos DB).
AI & Automation Innovations
• Architect and implement next-generation Agentic AI workflows and PoCs to automate cloud cost anomalies, forecasting, and recommendations.
Version Control & DevOps
• Manage codebase integrity and CI/CD alignment using Git and its associated ecosystem.
Required Skills
Data & Systems Engineering
• Hands-on experience building production-grade data pipelines.
• Experience working natively with large datasets using distributed computing (Spark/PySpark).
Big Data Ecosystem
• Strong working knowledge of Databricks.
• Familiarity with Data Lake architecture.
• Familiarity with Medallion Architecture.
Programming & Querying
• Advanced proficiency in Python.
• Advanced proficiency in SQL.
API Development
• Proven experience creating and consuming RESTful API calls to aggregate data.
Cloud Architecture
• Solid understanding of multi-cloud platforms, specifically:
o Azure Data Lake
o Cosmos DB
o AWS
o GCP
Version Control
• Proficient with Git.
• Standard branching and collaboration workflows.
Preferred Qualifications
Agentic AI
• Hands-on experience or deep conceptual understanding of Agentic AI frameworks, specifically:
o LangChain
o LangGraph
Web Frameworks
• Experience with FastAPI.
Cloud Data Warehousing
• Experience with:
o GCP BigQuery
o AWS Athena
FinOps Familiarity
• Basic understanding of cloud economics.
• Billing structures.
• FinOps framework principles.
Skills: Digital : Databricks ~ Data Build Tool
Experience Required: 8-10