AI & SAP Integration Expert – LLM / MCP / SAP On-Premise

Location

Remote / Europe preferred

Start

ASAP

Duration

6+ months

Language

English

Mission Context

As part of a broader SAP transformation and innovation roadmap, our client is looking for a senior AI & SAP Integration Expert to help introduce Generative AI capabilities into an existing SAP S/4HANA On-Premise environment.

Because the client is not running SAP S/4HANA Cloud / RISE, they do not have access to SAP Joule and are therefore exploring alternative enterprise AI architectures leveraging technologies such as Microsoft Copilot, ChatGPT/OpenAI, MCP servers, and custom LLM integrations.

The objective is to identify, design, and implement concrete AI use cases integrated directly with SAP backend systems while respecting enterprise security, governance, and on-premise architecture constraints.

Key Responsibilities

• Design enterprise AI architectures connected to SAP S/4HANA On-Premise landscapes
• Integrate LLMs such as OpenAI / ChatGPT, Microsoft Copilot, Azure OpenAI, or similar solutions with SAP environments
• Build and manage MCP server integrations and AI orchestration layers
• Design secure API-based communication between SAP backend systems and AI services
• Implement AI-powered use cases across SAP business processes (finance, procurement, supply chain, operations, etc.)
• Define and implement RAG (Retrieval-Augmented Generation) approaches leveraging SAP enterprise data
• Work on AI agent / copilot concepts connected to SAP transactional environments
• Collaborate with SAP functional and technical teams to expose relevant business data and services
• Support integration mechanisms using APIs, OData services, RFC/BAPI, middleware, SAP PI/PO, SAP Integration Suite, or equivalent technologies
• Contribute hands-on to configuration, prototyping, testing, debugging, and deployment activities
• Ensure security, governance, access management, and compliance considerations are integrated into the AI architecture
• Present architecture recommendations, prototypes, and implementation progress to technical and business stakeholders
• Support delivery within an international and complex enterprise environment

Required Skills & Expertise

• Strong experience with SAP S/4HANA or ECC On-Premise environments
• Proven experience integrating enterprise AI / Generative AI solutions into existing business applications
• Strong hands-on experience with LLM ecosystems:
  • OpenAI / ChatGPT
  • Azure OpenAI
  • Microsoft Copilot
  • AI Agents / Copilot frameworks
• Experience with MCP servers and AI orchestration architectures
• Strong understanding of enterprise integration patterns:
  • REST APIs
  • OData
  • RFC/BAPI
  • Middleware
  • Event-driven architectures
• Experience implementing RAG architectures and connecting LLMs to enterprise knowledge sources
• Good understanding of vector databases, embeddings, prompt orchestration, and AI governance concepts
• SAP technical understanding:
  • ABAP APIs
  • SAP integration layers
  • SAP security concepts
  • SAP data structures
• Experience working in secure enterprise environments with strong governance requirements
• Ability to move from architecture design to hands-on implementation
• Strong stakeholder management and communication capabilities

Nice to Have

• Experience with SAP BTP
• Experience with AI frameworks such as LangChain, Semantic Kernel, AutoGen, CrewAI, or similar
• Experience deploying AI solutions in regulated or industrial environments
• Knowledge of SAP Joule architecture and SAP AI strategy
• Experience with Azure AI Foundry or Microsoft AI ecosystem
• Experience with containerized AI deployments and Kubernetes environments

Profile

• Senior AI integration expert with strong SAP ecosystem exposure
• Hybrid architect / engineer profile
• Highly hands-on and delivery-focused
• Comfortable working in ambiguous innovation environments
• Able to challenge architecture decisions and propose pragmatic solutions
• Strong enterprise mindset with focus on scalability, security, and operationalization