Senior Software Engineer, Ailevate

Ankura Senior Software Engineer, Ailevate in North Carolina

Ankura is a team of excellence founded on innovation and growth.

Practice Overview:

AI is more than just chatbots—it is the foundation for autonomous intelligence, decision-making, and scalable automation. At Ailevate, an Ankura company, we are pioneering the next evolution of Agentic AI, where autonomous AI Agents leverage Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and real-time data processing to drive impact across industries.

Our platform is built for businesses that need scalable, privacy-first AI solutions to enhance efficiency, automation, and decision-making. Join us in shaping the future of AI beyond chatbots, where LLM-powered Agents solve complex problems, streamline workflows, and optimize analytics-driven insights.

Role Overview:

The ideal candidate selected for this role will ideally be working US eastern time hours. As a Senior Software Engineer (Python), you will be a core contributor to our Agentic AI Platform, designing and scaling microservices that power LLM-driven AI Agents. You will collaborate with AI researchers, data engineers, and cloud infrastructure teams to deploy AI models, optimize inference, and eventually fine-tune LLMs for industry-specific applications. Your work will directly impact how AI-driven agents extract insights, process structured and unstructured data, and enhance real-time analytics.

This is a fully remote position with a highly talented AI-first team. The selected candidate will have ownership and impact to help shape a next-generation AI platform with real-world applications. Ankura offers competitive compensation and a strong work culture that values both innovation and work-life balance. Based on the selected candidates’ experience and qualifications, this role will be filled at either the Senior Associate or Director level within Ankura’s structure.

Responsibilities:

1. Develop, optimize, and scale backend services using Python and FastAPI.
2. Design and implement microservices for LLM-powered AI Agents, focusing on real-time processing, inference, and decision-making.
3. Integrate LLM APIs (OpenAI, Anthropic, vLLM, etc.) to power AI-driven insights and automation.
4. Enhance our Retrieval-Augmented Generation (RAG) pipeline, enabling AI Agents to retrieve, process, and synthesize knowledge.
5. Implement messaging and event-driven workflows using RabbitMQ.
6. Fine-tune and optimize LLMs using TensorFlow and PyTorch as the platform evolves.
7. Deploy and manage AI workloads on Kubernetes, ensuring scalability and high availability.
8. Collaborate with infrastructure and DevOps teams to streamline CI/CD pipelines and cloud-based deployments.
9. Write well-structured, maintainable, and testable code following best practices.
10. Mentor junior engineers and contribute to technical decision-making.

Requirements:

1. 5+ years of Python software development experience.
2. Strong experience with FastAPI (or Flask) for building scalable APIs.
3. Experience with LLMs, NLP, and AI-driven applications.
4. Experience integrating LLM APIs such as OpenAI, Anthropic, or vLLM.
5. Proficiency in microservices architecture and distributed systems.
6. Familiarity with frameworks such as TensorFlow and PyTorch and model optimization techniques.
7. Experience with SQL and NoSQL databases such as Elasticsearch or PostgreSQL.
8. Cloud experience with Azure, AWS, or GCP, along with CI/CD automation.
9. Proficiency in containerization with Docker and orchestration using Kubernetes.
10. Experience with event-driven architectures using RabbitMQ or similar message brokers.
11. Strong problem-solving and debugging skills with a focus on performance optimization.

Preferred Qualifications:

1. Experience fine-tuning LLMs and optimizing inference workloads.
2. Familiarity with Neo4j, graph databases, or knowledge graphs.
3. Experience contributing to open-source AI or LLM projects.
4. Understanding of vector databases such as Elasticsearch, Weaviate, or Pinecone.
5. Experience in healthcare, fintech, or other highly regulated industries.

Ankura is an Affirmative Action and Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or protected veteran status and will not be discriminated against based on disability.