Conversational Agent SME (Freelancer)

About Us

Deccan AI Experts is a pioneering AI company founded by IIT Bombay and IIM Ahmedabad alumni, with a strong founding team from IITs, NITs, and BITS. We specialize in high-quality human-curated data, AI-first operations, and advanced AI evaluation systems.

About the Role

We are seeking a Conversational Agent SME (Freelancer) to support advanced AI evaluation initiatives focused on conversational AI, dialogue systems, virtual assistants, multilingual interactions, and human-AI communication.

In this role, you will evaluate AI-generated conversations, dialogue strategies, conversational workflows, and multilingual interactions. Your expertise will help improve AI systems designed for customer support, voice assistants, chatbots, AI agents, and enterprise conversational experiences.

This position is ideal for professionals with experience in conversational AI, NLP, dialogue design, voice assistants, chatbot development, AI evaluation, or conversation design.

Responsibilities

• Create deliverables addressing real-world conversational AI and virtual assistant scenarios.
• Annotate and evaluate AI-generated conversations, dialogue flows, conversation strategies, and multilingual interactions.
• Assess AI outputs for conversational quality, contextual understanding, user experience, linguistic accuracy, and task completion.
• Design and evaluate dialogue flows for customer service, enterprise assistants, transactional workflows, and information retrieval.
• Review turn-taking behavior to ensure natural conversational pacing, context retention, interruption recovery, and user engagement.
• Perform intent and slot labeling, ensuring accurate intent classification, entity extraction, parameter mapping, and annotation consistency.
• Evaluate barge-in handling for voice and multimodal conversational agents, ensuring graceful interruption management and seamless conversation continuity.
• Conduct multilingual conversational quality assurance (QA) across English, Hindi, and other regional languages by assessing linguistic fluency, cultural appropriateness, translation quality, and task completion.
• Identify conversational breakdowns, hallucinations, context loss, repetitive responses, incorrect intent recognition, slot extraction errors, and policy violations.
• Provide structured feedback to improve AI performance in dialogue management, conversational reasoning, and user experience.
• Review peer-developed deliverables to maintain quality and consistency standards.

Requirements

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Computational Linguistics, Natural Language Processing, Data Science, Linguistics, Human-Computer Interaction, or a related field.
• 3+ years of hands-on experience in conversational AI, chatbot development, NLP, dialogue design, AI evaluation, conversation design, or voice assistant development.
• Designing dialogue flows for conversational agents and enterprise virtual assistants.
• Evaluating turn-taking behavior, conversation state management, context retention, and multi-turn dialogue quality.
• Performing intent classification and slot labeling for NLP training datasets and conversational AI systems.
• Assessing barge-in handling, interruption recovery, confirmation strategies, and voice interaction quality.
• Conducting multilingual conversational QA, including fluency, localization, grammar, cultural appropriateness, and conversational consistency.
• Developing annotation guidelines, evaluation rubrics, and quality standards for conversational AI systems.
• Familiarity with conversational AI platforms such as Google Dialogflow, Amazon Lex, Microsoft Copilot Studio, Rasa, IBM Watson Assistant, Kore.ai, Yellow.ai, LangGraph, or similar frameworks is preferred.
• Basic understanding of APIs, JSON, prompt engineering, function calling, and AI agent workflows is highly desirable.
• Strong analytical thinking, attention to detail, and problem-solving skills.
• Excellent written and verbal English communication skills.
• Proficiency in Hindi and/or one or more Indian regional languages is highly desirable for multilingual evaluation.
• Ability to critically evaluate AI-generated conversations for quality, usability, and production readiness.
• Ability to work independently in a remote, fast-paced environment.

Preferred Qualifications

• Experience working on enterprise chatbots, voice bots, AI assistants, contact center automation, or customer support AI platforms.
• Experience with LLM evaluation, RLHF, AI benchmarking, prompt engineering, or human-in-the-loop evaluation.
• Familiarity with speech recognition (ASR), text-to-speech (TTS), voice UX, conversational analytics, and omnichannel AI systems.
• Experience designing multilingual conversational datasets and evaluation benchmarks.
• Knowledge of Python, SQL, conversational analytics, or AI workflow orchestration is an advantage.
• Contributions to conversational AI research, open-source NLP projects, or dialogue datasets are highly desirable.

Why Join Us

• Competitive hourly pay: ₹2,500/hour
• Fully remote with flexible working hours.
• Opportunity to contribute to cutting-edge conversational AI and agentic AI initiatives.
• Exposure to advanced AI systems focused on dialogue intelligence, multilingual communication, and enterprise virtual assistants.
• Flexible project-based opportunities with global teams.
• Work on next-generation AI solutions supporting natural, reliable, and human-centric conversational experiences