Project Overview
ReturnDeskEnv is a sophisticated OpenEnv-compatible reinforcement learning environment designed to simulate the complexities of a real-world e-commerce customer support desk. This project models high-stakes decision workflows including returns, refunds, fraud detection, and corporate chargeback disputes.
Key Features & Agent Capabilities
Procedural Content Generation: Every episode is unique, randomizing product data, prices, exchange rates, and customer histories to ensure agent generalization rather than rote memorization.
Seven Difficulty Tiers: Challenges range from simple damaged-item refunds to “Extreme” corporate chargeback disputes requiring 4-way data reconciliation across different currencies.
Fraud Detection Logic: Features a high-stakes penalty system where agents must identify suspicious signals (e.g., refund velocity, address mismatches) and escalate instead of refunding.
Multi-Turn Dialogue: Supports interactive information gathering, allowing agents to “ask” customers follow-up questions to uncover hidden variables.
Sophisticated Grading Rubric: A 9-component deterministic grader evaluates agents on policy compliance, cost efficiency, reply quality (via Jaccard similarity), and evidence coverage.
Curriculum Learning Support: Includes a built-in curriculum tracker that automatically scales task difficulty based on the agent’s rolling performance mean.
Technical Impact
In the real world, customer operations isn’t just about being “nice”—it’s about making cost-effective, policy-compliant decisions while mitigating fraud. ReturnDeskEnv provides a safe “digital twin” sandbox for companies to develop and test LLM-based agents on 12 distinct actions before deploying them to live customer environments.
Technical Stack & Compliance
Core: Python, FastAPI, Pydantic v2 (Typed Data Models).
AI/ML: RLlib/OpenEnv compliant, baseline inference using Qwen 2.5 & Llama 3 via Hugging Face.
DevOps: Dockerized deployment, automated pytest suite (36 tests), and Hugging Face Spaces integration.
Standards: Fully compliant with the OpenEnv v0.3.0 manifest for standardized RL benchmarking.