Human-in-the-Loop E-Commerce AI Assistant

The challenge

Generic chat wrappers hallucinated stock, sizing, and returns policy — unacceptable for a retailer with real fulfilment costs. The assistant also had to hand off gracefully instead of stalling when it was unsure.

Approach

• Exposed catalogue, order, and policy systems as typed MCP tools so the model retrieves facts instead of recalling them, with every answer traced back to a source record.
• Combined a hybrid Elasticsearch retriever with a learned re-ranker and a collaborative-filtering recommender, so suggestions respect both semantic intent and real purchase behaviour.
• Modelled conversation, escalation, and feedback state in PostgreSQL through SQLAlchemy with Alembic migrations, keeping the human-review queue transactional.
• Added confidence-gated escalation: low-margin, high-risk, or low-certainty turns route to an agent console where the human answer becomes a labelled preference pair.
• Provisioned the whole stack with Terraform across environments, with canary routing so each prompt or retriever change ships to a traffic slice first.

Outcome

72% of conversations resolve without a human while hallucinated claims sit at 0.3% of audited turns. Assisted sessions convert 14% higher than unassisted, and agent handling time dropped by a third thanks to pre-drafted context.