Human-in-the-Loop (HITL)

Human-in-the-Loop (HITL) is how Solo ensures you stay in control. Agents are powerful, but they always know when to involve you in decisions.

What is Human-in-the-Loop?

Human-in-the-Loop is:

  • Your safety net: Agents escalate when uncertain
  • Your control: You approve important actions
  • Your oversight: Review agent decisions
  • Your feedback: Teach agents what's right

HITL ensures agents are helpful, not harmful — they augment your work, not replace your judgment.

How HITL Works

1. Confidence Threshold

Agents have a confidence threshold (0-100%):

  • High confidence (above threshold): Agent acts automatically
  • Low confidence (below threshold): Agent escalates to you

Example:

  • Threshold: 70%
  • Agent confidence: 85% → Acts automatically
  • Agent confidence: 60% → Escalates to you

2. Intent Triggers

Some topics always require your input:

  • Legal matters: Contracts, disputes, legal questions
  • Billing disputes: Payment issues, refunds
  • Sensitive topics: As defined by you
  • High-value actions: Large purchases, major decisions

3. Approval Modes

You can set how agents handle actions:

Hold for Approval

  • Agent drafts action
  • Sends to you for review
  • You approve or modify
  • Action executes after approval

Send Unless Declined

  • Agent takes action
  • Notifies you
  • You can decline within time window
  • Action proceeds if not declined

4. Escalation Routes

Define who gets notified and when:

Level 1 Escalation

  • Notify: Manager role
  • Channel: In-app notification
  • Timeout: 30 minutes
  • If no response → Level 2

Level 2 Escalation

  • Notify: Owner
  • Channel: In-app + Email
  • Timeout: 2 hours
  • If no response → Hold action

Configuring HITL

Configure HITL in Agent Config → HITL:

Confidence Threshold

Set when agents should escalate:

  • Low (50%): Agents escalate frequently, more control
  • Medium (70%): Balanced approach (recommended)
  • High (90%): Agents act more autonomously

Intent Triggers

Define topics that always require escalation:

  • Legal: Legal matters, contracts, disputes
  • Billing: Payment issues, refunds, disputes
  • Sensitive: Topics you define
  • Custom: Add your own triggers

Approval Mode

Choose how actions are handled:

  • Hold for Approval: Review before action
  • Send Unless Declined: Act unless you decline

Escalation Routes

Set up escalation paths:

  1. Level 1: First escalation point
  2. Level 2: Second escalation point
  3. Level 3: Final escalation point

For each level:

  • Who: Role or specific person
  • Channel: In-app, email, SMS
  • Timeout: How long to wait
  • Next: What happens if no response

HITL Examples

Example 1: Email Reply

Scenario: Customer asks about refund policy

  1. Agent analyzes: Understands question, finds policy
  2. Confidence check: 75% (above 70% threshold)
  3. Intent check: Not a billing dispute (no trigger)
  4. Action: Drafts reply, sends for approval
  5. You review: Approve, modify, or decline
  6. Result: Reply sent (or modified and sent)

Example 2: Billing Dispute

Scenario: Customer disputes a charge

  1. Agent analyzes: Identifies billing dispute
  2. Intent trigger: Billing dispute → Always escalate
  3. Action: Creates task, notifies you
  4. You handle: Review dispute, take action
  5. Agent learns: Notes your resolution for future

Example 3: Low Confidence

Scenario: Unclear email request

  1. Agent analyzes: Can't determine intent clearly
  2. Confidence check: 45% (below 70% threshold)
  3. Action: Escalates to you with context
  4. You decide: Clarify intent, provide guidance
  5. Agent learns: Better understanding for next time

HITL Best Practices

Start Conservative

  • Begin with lower confidence threshold (60-70%)
  • Add more intent triggers
  • Use "Hold for Approval" mode
  • Gradually increase autonomy as you trust agents

Define Clear Triggers

  • List topics that always need your input
  • Be specific: "billing disputes" not "money"
  • Review and update triggers regularly
  • Remove triggers that aren't needed

Set Up Escalation Routes

  • Define clear escalation paths
  • Set appropriate timeouts
  • Use multiple channels (in-app + email)
  • Test escalation flows

Provide Feedback

  • Review escalated items promptly
  • Approve or correct agent actions
  • Explain corrections to help agents learn
  • Agents improve from your feedback

Monitor and Adjust

  • Review escalation frequency
  • Adjust confidence threshold if needed
  • Add/remove intent triggers
  • Refine approval modes

HITL vs. Full Automation

Human-in-the-Loop (Recommended)

  • ✅ You stay in control
  • ✅ Quality assurance
  • ✅ Learning opportunity
  • ✅ Brand safety
  • ⚠️ Requires your time

Full Automation

  • ✅ Faster response
  • ✅ No human time needed
  • ✅ 24/7 operation
  • ⚠️ Less control
  • ⚠️ Higher risk

Recommendation: Start with HITL, gradually increase automation as you trust agents.

Troubleshooting

Too many escalations?

  • Lower confidence threshold
  • Remove unnecessary intent triggers
  • Improve agent knowledge base
  • Review agent configuration

Not enough escalations?

  • Raise confidence threshold
  • Add more intent triggers
  • Use "Hold for Approval" mode
  • Review agent actions more closely

Escalations not reaching you?

  • Check notification settings
  • Verify escalation routes
  • Test escalation flow
  • Check email/spam filters

Next Steps

Version: 1.0
Last Updated: November 2025