ElevenLabs

ElevenLabs Agents connect to AutoMem through the same remote MCP sidecar used by other cloud platforms (ChatGPT, Claude.ai). ElevenLabs supports custom HTTP headers, so it can use the more secure header-based authentication (as can Claude.ai (Web); ChatGPT is URL-auth only).

Note

ElevenLabs has a 30-second idle timeout. The sidecar’s SSE transport sends heartbeats every 20 seconds to keep the connection alive. Streamable HTTP is also supported.

---

Architecture

The MCP sidecar bridges ElevenLabs voice agents to AutoMem’s HTTP API:

graph TB
    subgraph "Remote MCP (HTTP/SSE Transport)"
        EL["ElevenLabs Agents Platform"]
        SIDECAR["MCP Sidecar — POST /mcp / GET /mcp/sse"]
        AUTOMEM["AutoMem API :8001"]

        EL -->|"HTTPS + Bearer token"| SIDECAR
        SIDECAR -->|"HTTP"| AUTOMEM
    end

---

Prerequisites

1. A deployed AutoMem service
2. The MCP sidecar deployed and accessible over HTTPS
3. An ElevenLabs account with Agents access

---

Deploy the MCP Sidecar

The sidecar is included in the AutoMem Railway one-click template. See Claude.ai (Web) for full deployment instructions.

Required environment variables:

Variable	Purpose
AUTOMEM_API_URL	AutoMem service URL
AUTOMEM_API_TOKEN	Default token for API authentication
PORT	Listen port (default: 8080)

ElevenLabs-specific note: The SSE transport heartbeat (every 20 seconds) is specifically designed to keep connections alive past ElevenLabs’ 30-second idle timeout. Ensure the sidecar deployment does not have its own idle timeout shorter than 30 seconds.

---

Connect ElevenLabs Agent

Configure the MCP server in your ElevenLabs Agent settings:

Option 1: Header-Based Authentication (Recommended)

ElevenLabs supports custom HTTP headers, so you can use the more secure Authorization header:

• Server URL: https://your-mcp-bridge.up.railway.app/mcp/sse
• Custom Header Name: Authorization
• Custom Header Value: Bearer YOUR_AUTOMEM_TOKEN

Or using Streamable HTTP:

• Server URL: https://your-mcp-bridge.up.railway.app/mcp
• Custom Header Name: Authorization
• Custom Header Value: Bearer YOUR_AUTOMEM_TOKEN

Option 2: URL-Based Authentication

SSE transport:

https://your-mcp-bridge.up.railway.app/mcp/sse?api_token=YOUR_AUTOMEM_TOKEN

Streamable HTTP:

https://your-mcp-bridge.up.railway.app/mcp?api_token=YOUR_AUTOMEM_TOKEN

Tip

Use Option 1 (header-based) when possible. URL-based tokens appear in server logs, reverse proxy logs, and potentially in ElevenLabs’ own analytics.

---

Voice Agent Memory Patterns

ElevenLabs voice agents benefit from memory in several ways:

User preference recall — Remember communication preferences across sessions:

recall_memory(
  tags: ["preference", "elevenlabs"],
  limit: 5
)

Context continuity — Resume conversations with context from previous sessions:

recall_memory(
  query: "last conversation with this user",
  time_query: "last 7 days"
)

Knowledge persistence — Store facts the user shares:

store_memory(
  content: "User's name is Alex. Prefers brief responses.",
  type: "Preference",
  importance: 0.9,
  tags: ["elevenlabs", "user-preferences"]
)

---

Available Memory Tools

All six AutoMem tools are accessible to ElevenLabs agents:

Tool	Description
store_memory	Store voice-captured information
recall_memory	Find relevant memories for current conversation
associate_memories	Create relationships between related memories
update_memory	Update existing memory content or metadata
delete_memory	Remove a specific memory
check_database_health	Verify AutoMem service is running

---

Connection Stability

ElevenLabs has a 30-second idle timeout on SSE connections. The sidecar handles this automatically:

• SSE heartbeats sent every 20 seconds (within the 30s timeout window)
• Heartbeat format: : ping\n\n (SSE comment, not a data event)
• Anti-buffering headers set: X-Accel-Buffering: no, Cache-Control: no-cache, no-transform

If you observe connection drops:

1. Check that no intermediate proxy is buffering the SSE stream
2. Verify the proxy passes through SSE headers
3. Consider using Streamable HTTP (/mcp) instead of SSE (/mcp/sse)

---

Verification

Test the connection through your ElevenLabs agent:

"Check the health of the AutoMem service"

Test memory:

"Remember that I prefer concise responses."
"What are my preferences?"

---

Troubleshooting

Agent cannot connect

1. Verify sidecar health: curl https://your-mcp-bridge.up.railway.app/health — this always returns 200 if the sidecar itself is up; read the upstream field to see whether AutoMem is reachable. Use /ready instead if you want a non-200 (503) when the upstream is unhealthy.
2. Check TLS certificate is valid
3. Ensure port 443 is reachable from ElevenLabs’ servers

401 Unauthorized

1. For header auth: verify the header name is exactly Authorization and value is Bearer YOUR_TOKEN
2. For URL auth: verify token is URL-encoded if it contains special characters
3. Test token: curl -H "Authorization: Bearer $TOKEN" https://your-automem-service/health — the sidecar’s own /health is unauthenticated and cannot validate a token. The sidecar returns 401 only when no token is presented at all; a token that is present but wrong is forwarded to AutoMem and rejected there.

Connection drops during agent conversation

The SSE heartbeat (every 20 seconds) should prevent ElevenLabs’ 30-second timeout. If drops persist:

• Switch to Streamable HTTP (/mcp) — lower latency, no idle timeout issues
• Check sidecar logs for session expiration (sessions expire after 1 hour of inactivity)

Memory service unreachable

The sidecar does not return 502. When AutoMem is unreachable the tool call comes back as an MCP tool error (isError: true) with the failure text, and a missing AUTOMEM_API_URL returns 500 AUTOMEM_API_URL not configured.

Verify AUTOMEM_API_URL in sidecar environment variables points to the correct AutoMem service. On Railway, use the internal hostname: http://memory-service.railway.internal:8001.

---

Related Platforms

Other cloud platforms using the same MCP sidecar:

• ChatGPT — URL auth only
• Claude.ai (Web) — header or URL auth