Innovation 8: Conversation Memory via Summary

Overview

The Context Window Problem

Traditional approaches store full conversation history:

• Context window fills up quickly
• Old messages get dropped
• Important facts lost
• Character "forgets" key information

Summary-Based Memory

Our approach maintains compressed memory:

• Key facts preserved indefinitely
• Relationship evolution tracked
• Important moments remembered
• Efficient use of context window

Implementation

Summary Generation (from servicesChatSummary.py)

async def generate_chat_summary(chat_id, messages):
    """Generate condensed summary of conversation"""
    
    # Get recent messages (last 20-30 exchanges)
    recent_messages = messages[-30:]
    
    # Format conversation history
    conversation_text = format_conversation(recent_messages)
    
    # Generate summary with GPT-4
    summary_prompt = f"""
Analyze this conversation and create a brief summary capturing:
1. Key facts shared (names, locations, events, preferences)
2. Relationship dynamics and emotional tone
3. Important topics discussed
4. Any decisions or plans made
5. Overall conversation trajectory

Conversation:
{conversation_text}

Summary (2-3 paragraphs max):"""

    summary = await call_llm(
        prompt=summary_prompt,
        model="gpt-4",
        temperature=0.3,
        max_tokens=300
    )
    
    return summary

Summary Usage in Prompts

# Important: when {character_name} reasons for answers it considers: 
# '''{chat_summary}'''

# This gives character access to full conversation context
# without including every message in the prompt

When to Update Summary

def should_update_summary(chat):
    """Determine if summary needs updating"""
    
    # Update every 10 messages
    if chat.message_count % 10 == 0:
        return True
    
    # Update if significant time has passed
    if time.time() - chat.last_summary_time > 86400:  # 24 hours
        return True
    
    # Update if conversation topic shifted
    if chat.topic != chat.previous_topic:
        return True
    
    return False

Example Summary

After 30 Messages

Chat Summary:
Alex (28, marketing professional) and Sarah (friend/mentor) have been discussing 
Alex's career transition. Alex revealed feeling stuck in current role despite recent 
promotion, wondering about switching to product management. Sarah shared her own 
career pivot story from 3 years ago. They discovered mutual love of rooftop bars 
and made plans to meet up this weekend. Alex mentioned struggling with work-life 
balance and imposter syndrome. Sarah offered to introduce Alex to PM contacts. 
Conversation tone: supportive, authentic, occasionally playful. Alex seems to trust 
Sarah's advice and values the friendship.

How It's Used

User (Message 35): "So about that PM role..."

Prompt includes:
# Important: when Sarah reasons for answers it considers:
# '''[Full summary above]'''

Response: "oh yeah! so I actually reached out to my friend Maya who's a senior PM 
at that startup I mentioned. she said she'd be happy to chat with you about what 
the role is really like day-to-day. want me to intro you two?"

# Character remembers: previous PM discussion, offer to make intros, Alex's interest

Replication Guide

Step 1: Set Up Summary Storage

class Chat(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    summary = db.Column(db.Text, default="")
    summary_updated_at = db.Column(db.Float, default=0)
    message_count = db.Column(db.Integer, default=0)

Step 2: Generate Summary Periodically

async def update_chat_summary_if_needed(chat_id):
    chat = get_chat(chat_id)
    
    if should_update_summary(chat):
        messages = get_all_messages(chat_id)
        summary = await generate_chat_summary(chat_id, messages)
        
        chat.summary = summary
        chat.summary_updated_at = time.time()
        db.session.commit()
    
    return chat.summary

Step 3: Inject into Prompts

def build_prompt_with_memory(character, host, chat, last_message):
    # Get summary
    summary = chat.summary or "Beginning of conversation"
    
    prompt = f"""
# Important: when {character.name} reasons for answers it considers:
# '''{summary}'''

[Rest of prompt...]

[{host.name}]: {last_message}
[{character.name}]:"""
    
    return prompt

Performance Metrics