Enhance OpenAgents: Specstory & .tricoder Integration

by Admin 54 views
Enhance OpenAgents: Specstory & .tricoder Integration

Hey guys! Today, we're diving deep into enhancing OpenAgents with two killer features: Specstory support and .tricoder transcript storage. These additions will not only streamline your workflow but also boost collaboration and documentation. Let's get started!

Feature Request: Specstory Support & .tricoder Transcript Storage

Summary

This upgrade introduces two major enhancements for OpenAgents:

  1. Specstory Integration: Native support for specification stories/user stories.
  2. Transcript Storage: Implement a structured directory for chatlog/transcript persistence using .tricoder.

Research Findings

Current State - Specstory Support ❌

Currently, no native specstory support exists within the OpenAgents codebase. There are no direct mentions of "specstory", "spec story", or structured user story management. While basic todo tracking is available via ProjectTodo, it lacks story-specific workflows. To address this gap, integrating Specstory support is crucial for managing project requirements effectively.

The absence of native specstory support hinders the ability to systematically define, track, and manage user stories within OpenAgents. User stories are essential for capturing user requirements and guiding the development process. Without this feature, teams may struggle to maintain a clear understanding of project goals and priorities, leading to inefficiencies and potential miscommunication. By adding native support for specstories, OpenAgents can empower teams to create, organize, and track user stories throughout the project lifecycle, ensuring that everyone is aligned on the project's objectives and delivering value to users. Furthermore, this integration can enable the creation of custom workflows tailored to specific story types and statuses, providing a flexible and adaptable approach to project management.

Current State - Transcript Storage ✅ (Strong Foundation)

Good news! A robust JSONL infrastructure is already in place via history.rs. This includes:

  • Codex integration with JSONL parsing from ~/.codex/sessions/.
  • Tinyvex SQLite database for real-time sync.
  • Mobile app AsyncStorage with history caching.
  • WebSocket streaming for live collaboration.

With a solid foundation in place, enhancing transcript storage is a natural progression for OpenAgents. The existing JSONL infrastructure provides a reliable and efficient way to store and retrieve chatlogs and transcripts, while the Tinyvex SQLite database ensures real-time synchronization across devices. This robust backend enables seamless collaboration and ensures that users always have access to the latest conversation history. By building upon this existing infrastructure, the addition of .tricoder transcript storage can be implemented with minimal disruption and maximum efficiency, further enhancing the usability and value of OpenAgents.

Implementation Plan

Phase 1: Specstory Support

1. Schema Extension

We'll create a spec.schema.json file to define the structure of a user story:

{
  "title": "User Story",
  "description": "As a user, I want...",
  "type": "epic|story|task|bug",
  "status": "draft|review|approved|implemented|done",
  "priority": "low|medium|high|critical",
  "acceptanceCriteria": [],
  "dependencies": [],
  "projectId": "reference"
}

2. Rust Backend Changes

  • File: crates/oa-bridge/src/projects.rs

    • Extend the Project struct with specifications: Vec<SpecStory>. Implementing this will allow each project to hold multiple specstories that can be referenced and displayed to end-users.
    • Add a SpecStory struct to manage the full story lifecycle.

  • File: crates/oa-bridge/src/controls.rs

    • Add WebSocket controls: specs.list, specs.save, specs.update_status to allow us to control our specstories.

3. Mobile App Integration

  • File: expo/lib/projects-store.ts
    • Extend TypeScript interfaces for story management.
    • Add story CRUD operations to provide the ability to easily manipulate the data.

Phase 2: .tricoder Transcript Storage

1. Directory Structure

We'll organize transcripts within the .tricoder directory:

.tricoder/
├── transcripts/
│   ├── YYYY/MM/DD/
│   │   └── transcript-YYYY-MM-DDThh-mm-ss-<UUID>.jsonl
├── metadata/
│   └── sessions.json
└── config.json

2. Backend Extensions

  • File: crates/oa-bridge/src/history.rs

    • Extend scan_history() to include .tricoder/ directories for searching.
    • Add scan_tricoder_history() function for ease of access to the directories.

  • File: crates/oa-bridge/src/controls.rs

    • Add controls: tricoder.list, tricoder.get, tricoder.save for transcript management.

3. Mobile App Updates

  • File: expo/lib/threads-store.ts
    • Extend HistoryItem type for .tricoder sources.
    • Add .tricoder-specific loading functions.

Implementation Advantages

Specstory Integration

  • Leverages existing Projects system: Stories are linked directly to projects, ensuring a cohesive and organized structure.
  • Reuses Todo infrastructure: Extends current todo tracking capabilities, streamlining development tasks.
  • Skills integration: Can create a "spec-story" skill for automated workflows, enhancing efficiency and collaboration.
  • WebSocket real-time updates: Story status changes sync instantly, keeping team members informed and aligned.

The specstory integration offers a multitude of advantages, including improved requirement tracking and enhanced collaboration. By leveraging the existing Projects system, stories are seamlessly linked to projects, ensuring a cohesive and organized structure. This integration allows for the creation of a "spec-story" skill, enabling automated workflows and enhancing efficiency. Real-time updates via WebSocket keep team members informed of story status changes, fostering collaboration and ensuring that everyone is on the same page. Furthermore, the reusability of the Todo infrastructure streamlines development tasks, reducing redundancy and improving overall productivity. With these combined benefits, specstory integration promises to significantly enhance project management capabilities within OpenAgents.

.tricoder Directory

  • Builds on existing JSONL infrastructure: No new parsing is needed, saving time and resources.
  • Natural extension: Fits seamlessly alongside .openagents and .codex directories, maintaining a consistent file structure.
  • Reuses history scanning: Leverages existing metadata extraction processes, reducing development efforts.
  • Mobile UI components: Can reuse existing transcript viewers, accelerating development and ensuring a consistent user experience.

The .tricoder directory offers several key advantages for transcript storage within OpenAgents. By building upon the existing JSONL infrastructure, no new parsing is required, saving time and resources. Its seamless integration alongside .openagents and .codex directories maintains a consistent file structure, enhancing organization and usability. Furthermore, the reusability of history scanning and mobile UI components accelerates development and ensures a familiar user experience. With these combined benefits, the .tricoder directory provides a robust and efficient solution for transcript storage, further enhancing the functionality and usability of OpenAgents.

Files to Modify

High Priority:

  • crates/oa-bridge/src/history.rs: JSONL scanning extension to find all transcripts.
  • crates/oa-bridge/src/projects.rs: Spec story schema integration, essential for setting up the new format.
  • crates/oa-bridge/src/controls.rs: WebSocket controls for both features, the lifeline of communication.
  • expo/lib/projects-store.ts: TypeScript interfaces and CRUD, making sure data is properly managed and displayed.

Medium Priority:

  • crates/oa-bridge/src/main.rs: Directory initialization, to verify existence of the folders we are creating.
  • expo/lib/threads-store.ts: Transcript storage integration, to display historical context.
  • expo/components/jsonl/TodoListCard.tsx: Story management UI, to have an appealing way to interact with the data.

Impact Assessment

Benefits:

  • 🎯 Better requirement tracking: Structured story management for clear project goals.
  • 📝 Improved documentation: Persistent transcript storage for reference, crucial for legal data.
  • 🔄 Real-time collaboration: Story status updates and transcript sharing for effective teamwork.
  • 🏗️ Scalable architecture: Builds on existing proven infrastructure, meaning easy and fast future enhancements.

Risks:

  • Low technical risk: Extends existing, well-tested systems, so there are no surprises.
  • Schema migration: Need to handle project schema updates gracefully, to prevent data loss.
  • Namespace management: Ensure .tricoder doesn't conflict with existing storage, which means we need to keep the structure organized.

Next Steps

  1. Schema Design: Finalize JSON schemas for spec stories and .tricoder metadata to set up the foundation of our data.
  2. Backend Implementation: Rust components for both features will provide the structure for the system.
  3. Mobile Integration: TypeScript interfaces and UI components to provide the access to the data.
  4. Testing: Comprehensive integration tests for new workflows will ensure the stability of the code.
  5. Documentation: Update project docs and migration guides so users can understand how to use the system.

Acceptance Criteria

  • [ ] Spec stories can be created, updated, and tracked via WebSocket controls, which are essential for remote management.
  • [ ] Stories are properly linked to projects with status workflows to maintain consistent data.
  • [ ] .tricoder directory stores transcripts in compatible JSONL format so we can read the transcripts.
  • [ ] Historical transcripts from .tricoder appear in the mobile app history, because context is important.
  • [ ] Real-time updates work for both story status changes and new transcripts to promote real-time collaboration.
  • [ ] Existing functionality remains unaltered (backward compatibility) to avoid disruption of existing systems.

Dependencies

  • Blocked by: None (builds on existing infrastructure)
  • Blocks: Enhanced project management workflows will be enabled by this.
  • Related: Project collaboration features, mobile app improvements will follow this improvement.

There you have it, guys! By implementing these enhancements, we're not just adding features; we're building a more robust, collaborative, and efficient environment for OpenAgents. Let's get to work!