20 KiB

Raw Blame History

OpenHands SDK: GitHub Actions vs n8n SSH Analysis

Date: 2025-12-02 Status: Comparison Analysis Complete Author: Claude Code

Executive Summary

This analysis compares two integration approaches for OpenHands SDK:

Current Approach (Phase 2): n8n → SSH → OpenHands CLI
New Approach (GitHub Actions): GitHub Actions → Python SDK → OpenHands

Recommendation: Hybrid approach combining the best of both worlds. Use n8n for orchestration and GitHub Actions for execution.

1. APPROACH COMPARISON

Current Approach: n8n → SSH → OpenHands CLI

Architecture:

Gitea Push → n8n Webhook → SSH Node → Wrapper Script → OpenHands CLI → Build

Components:

n8n workflow (ID: j1MmXaRhDjvkRSLa)
SSH authentication (n8n → localhost)
Shell wrapper script: /home/bam/openhands-sdk-wrapper-sh.sh
OpenHands CLI execution: /home/bam/.local/bin/openhands

Data Flow:

// Webhook receives Git push
// SSH node executes: sh /home/bam/openhands-sdk-wrapper-sh.sh "<task>"
// Output: {code, stdout, stderr}

Pros:

✅ Works with self-hosted Gitea (not requiring GitHub)
✅ Direct control via n8n UI
✅ Already operational and tested
✅ Good for self-contained environments
✅ Retry logic implemented (Phase 3 plan)

Cons:

❌ SSH overhead (latency, complexity)
❌ Data loss in SSH nodes (requires $node pattern)
❌ Shell script wrapper adds complexity
❌ Less standardized (custom wrapper)
❌ n8n-specific solution (not portable)

New Approach: GitHub Actions → Python SDK → OpenHands

Architecture:

Git Push → GitHub Actions → Python agent_script.py → OpenHands SDK → Build

Components:

GitHub Actions workflow (.github/workflows/openhands.yml)
Python agent script (agent_script.py)
OpenHands SDK (Python library)
GitHub Secrets for API keys

Data Flow:

# GitHub Actions runs Python script
llm = LLM(model="anthropic/claude-sonnet-4-5-20250929", api_key=os.getenv("LLM_API_KEY"))
agent = get_default_agent(llm=llm, cli_mode=True)
conversation = Conversation(agent=agent, workspace=cwd)
conversation.send_message(prompt)
conversation.run()

Pros:

✅ Direct SDK usage (no SSH overhead)
✅ Native GitHub Actions integration
✅ Better error handling and logging
✅ Standard Python patterns (easier to maintain)
✅ Built-in tools: TerminalTool, FileEditorTool, TaskTrackerTool
✅ Artifact uploads (logs preserved)
✅ Parallel execution support
✅ Marketplace-ready solution
✅ Better developer experience

Cons:

❌ Requires GitHub (not compatible with Gitea)
❌ More infrastructure (GitHub Actions runners)
❌ Learning curve for Python/SDK
❌ Potential cost for GitHub Actions minutes
❌ Less visual workflow control

2. INTEGRATION STRATEGY RECOMMENDATION

Recommended: Hybrid Approach

Structure:

Git Push → n8n Webhook → GitHub Actions Workflow → Python SDK → OpenHands
                    ↓
              (Orchestration)

Why Hybrid?

Best of Both Worlds:
- n8n: Orchestration, visual workflow, retry logic, Gitea integration
- GitHub Actions: Direct SDK execution, better tooling, standardized approach
Migration-Friendly:
- Keep existing n8n infrastructure
- Gradually migrate to GitHub Actions
- Can run both in parallel during transition
Practical for Your Setup:
- Gitea hooks into n8n (already working)
- n8n triggers GitHub Actions workflow
- GitHub Actions executes OpenHands SDK
- Returns results back to n8n

Architecture Diagram

┌──────────┐      ┌──────────┐      ┌────────────────┐      ┌──────────┐
│   Gitea  │─────▶│    n8n   │─────▶│ GitHub Actions │─────▶│ OpenHands│
│  (Push)  │      │ (Webhook)│      │   (Workflow)   │      │   SDK    │
└──────────┘      └──────────┘      └────────────────┘      └──────────┘
                      │                      │                      │
                      │              ┌───────▼───────┐              │
                      │              │ Python Script │              │
                      │              └───────┬───────┘              │
                      │                      │                      │
                      └──────────────────────┴──────────────────────┘
                                       (Results Flow)

3. IMPLEMENTATION PLAN

Phase A: GitHub Actions Foundation (1 hour)

Create GitHub Actions Workflow
- Location: .github/workflows/openhands-build.yml
- Purpose: Execute build/test via Python SDK
- Triggered by: n8n HTTP request
Create Python Agent Script
- Location: agent_script.py (in repo)
- Purpose: Build/test project workspace
- Uses: OpenHands SDK directly
Configure Repository Secrets
- LLM_API_KEY (OpenHands API key)
- Optional: GITHUB_TOKEN

Phase B: n8n GitHub Integration (1 hour)

Add GitHub Actions Trigger to n8n Workflow
- HTTP node to call GitHub Actions API
- Pass task details and project info
- Wait for GitHub Actions to complete
Add Result Processing
- Parse GitHub Actions response
- Update Gitea commit status
- Send notifications

Phase C: Testing & Migration (1-2 hours)

Test Integration
- Verify GitHub Actions runs
- Check OpenHands SDK execution
- Validate result flow back to n8n
Compare Performance
- Measure build time (SSH vs SDK)
- Compare error handling quality
- Check token usage/cost

Phase D: Optimization (1 hour)

Optimize Based on Results
- Keep SSH approach if faster/more reliable
- Or fully migrate to GitHub Actions
- Document findings

4. EXAMPLE IMPLEMENTATION

A. GitHub Actions Workflow

File: .github/workflows/openhands-build.yml

name: OpenHands Build

on:
  workflow_dispatch:
    inputs:
      task:
        description: 'Task to execute'
        required: true
        type: string
      workspace_path:
        description: 'Path to project workspace'
        required: true
        type: string
      retry_count:
        description: 'Retry attempt number'
        required: false
        type: number
        default: 0

permissions:
  contents: write
  pull-requests: write

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      TASK: ${{ github.event.inputs.task }}
      WORKSPACE_PATH: ${{ github.event.inputs.workspace_path }}
      RETRY_COUNT: ${{ github.event.inputs.retry_count || 0 }}
      LLM_MODEL: anthropic/claude-sonnet-4-5-20250929

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          path: ${{ env.WORKSPACE_PATH }}

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install uv
        uses: astral-sh/setup-uv@v6
        with:
          enable-cache: true

      - name: Install OpenHands SDK
        run: |
          uv pip install --system "openhands-sdk @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-sdk"
          uv pip install --system "openhands-tools @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-tools"          

      - name: Run OpenHands Build
        env:
          LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
          RETRY_COUNT: ${{ env.RETRY_COUNT }}
        run: |
          cd ${{ env.WORKSPACE_PATH }}
          uv run python ../agent_script.py          

      - name: Upload Build Artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: build-logs-${{ github.run_number }}
          path: |
            *.log
            output/            
          retention-days: 7

      - name: Upload Test Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results-${{ github.run_number }}
          path: |
            test-results/
            coverage/            
          retention-days: 30

B. Python Agent Script

File: agent_script.py

#!/usr/bin/env python3
"""
OpenHands Build Agent

Executes build/test tasks using OpenHands SDK.
Designed for GitHub Actions integration with n8n orchestration.

Usage:
  python agent_script.py

Environment Variables:
  TASK: Task description to execute
  WORKSPACE_PATH: Path to project workspace
  RETRY_COUNT: Current retry attempt number
  LLM_API_KEY: API key for LLM (required)
  LLM_MODEL: Model to use (default: anthropic/claude-sonnet-4-5-20250929)
"""

import os
import json
import sys
from datetime import datetime
from pathlib import Path

from openhands.sdk import LLM, Conversation, get_logger
from openhands.tools.preset.default import get_default_agent


logger = get_logger(__name__)


def create_build_task(workspace_path, task, retry_count):
    """Create enhanced build task with context."""

    # Check for previous errors
    error_file = Path(workspace_path) / "build-errors.json"
    previous_errors = ""
    if error_file.exists():
        try:
            with open(error_file) as f:
                errors = json.load(f)
                previous_errors = json.dumps(errors, indent=2)
        except Exception as e:
            logger.warning(f"Could not load previous errors: {e}")

    # Build enhanced prompt
    build_prompt = f"""
Build and test the project at: {workspace_path}

TASK: {task}

CURRENT ATTEMPT: {retry_count + 1}

REQUIREMENTS:
1. Install dependencies (npm install / pip install / etc.)
2. Run build process
3. Execute tests
4. Generate build report

ERROR HANDLING:
- Capture all build errors (stdout, stderr, exit codes)
- Save errors to: {workspace_path}/build-errors.json
- Report success/failure clearly

OUTPUT FORMAT:
{{
  "success": true/false,
  "exit_code": 0/1,
  "errors": ["list of errors"],
  "build_time": "seconds",
  "artifacts": ["list of generated files"]
}}

FOCUS AREAS:
- Dependency installation issues
- Build script failures
- Test failures
- Configuration problems

Return ONLY the JSON output, nothing else.
"""

    # Add previous errors if this is a retry
    if previous_errors and retry_count > 0:
        build_prompt += f"\n\nPREVIOUS BUILD ERRORS (attempt #{retry_count}):\n{previous_errors}\n\nPlease fix these specific issues."

    return build_prompt


def save_build_report(workspace_path, result):
    """Save build report to file."""

    report = {
        "timestamp": datetime.now().isoformat(),
        "workspace": workspace_path,
        "result": result,
    }

    report_path = Path(workspace_path) / "build-report.json"
    with open(report_path, "w") as f:
        json.dump(report, f, indent=2)

    logger.info(f"Build report saved to: {report_path}")
    return report_path


def main():
    """Execute build task with OpenHands SDK."""

    # Get configuration from environment
    task = os.getenv("TASK", "")
    workspace_path = os.getenv("WORKSPACE_PATH", os.getcwd())
    retry_count = int(os.getenv("RETRY_COUNT", "0"))
    api_key = os.getenv("LLM_API_KEY")
    model = os.getenv("LLM_MODEL", "anthropic/claude-sonnet-4-5-20250929")

    # Validate inputs
    if not task:
        logger.error("TASK environment variable is required")
        sys.exit(1)

    if not api_key:
        logger.error("LLM_API_KEY environment variable is required")
        sys.exit(1)

    if not os.path.exists(workspace_path):
        logger.error(f"Workspace path does not exist: {workspace_path}")
        sys.exit(1)

    logger.info(f"Starting build task (attempt #{retry_count + 1})")
    logger.info(f"Workspace: {workspace_path}")
    logger.info(f"Task: {task[:100]}...")

    try:
        # Configure LLM
        llm = LLM(
            model=model,
            api_key=api_key,
            usage_id="openhands-build",
            drop_params=True,
        )

        # Create agent
        agent = get_default_agent(
            llm=llm,
            cli_mode=True,
        )

        # Create conversation
        conversation = Conversation(
            agent=agent,
            workspace=workspace_path,
        )

        # Build task with context
        build_task = create_build_task(workspace_path, task, retry_count)

        # Execute task
        conversation.send_message(build_task)
        conversation.run()

        # Load build report
        report_path = Path(workspace_path) / "build-report.json"
        if report_path.exists():
            with open(report_path) as f:
                result = json.load(f)
                print(json.dumps(result, indent=2))
                sys.exit(0 if result.get("success") else 1)
        else:
            logger.error("Build report not found")
            sys.exit(1)

    except Exception as e:
        logger.error(f"Build failed with exception: {e}")
        # Save error report
        error_report = {
            "success": False,
            "error": str(e),
            "timestamp": datetime.now().isoformat(),
        }
        error_path = Path(workspace_path) / "build-errors.json"
        with open(error_path, "w") as f:
            json.dump(error_report, f, indent=2)
        sys.exit(1)


if __name__ == "__main__":
    main()

C. n8n Integration Node

New HTTP Node to Add to Workflow:

// Node: "Trigger GitHub Actions Build"

// URL
const githubRepo = "your-github-username/your-repo";
const url = `https://api.github.com/repos/${githubRepo}/actions/workflows/openhands-build.yml/dispatches`;

// Headers
const headers = {
  "Authorization": "Bearer " + $node["Get GITHUB_TOKEN"].json.token,
  "Accept": "application/vnd.github+json",
  "Content-Type": "application/json"
};

// Body
const body = {
  "ref": "main",
  "inputs": {
    "task": `Build and test project: ${$node["Extract Repo Info"].json.repo_name}`,
    "workspace_path": `/workspace/${$node["Extract Repo Info"].json.repo_name}`,
    "retry_count": $workflow.staticData.retry_count || 0
  }
};

return {
  url: url,
  method: "POST",
  headers: headers,
  body: JSON.stringify(body)
};

D. n8n Result Processing Node

// Node: "Process GitHub Actions Result"

// Get GitHub Actions response
const ghResponse = $json;

// Wait for GitHub Actions to complete
const runId = ghResponse.data.id;
const repo = "your-github-username/your-repo";

// Poll for completion
const checkUrl = `https://api.github.com/repos/${repo}/actions/runs/${runId}`;
const headers = {
  "Authorization": "Bearer " + $node["Get GITHUB_TOKEN"].json.token
};

// Make HTTP request to check status
return {
  url: checkUrl,
  headers: headers
};

5. MIGRATION PATH

Option 1: Gradual Migration (Recommended)

Timeline: 2-3 weeks

Week 1: Setup GitHub Actions

Create GitHub Actions workflow
Create Python agent script
Test in isolation

Week 2: Dual Execution

Run both n8n→SSH and n8n→GitHub Actions in parallel
Compare performance and reliability
Document differences

Week 3: Choose & Migrate

Analyze results from dual execution
Keep best approach (or hybrid)
Remove old approach
Complete Phase 3 with chosen method

Option 2: Full Migration to GitHub Actions

Timeline: 1 week

Day 1-2: GitHub Actions Setup

Create workflow and agent script
Configure secrets

Day 3-4: n8n Integration

Add HTTP trigger node
Add result processing

Day 5-7: Testing & Optimization

End-to-end testing
Performance tuning
Documentation

Option 3: Keep Current (n8n→SSH)

Timeline: Minimal

Just complete Phase 3 as planned

Already working
Familiar infrastructure
Known patterns

6. DETAILED COMPARISON

Aspect	n8n→SSH	GitHub Actions	Hybrid
Setup Time	~1 hour (done)	~2 hours	~3 hours
Complexity	Medium (SSH)	Low (SDK)	Medium-High
Performance	Slower (SSH overhead)	Faster (direct SDK)	Medium
Error Handling	Basic	Advanced	Advanced
Visual Control	Excellent (n8n UI)	None	Good
Portability	n8n-specific	GitHub standard	Partial
Learning Curve	Low (shell)	Medium (Python)	High
Maintenance	High (wrapper)	Low (SDK)	Medium
Developer Experience	Medium	Excellent	Good
Integration Effort	High (custom)	Low (standard)	Medium
Cost	No extra	GitHub Actions minutes	GitHub Actions minutes
Reliability	Good (tested)	Excellent (mature)	Good

7. RECOMMENDATION SUMMARY

For Your Current Setup:

Use: Hybrid Approach (n8n + GitHub Actions)

Why:

Leverages Existing Infrastructure: Keep n8n working, add GitHub Actions
Better Error Handling: GitHub Actions SDK provides superior error reporting
Standard Practices: GitHub Actions is industry-standard
Migration-Friendly: Can switch back if needed
Future-Proof: GitHub Actions widely adopted

Action Plan:

Create GitHub Actions workflow + Python script (1 hour)
Add GitHub Actions trigger to n8n workflow (30 min)
Test end-to-end integration (1 hour)
Compare performance with current approach (30 min)
Choose final approach based on results

Files to Create:

.github/workflows/openhands-build.yml
agent_script.py
Update n8n workflow (add 2 nodes)

If Switching to GitHub-Only:

Use: GitHub Actions (no n8n)

Why:

Simplest architecture
Best performance
Industry standard
Better tooling

Action Plan:

Create GitHub Actions workflow
Create Python agent script
Use GitHub webhooks instead of Gitea
Migrate from Gitea to GitHub (major effort)

8. PERFORMANCE METRICS TO TRACK

During Testing:

Build Time
- n8n→SSH: _____ seconds
- n8n→GitHub Actions: _____ seconds
- Difference: _____% faster/slower
Error Handling
- n8n→SSH: _____% accuracy
- n8n→GitHub Actions: _____% accuracy
Token Usage
- n8n→SSH: _____ tokens per build
- n8n→GitHub Actions: _____ tokens per build
Reliability
- n8n→SSH: _____% success rate
- n8n→GitHub Actions: _____ % success rate
Developer Experience
- n8n→SSH: Rating (1-5): _____
- n8n→GitHub Actions: Rating (1-5): _____

9. CONCLUSION

Current State: Phase 2 working with n8n→SSH

New Opportunity: GitHub Actions SDK integration

Recommendation: Hybrid approach (n8n + GitHub Actions)

Next Steps:

Implement GitHub Actions workflow (1 hour)
Integrate with n8n (30 min)
Test & compare (1 hour)
Choose final approach
Complete Phase 3

Benefits of GitHub Actions Approach:

Better error handling
Standard tooling
Easier maintenance
Future scalability

Risks of Migration:

Added complexity (GitHub + n8n)
Learning curve for Python/SDK
Potential GitHub Actions costs

Recommendation Strength: High Confidence Level: 85%

10. REFERENCES

Files Referenced:

/home/bam/claude/mvp-factory/phase3.md - Current Phase 3 plan
/home/bam/claude/mvp-factory/n8n-api.md - n8n integration details
/home/bam/claude/mvp-factory/openhands-subagents-doc.md - OpenHands optimization

External Resources:

GitHub: OpenHands SDK Examples - https://github.com/OpenHands/software-agent-sdk
OpenHands Documentation - https://docs.openhands.dev/sdk
GitHub Actions Documentation - https://docs.github.com/en/actions

Current Working Setup:

n8n Workflow ID: j1MmXaRhDjvkRSLa
Webhook: https://n8n.oky.sh/webhook/openhands-fixed-test
SDK Wrapper: /home/bam/openhands-sdk-wrapper-sh.sh

Analysis Complete Ready for Implementation

Created: 2025-12-02 Status: Ready for review and implementation

20 KiB Raw Blame History