mvp-factory-openhands/github-actions-vs-n8n-analysis.md

# OpenHands SDK: GitHub Actions vs n8n SSH Analysis

**Date:** 2025-12-02
**Status:** Comparison Analysis Complete
**Author:** Claude Code

---

## Executive Summary

This analysis compares two integration approaches for OpenHands SDK:

1. **Current Approach (Phase 2):** n8n → SSH → OpenHands CLI
2. **New Approach (GitHub Actions):** GitHub Actions → Python SDK → OpenHands

**Recommendation:** Hybrid approach combining the best of both worlds. Use **n8n for orchestration** and **GitHub Actions for execution**.

---

## 1. APPROACH COMPARISON

### Current Approach: n8n → SSH → OpenHands CLI

**Architecture:**
```
Gitea Push → n8n Webhook → SSH Node → Wrapper Script → OpenHands CLI → Build
```

**Components:**
- n8n workflow (ID: j1MmXaRhDjvkRSLa)
- SSH authentication (n8n → localhost)
- Shell wrapper script: `/home/bam/openhands-sdk-wrapper-sh.sh`
- OpenHands CLI execution: `/home/bam/.local/bin/openhands`

**Data Flow:**
```javascript
// Webhook receives Git push
// SSH node executes: sh /home/bam/openhands-sdk-wrapper-sh.sh "<task>"
// Output: {code, stdout, stderr}
```

**Pros:**
- ✅ Works with self-hosted Gitea (not requiring GitHub)
- ✅ Direct control via n8n UI
- ✅ Already operational and tested
- ✅ Good for self-contained environments
- ✅ Retry logic implemented (Phase 3 plan)

**Cons:**
- ❌ SSH overhead (latency, complexity)
- ❌ Data loss in SSH nodes (requires `$node` pattern)
- ❌ Shell script wrapper adds complexity
- ❌ Less standardized (custom wrapper)
- ❌ n8n-specific solution (not portable)

### New Approach: GitHub Actions → Python SDK → OpenHands

**Architecture:**
```
Git Push → GitHub Actions → Python agent_script.py → OpenHands SDK → Build
```

**Components:**
- GitHub Actions workflow (.github/workflows/openhands.yml)
- Python agent script (agent_script.py)
- OpenHands SDK (Python library)
- GitHub Secrets for API keys

**Data Flow:**
```python
# GitHub Actions runs Python script
llm = LLM(model="anthropic/claude-sonnet-4-5-20250929", api_key=os.getenv("LLM_API_KEY"))
agent = get_default_agent(llm=llm, cli_mode=True)
conversation = Conversation(agent=agent, workspace=cwd)
conversation.send_message(prompt)
conversation.run()
```

**Pros:**
- ✅ Direct SDK usage (no SSH overhead)
- ✅ Native GitHub Actions integration
- ✅ Better error handling and logging
- ✅ Standard Python patterns (easier to maintain)
- ✅ Built-in tools: TerminalTool, FileEditorTool, TaskTrackerTool
- ✅ Artifact uploads (logs preserved)
- ✅ Parallel execution support
- ✅ Marketplace-ready solution
- ✅ Better developer experience

**Cons:**
- ❌ Requires GitHub (not compatible with Gitea)
- ❌ More infrastructure (GitHub Actions runners)
- ❌ Learning curve for Python/SDK
- ❌ Potential cost for GitHub Actions minutes
- ❌ Less visual workflow control

---

## 2. INTEGRATION STRATEGY RECOMMENDATION

### Recommended: Hybrid Approach

**Structure:**
```
Git Push → n8n Webhook → GitHub Actions Workflow → Python SDK → OpenHands
                    ↓
              (Orchestration)
```

**Why Hybrid?**

1. **Best of Both Worlds:**
   - n8n: Orchestration, visual workflow, retry logic, Gitea integration
   - GitHub Actions: Direct SDK execution, better tooling, standardized approach

2. **Migration-Friendly:**
   - Keep existing n8n infrastructure
   - Gradually migrate to GitHub Actions
   - Can run both in parallel during transition

3. **Practical for Your Setup:**
   - Gitea hooks into n8n (already working)
   - n8n triggers GitHub Actions workflow
   - GitHub Actions executes OpenHands SDK
   - Returns results back to n8n

### Architecture Diagram

```
┌──────────┐      ┌──────────┐      ┌────────────────┐      ┌──────────┐
│   Gitea  │─────▶│    n8n   │─────▶│ GitHub Actions │─────▶│ OpenHands│
│  (Push)  │      │ (Webhook)│      │   (Workflow)   │      │   SDK    │
└──────────┘      └──────────┘      └────────────────┘      └──────────┘
                      │                      │                      │
                      │              ┌───────▼───────┐              │
                      │              │ Python Script │              │
                      │              └───────┬───────┘              │
                      │                      │                      │
                      └──────────────────────┴──────────────────────┘
                                       (Results Flow)
```

---

## 3. IMPLEMENTATION PLAN

### Phase A: GitHub Actions Foundation (1 hour)

1. **Create GitHub Actions Workflow**
   - Location: `.github/workflows/openhands-build.yml`
   - Purpose: Execute build/test via Python SDK
   - Triggered by: n8n HTTP request

2. **Create Python Agent Script**
   - Location: `agent_script.py` (in repo)
   - Purpose: Build/test project workspace
   - Uses: OpenHands SDK directly

3. **Configure Repository Secrets**
   - LLM_API_KEY (OpenHands API key)
   - Optional: GITHUB_TOKEN

### Phase B: n8n GitHub Integration (1 hour)

4. **Add GitHub Actions Trigger to n8n Workflow**
   - HTTP node to call GitHub Actions API
   - Pass task details and project info
   - Wait for GitHub Actions to complete

5. **Add Result Processing**
   - Parse GitHub Actions response
   - Update Gitea commit status
   - Send notifications

### Phase C: Testing & Migration (1-2 hours)

6. **Test Integration**
   - Verify GitHub Actions runs
   - Check OpenHands SDK execution
   - Validate result flow back to n8n

7. **Compare Performance**
   - Measure build time (SSH vs SDK)
   - Compare error handling quality
   - Check token usage/cost

### Phase D: Optimization (1 hour)

8. **Optimize Based on Results**
   - Keep SSH approach if faster/more reliable
   - Or fully migrate to GitHub Actions
   - Document findings

---

## 4. EXAMPLE IMPLEMENTATION

### A. GitHub Actions Workflow

**File:** `.github/workflows/openhands-build.yml`

```yaml
name: OpenHands Build

on:
  workflow_dispatch:
    inputs:
      task:
        description: 'Task to execute'
        required: true
        type: string
      workspace_path:
        description: 'Path to project workspace'
        required: true
        type: string
      retry_count:
        description: 'Retry attempt number'
        required: false
        type: number
        default: 0

permissions:
  contents: write
  pull-requests: write

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      TASK: ${{ github.event.inputs.task }}
      WORKSPACE_PATH: ${{ github.event.inputs.workspace_path }}
      RETRY_COUNT: ${{ github.event.inputs.retry_count || 0 }}
      LLM_MODEL: anthropic/claude-sonnet-4-5-20250929

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          path: ${{ env.WORKSPACE_PATH }}

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install uv
        uses: astral-sh/setup-uv@v6
        with:
          enable-cache: true

      - name: Install OpenHands SDK
        run: |
          uv pip install --system "openhands-sdk @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-sdk"
          uv pip install --system "openhands-tools @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-tools"          

      - name: Run OpenHands Build
        env:
          LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
          RETRY_COUNT: ${{ env.RETRY_COUNT }}
        run: |
          cd ${{ env.WORKSPACE_PATH }}
          uv run python ../agent_script.py          

      - name: Upload Build Artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: build-logs-${{ github.run_number }}
          path: |
            *.log
            output/            
          retention-days: 7

      - name: Upload Test Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results-${{ github.run_number }}
          path: |
            test-results/
            coverage/            
          retention-days: 30
```

### B. Python Agent Script

**File:** `agent_script.py`

```python
#!/usr/bin/env python3
"""
OpenHands Build Agent

Executes build/test tasks using OpenHands SDK.
Designed for GitHub Actions integration with n8n orchestration.

Usage:
  python agent_script.py

Environment Variables:
  TASK: Task description to execute
  WORKSPACE_PATH: Path to project workspace
  RETRY_COUNT: Current retry attempt number
  LLM_API_KEY: API key for LLM (required)
  LLM_MODEL: Model to use (default: anthropic/claude-sonnet-4-5-20250929)
"""

import os
import json
import sys
from datetime import datetime
from pathlib import Path

from openhands.sdk import LLM, Conversation, get_logger
from openhands.tools.preset.default import get_default_agent


logger = get_logger(__name__)


def create_build_task(workspace_path, task, retry_count):
    """Create enhanced build task with context."""

    # Check for previous errors
    error_file = Path(workspace_path) / "build-errors.json"
    previous_errors = ""
    if error_file.exists():
        try:
            with open(error_file) as f:
                errors = json.load(f)
                previous_errors = json.dumps(errors, indent=2)
        except Exception as e:
            logger.warning(f"Could not load previous errors: {e}")

    # Build enhanced prompt
    build_prompt = f"""
Build and test the project at: {workspace_path}

TASK: {task}

CURRENT ATTEMPT: {retry_count + 1}

REQUIREMENTS:
1. Install dependencies (npm install / pip install / etc.)
2. Run build process
3. Execute tests
4. Generate build report

ERROR HANDLING:
- Capture all build errors (stdout, stderr, exit codes)
- Save errors to: {workspace_path}/build-errors.json
- Report success/failure clearly

OUTPUT FORMAT:
{{
  "success": true/false,
  "exit_code": 0/1,
  "errors": ["list of errors"],
  "build_time": "seconds",
  "artifacts": ["list of generated files"]
}}

FOCUS AREAS:
- Dependency installation issues
- Build script failures
- Test failures
- Configuration problems

Return ONLY the JSON output, nothing else.
"""

    # Add previous errors if this is a retry
    if previous_errors and retry_count > 0:
        build_prompt += f"\n\nPREVIOUS BUILD ERRORS (attempt #{retry_count}):\n{previous_errors}\n\nPlease fix these specific issues."

    return build_prompt


def save_build_report(workspace_path, result):
    """Save build report to file."""

    report = {
        "timestamp": datetime.now().isoformat(),
        "workspace": workspace_path,
        "result": result,
    }

    report_path = Path(workspace_path) / "build-report.json"
    with open(report_path, "w") as f:
        json.dump(report, f, indent=2)

    logger.info(f"Build report saved to: {report_path}")
    return report_path


def main():
    """Execute build task with OpenHands SDK."""

    # Get configuration from environment
    task = os.getenv("TASK", "")
    workspace_path = os.getenv("WORKSPACE_PATH", os.getcwd())
    retry_count = int(os.getenv("RETRY_COUNT", "0"))
    api_key = os.getenv("LLM_API_KEY")
    model = os.getenv("LLM_MODEL", "anthropic/claude-sonnet-4-5-20250929")

    # Validate inputs
    if not task:
        logger.error("TASK environment variable is required")
        sys.exit(1)

    if not api_key:
        logger.error("LLM_API_KEY environment variable is required")
        sys.exit(1)

    if not os.path.exists(workspace_path):
        logger.error(f"Workspace path does not exist: {workspace_path}")
        sys.exit(1)

    logger.info(f"Starting build task (attempt #{retry_count + 1})")
    logger.info(f"Workspace: {workspace_path}")
    logger.info(f"Task: {task[:100]}...")

    try:
        # Configure LLM
        llm = LLM(
            model=model,
            api_key=api_key,
            usage_id="openhands-build",
            drop_params=True,
        )

        # Create agent
        agent = get_default_agent(
            llm=llm,
            cli_mode=True,
        )

        # Create conversation
        conversation = Conversation(
            agent=agent,
            workspace=workspace_path,
        )

        # Build task with context
        build_task = create_build_task(workspace_path, task, retry_count)

        # Execute task
        conversation.send_message(build_task)
        conversation.run()

        # Load build report
        report_path = Path(workspace_path) / "build-report.json"
        if report_path.exists():
            with open(report_path) as f:
                result = json.load(f)
                print(json.dumps(result, indent=2))
                sys.exit(0 if result.get("success") else 1)
        else:
            logger.error("Build report not found")
            sys.exit(1)

    except Exception as e:
        logger.error(f"Build failed with exception: {e}")
        # Save error report
        error_report = {
            "success": False,
            "error": str(e),
            "timestamp": datetime.now().isoformat(),
        }
        error_path = Path(workspace_path) / "build-errors.json"
        with open(error_path, "w") as f:
            json.dump(error_report, f, indent=2)
        sys.exit(1)


if __name__ == "__main__":
    main()
```

### C. n8n Integration Node

**New HTTP Node to Add to Workflow:**

```javascript
// Node: "Trigger GitHub Actions Build"

// URL
const githubRepo = "your-github-username/your-repo";
const url = `https://api.github.com/repos/${githubRepo}/actions/workflows/openhands-build.yml/dispatches`;

// Headers
const headers = {
  "Authorization": "Bearer " + $node["Get GITHUB_TOKEN"].json.token,
  "Accept": "application/vnd.github+json",
  "Content-Type": "application/json"
};

// Body
const body = {
  "ref": "main",
  "inputs": {
    "task": `Build and test project: ${$node["Extract Repo Info"].json.repo_name}`,
    "workspace_path": `/workspace/${$node["Extract Repo Info"].json.repo_name}`,
    "retry_count": $workflow.staticData.retry_count || 0
  }
};

return {
  url: url,
  method: "POST",
  headers: headers,
  body: JSON.stringify(body)
};
```

### D. n8n Result Processing Node

```javascript
// Node: "Process GitHub Actions Result"

// Get GitHub Actions response
const ghResponse = $json;

// Wait for GitHub Actions to complete
const runId = ghResponse.data.id;
const repo = "your-github-username/your-repo";

// Poll for completion
const checkUrl = `https://api.github.com/repos/${repo}/actions/runs/${runId}`;
const headers = {
  "Authorization": "Bearer " + $node["Get GITHUB_TOKEN"].json.token
};

// Make HTTP request to check status
return {
  url: checkUrl,
  headers: headers
};
```

---

## 5. MIGRATION PATH

### Option 1: Gradual Migration (Recommended)

**Timeline:** 2-3 weeks

**Week 1: Setup GitHub Actions**
- [x] Create GitHub Actions workflow
- [x] Create Python agent script
- [x] Test in isolation

**Week 2: Dual Execution**
- [ ] Run both n8n→SSH and n8n→GitHub Actions in parallel
- [ ] Compare performance and reliability
- [ ] Document differences

**Week 3: Choose & Migrate**
- [ ] Analyze results from dual execution
- [ ] Keep best approach (or hybrid)
- [ ] Remove old approach
- [ ] Complete Phase 3 with chosen method

### Option 2: Full Migration to GitHub Actions

**Timeline:** 1 week

**Day 1-2: GitHub Actions Setup**
- Create workflow and agent script
- Configure secrets

**Day 3-4: n8n Integration**
- Add HTTP trigger node
- Add result processing

**Day 5-7: Testing & Optimization**
- End-to-end testing
- Performance tuning
- Documentation

### Option 3: Keep Current (n8n→SSH)

**Timeline:** Minimal

**Just complete Phase 3 as planned**
- Already working
- Familiar infrastructure
- Known patterns

---

## 6. DETAILED COMPARISON

| Aspect | n8n→SSH | GitHub Actions | Hybrid |
|--------|---------|----------------|--------|
| **Setup Time** | ~1 hour (done) | ~2 hours | ~3 hours |
| **Complexity** | Medium (SSH) | Low (SDK) | Medium-High |
| **Performance** | Slower (SSH overhead) | Faster (direct SDK) | Medium |
| **Error Handling** | Basic | Advanced | Advanced |
| **Visual Control** | Excellent (n8n UI) | None | Good |
| **Portability** | n8n-specific | GitHub standard | Partial |
| **Learning Curve** | Low (shell) | Medium (Python) | High |
| **Maintenance** | High (wrapper) | Low (SDK) | Medium |
| **Developer Experience** | Medium | Excellent | Good |
| **Integration Effort** | High (custom) | Low (standard) | Medium |
| **Cost** | No extra | GitHub Actions minutes | GitHub Actions minutes |
| **Reliability** | Good (tested) | Excellent (mature) | Good |

---

## 7. RECOMMENDATION SUMMARY

### For Your Current Setup:

**Use: Hybrid Approach (n8n + GitHub Actions)**

**Why:**
1. **Leverages Existing Infrastructure:** Keep n8n working, add GitHub Actions
2. **Better Error Handling:** GitHub Actions SDK provides superior error reporting
3. **Standard Practices:** GitHub Actions is industry-standard
4. **Migration-Friendly:** Can switch back if needed
5. **Future-Proof:** GitHub Actions widely adopted

**Action Plan:**
1. Create GitHub Actions workflow + Python script (1 hour)
2. Add GitHub Actions trigger to n8n workflow (30 min)
3. Test end-to-end integration (1 hour)
4. Compare performance with current approach (30 min)
5. Choose final approach based on results

**Files to Create:**
- `.github/workflows/openhands-build.yml`
- `agent_script.py`
- Update n8n workflow (add 2 nodes)

### If Switching to GitHub-Only:

**Use: GitHub Actions (no n8n)**

**Why:**
- Simplest architecture
- Best performance
- Industry standard
- Better tooling

**Action Plan:**
1. Create GitHub Actions workflow
2. Create Python agent script
3. Use GitHub webhooks instead of Gitea
4. Migrate from Gitea to GitHub (major effort)

---

## 8. PERFORMANCE METRICS TO TRACK

### During Testing:

1. **Build Time**
   - n8n→SSH: _____ seconds
   - n8n→GitHub Actions: _____ seconds
   - Difference: _____% faster/slower

2. **Error Handling**
   - n8n→SSH: _____% accuracy
   - n8n→GitHub Actions: _____% accuracy

3. **Token Usage**
   - n8n→SSH: _____ tokens per build
   - n8n→GitHub Actions: _____ tokens per build

4. **Reliability**
   - n8n→SSH: _____% success rate
   - n8n→GitHub Actions: _____ % success rate

5. **Developer Experience**
   - n8n→SSH: Rating (1-5): _____
   - n8n→GitHub Actions: Rating (1-5): _____

---

## 9. CONCLUSION

**Current State:** Phase 2 working with n8n→SSH

**New Opportunity:** GitHub Actions SDK integration

**Recommendation:** Hybrid approach (n8n + GitHub Actions)

**Next Steps:**
1. Implement GitHub Actions workflow (1 hour)
2. Integrate with n8n (30 min)
3. Test & compare (1 hour)
4. Choose final approach
5. Complete Phase 3

**Benefits of GitHub Actions Approach:**
- Better error handling
- Standard tooling
- Easier maintenance
- Future scalability

**Risks of Migration:**
- Added complexity (GitHub + n8n)
- Learning curve for Python/SDK
- Potential GitHub Actions costs

**Recommendation Strength:** High
**Confidence Level:** 85%

---

## 10. REFERENCES

### Files Referenced:
- `/home/bam/claude/mvp-factory/phase3.md` - Current Phase 3 plan
- `/home/bam/claude/mvp-factory/n8n-api.md` - n8n integration details
- `/home/bam/claude/mvp-factory/openhands-subagents-doc.md` - OpenHands optimization

### External Resources:
- GitHub: OpenHands SDK Examples - https://github.com/OpenHands/software-agent-sdk
- OpenHands Documentation - https://docs.openhands.dev/sdk
- GitHub Actions Documentation - https://docs.github.com/en/actions

### Current Working Setup:
- n8n Workflow ID: `j1MmXaRhDjvkRSLa`
- Webhook: `https://n8n.oky.sh/webhook/openhands-fixed-test`
- SDK Wrapper: `/home/bam/openhands-sdk-wrapper-sh.sh`

---

**Analysis Complete**
**Ready for Implementation**

*Created: 2025-12-02*
*Status: Ready for review and implementation*