mvp-factory-openhands/phase3-implementation-plan.md

# Phase 3 Implementation Plan: Autonomous Build Test MVP

**Date:** 2025-12-02
**Estimated Duration:** 4-5 hours
**Current Workflow:** ID j1MmXaRhDjvkRSLa (7 nodes) → Target: 11 nodes

---

## 📋 EXECUTIVE SUMMARY

Transform the current 7-node basic workflow into a production-ready autonomous CI/CD system with:
- Retry logic (max 3 attempts)
- Error feedback to OpenHands
- Gitea commit status updates
- Real project build testing

**Current State:**
- ✅ Workflow active: `j1MmXaRhDjvkRSLa`
- ✅ SSH credentials configured: `/home/bam/.ssh/n8n_key`
- ✅ OpenHands SDK wrapper: `/home/bam/openhands-sdk-wrapper-sh.sh`
- ✅ API keys available: MiniMax & DeepSeek

---

## 🎯 IMPLEMENTATION ROADMAP

### Phase 3 Workflow Design (11 Nodes)

```
[1] Gitea Webhook (existing)
    ↓
[2] Extract Repo Info (modify existing)
    ↓
[3] Initialize Retry Counter (NEW)
    ↓
[4] Start OpenHands Build (modify existing)
    ↓
[5] Wait for Completion (modify existing)
    ↓
[6] Check Build Results (modify existing)
    ↓
[7] Decision: Build OK? (NEW)
    ├─ YES → [8] Update Gitea Success → [11] Success Response
    └─ NO → [9] Format Error Feedback (NEW)
                 ↓
             [10] Check Retry Count (NEW)
                 ├─ < 3 → Loop back to [4]
                 └─ ≥ 3 → Update Gitea Failure → [11] Final Failure
```

---

## 📝 STEP-BY-STEP IMPLEMENTATION

### STEP 1: Setup Test Repository (20 min)

**Action:** Create a test repository with intentional build errors

```bash
# Via Gitea API
curl -X POST https://git.oky.sh/api/v1/user/repos \
  -H "X-Gitea-Token: {YOUR_GITEA_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "autonomous-build-test",
    "description": "Phase 3 test repository with intentional errors",
    "private": false
  }'

# Alternative: Use Gitea UI
# 1. Go to https://git.oky.sh
# 2. Click "+" → New Repository
# 3. Name: autonomous-build-test
# 4. Create with sample Node.js project
```

**Sample Test Files:**

`package.json` (with intentional error):
```json
{
  "name": "autonomous-build-test",
  "version": "1.0.0",
  "scripts": {
    "build": "node build.js"
  },
  "dependencies": {
    "express": "^4.18.2"
  }
}
```

`build.js` (intentional syntax error):
```javascript
// This will fail - missing closing brace
const express = require('express');
const app = express();

app.get('/', (req, res) => {
  res.send('Hello World!'
  // Missing closing brace
});

app.listen(3000);
console.log('Server running');
```

### STEP 2: Configure Gitea Webhook (15 min)

**Action:** Set up webhook to trigger n8n workflow

1. **Go to Repository Settings:**
   - Navigate to: https://git.oky.sh/gitadmin/autonomous-build-test
   - Settings → Webhooks

2. **Add Webhook:**
   - URL: `https://n8n.oky.sh/webhook/openhands-autonomous-build`
   - Trigger: Push events
   - HTTP Method: POST
   - Active: ✓
   - Save

3. **Test Webhook:**
   - Click "Test Delivery"
   - Should see 200 OK response

### STEP 3: Modify n8n Workflow Structure (60 min)

**Current Workflow (7 nodes):**
```
1. Gitea Webhook
2. Extract Repo Info
3. Start OpenHands Build
4. Wait 10s
5. Check Build Status
6. Format Response
7. Send Response
```

**Target Workflow (11 nodes):**

#### Node 1: Gitea Webhook ✅ (Existing)
**Configuration:**
- Path: `/webhook/openhands-autonomous-build`
- HTTP Method: POST
- No changes needed

#### Node 2: Extract Repo Info ✅ (Modify Existing)
**Current Code:**
```javascript
// Keep existing code, add repo info extraction
const data = $json;

// Extract repository information
const repoData = {
  repo_name: data.repository?.name || 'unknown',
  repo_full_name: data.repository?.full_name || 'unknown',
  owner: data.repository?.owner?.name || data.repository?.owner?.username || 'unknown',
  branch: data.ref?.replace('refs/heads/', '') || 'main',
  commit_sha: data.after || 'unknown',
  pusher: data.pusher?.name || 'unknown'
};

return repoData;
```

**Changes:** Add `repo_name`, `owner`, `branch`, `commit_sha` fields

#### Node 3: Initialize Retry Counter (NEW)
**Type:** Code Node
**Name:** "Initialize Retry Counter"

**Code:**
```javascript
// Initialize retry counter in workflow staticData
$workflow.staticData = $workflow.staticData || {};

// Initialize or increment retry count
$workflow.staticData.retry_count = ($workflow.staticData.retry_count || 0);

// Preserve repo data from previous node
const repoData = $node["Extract Repo Info"].json;

return {
  ...repoData,
  retry_count: 0,
  status: 'INITIALIZED',
  attempt: 1
};
```

**Configuration:**
- Execute Once: False (important for retry loops)

#### Node 4: Start OpenHands Build (Modify Existing)
**Type:** SSH Node
**Name:** "Execute OpenHands Build"

**Configuration:**
```
Authentication: Private Key
Host: localhost
User: bam
Private Key: /home/bam/.ssh/n8n_key
Timeout: 300000 (5 minutes)
```

**Command (JavaScript - Enhanced with feedback):**
```javascript
// Enhanced task with retry feedback
const repoData = $node["Extract Repo Info"].json;
const retryCount = $node["Initialize Retry Counter"].json.retry_count;
const buildDir = `/workspace/${repoData.repo_name}`;

// Base task
let task = `Build and test the project at ${buildDir}.

Execute the following steps:
1. cd ${buildDir}
2. If package.json exists: npm install
3. If build script exists: npm run build
4. Report build status (success/failure)
5. Capture and report any errors

Repository: ${repoData.repo_name}
Branch: ${repoData.branch}
Commit: ${repoData.commit_sha.substring(0, 8)}
`;

// Add feedback if this is a retry
if (retryCount > 0) {
  const previousOutput = $node["Check Build Results"].json;
  const errorDetails = previousOutput?.error_message || 'Unknown error';

  task += `

PREVIOUS BUILD FAILED (Attempt ${retryCount}):
Error Details:
${errorDetails}

Please analyze the previous errors and fix them. Be thorough and ensure all issues are resolved before attempting the build again.

This is retry attempt #${retryCount + 1}. Please be extremely careful and fix ALL problems.`;
}

// Execute via SDK wrapper
return `sh /home/bam/openhands-sdk-wrapper-sh.sh "${task.replace(/"/g, '\\"')}"`;
```

**Data Preservation (CRITICAL):**
```javascript
const sshOutput = $json;
const repoData = $node["Extract Repo Info"].json;
const retryCount = $node["Initialize Retry Counter"].json.retry_count;

return {
  ...repoData,              // Preserve repository data
  code: sshOutput.code,
  stdout: sshOutput.stdout,
  stderr: sshOutput.stderr,
  status: sshOutput.code === 0 ? 'SUCCESS' : 'FAILED',
  retry_count: retryCount,
  attempt: retryCount + 1
};
```

#### Node 5: Wait for Completion (Modify Existing)
**Type:** Wait Node
**Name:** "Wait for Build Completion"

**Configuration:**
- Amount: 10
- Unit: Seconds
- No changes to existing configuration

#### Node 6: Check Build Results (Modify Existing)
**Type:** Code Node
**Name:** "Evaluate Build Results"

**Code:**
```javascript
// Get OpenHands output
const openhandsOutput = $json;

// Determine success/failure
const buildSuccess = openhandsOutput.code === 0;

// Collect errors if build failed
let errorDetails = '';
if (!buildSuccess) {
  // Prefer stderr, fallback to stdout
  errorDetails = openhandsOutput.stderr || openhandsOutput.stdout || 'Build failed with no error output';
}

// Return structured results
return {
  ...openhandsOutput,  // Preserve all data
  build_success: buildSuccess,
  error_details: errorDetails,
  timestamp: new Date().toISOString()
};
```

#### Node 7: Decision: Build OK? (NEW)
**Type:** IF Node
**Name:** "Decision: Build Success?"

**Configuration:**
```
Condition: JSON
Value 1: {{ $json.build_success }}
Operation: Equal
Value 2: true
```

**True Path (YES):** → Node 8 (Update Gitea Success)
**False Path (NO):** → Node 9 (Format Error Feedback)

#### Node 8: Update Gitea Success (NEW)
**Type:** HTTP Request Node
**Name:** "Update Gitea - Success"

**Configuration:**
```
Method: POST
URL: https://git.oky.sh/api/v1/repos/{{ $node["Extract Repo Info"].json.owner }}/{{ $node["Extract Repo Info"].json.repo_name }}/statuses/{{ $node["Extract Repo Info"].json.commit_sha }}
Headers:
  - X-Gitea-Token: {YOUR_GITEA_API_TOKEN}
  - Content-Type: application/json

Body:
{
  "state": "success",
  "description": "✅ Build passed after {{ $node["Initialize Retry Counter"].json.retry_count }} attempt(s)",
  "context": "openhands/autonomous-build",
  "target_url": "https://n8n.oky.sh"
}
```

**Response Handling:**
```javascript
// Preserve data and add Gitea response
const giteaResponse = $json;
const previousData = $node["Extract Repo Info"].json;

return {
  ...previousData,
  gitea_status: 'success',
  gitea_response: giteaResponse,
  final_status: 'SUCCESS',
  build_success: true
};
```

#### Node 9: Format Error Feedback (NEW)
**Type:** Code Node
**Name:** "Format Error for Retry"

**Code:**
```javascript
// Get build results and repo data
const buildResults = $node["Check Build Results"].json;
const repoData = $node["Extract Repo Info"].json;
const retryCount = $node["Initialize Retry Counter"].json.retry_count;

// Format comprehensive error message
const errorMsg = `Build failed with the following errors:

REPOSITORY: ${repoData.repo_name}
BRANCH: ${repoData.branch}
COMMIT: ${repoData.commit_sha.substring(0, 8)}
ATTEMPT: ${retryCount + 1}/3

ERROR DETAILS:
${buildResults.error_details}

BUILD OUTPUT (stderr):
${buildResults.stderr || 'No stderr output'}

BUILD OUTPUT (stdout):
${buildResults.stdout || 'No stdout output'}

NEXT STEPS:
Please analyze these errors and fix all issues to ensure a successful build.
Focus on:
1. Dependency issues (npm install errors)
2. Build script failures
3. Code syntax errors
4. Configuration problems

After fixing, the project should build successfully with: npm install && npm run build

This is attempt ${retryCount + 1} of 3. You have ${2 - retryCount} retry(s) remaining.`;
```

**Return:**
```javascript
return {
  ...repoData,
  ...buildResults,
  status: 'FAILED',
  error_message: errorMsg,
  retry_count: retryCount,
  can_retry: retryCount < 2,  // < 3 total attempts
  formatted_error: errorMsg
};
```

#### Node 10: Check Retry Count (NEW)
**Type:** IF Node
**Name:** "Can We Retry?"

**Configuration:**
```
Condition: JSON
Value 1: {{ $json.can_retry }}
Operation: Equal
Value 2: true
```

**True Path (YES - Can Retry):** → Loop back to Node 4
**False Path (NO - Max Retries):** → Node 11 (Final Failure)

#### Node 11: Final Response (Modify Existing)
**Type:** Code Node
**Name:** "Final Response"

**Success Path Code:**
```javascript
// Format success response
const repoData = $node["Extract Repo Info"].json;
const retryCount = $node["Initialize Retry Counter"].json.retry_count;

const successResponse = {
  status: 'SUCCESS',
  repo: repoData.repo_name,
  branch: repoData.branch,
  commit: repoData.commit_sha.substring(0, 8),
  attempts: retryCount + 1,
  message: 'Build completed successfully ✅',
  timestamp: new Date().toISOString(),
  gitea_status: 'success'
};

return successResponse;
```

**Failure Path Code:**
```javascript
// Format failure response
const repoData = $node["Extract Repo Info"].json;
const errorData = $node["Format Error for Retry"].json;

const failureResponse = {
  status: 'FAILED',
  repo: repoData.repo_name,
  branch: repoData.branch,
  commit: repoData.commit_sha.substring(0, 8),
  attempts: errorData.retry_count + 1,
  max_attempts: 3,
  message: 'Build failed after 3 attempts ❌',
  errors: errorData.error_details,
  timestamp: new Date().toISOString(),
  gitea_status: 'failure',
  next_steps: 'Please review the error messages and fix the issues manually before pushing again.'
};

return failureResponse;
```

**HTTP Response Node:**
```javascript
const response = $json;
return [
  {
    statusCode: response.status === 'SUCCESS' ? 200 : 500,
    body: response
  }
];
```

---

## 🔧 CONFIGURATION CHECKLIST

### Credentials Required

- [ ] **n8n API Key:** `/home/bam/.n8n_api_key` ✅
- [ ] **SSH Key:** `/home/bam/.ssh/n8n_key` ✅
- [ ] **OpenHands API Keys:** `/home/bam/openhands/.env` ✅
- [ ] **Gitea API Token:** ⚠️ NEED TO GENERATE

### Generate Gitea API Token

```bash
# Via Gitea API (or use UI)
curl -X POST https://git.oky.sh/api/v1/users/gitadmin/tokens \
  -H "X-Gitea-Token: {ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "n8n-autonomous-build",
    "scopes": ["repo", "admin:repo_hook"]
  }'

# OR via UI:
# 1. Go to https://git.oky.sh/user/settings/applications
# 2. Click "Generate Token"
# 3. Name: n8n-autonomous-build
# 4. Scopes: repo, admin:repo_hook
# 5. Copy token (will look like: gho_xxxxxxxxxxxxxxxxx)
```

Store token securely:
```bash
echo "GITEA_API_TOKEN=gho_xxxxxxxxxxxxxxxxx" > /home/bam/.gitea_token
chmod 600 /home/bam/.gitea_token
```

---

## 🧪 TESTING STRATEGY

### Test 1: Success Path (30 min)

**Setup:**
```bash
# Create clean test repo
git clone https://git.oky.sh/gitadmin/test-success.git
cd test-success

# Create valid package.json
cat > package.json << 'EOF'
{
  "name": "test-success",
  "version": "1.0.0",
  "scripts": {
    "build": "echo 'Build successful'"
  }
}
EOF

# Create simple build script
cat > build.js << 'EOF'
console.log('Build completed successfully!');
process.exit(0);
EOF

# Commit and push
git add .
git commit -m "Test successful build"
git push origin main
```

**Expected Results:**
1. Webhook triggered ✅
2. Workflow executes ✅
3. OpenHands builds successfully ✅
4. Gitea status: "success" ✅
5. Response: 200 OK ✅

**Success Criteria:**
```json
{
  "status": "SUCCESS",
  "attempts": 1,
  "gitea_status": "success"
}
```

### Test 2: Retry Logic with Fixable Errors (45 min)

**Setup:**
```bash
# Clone test repo
git clone https://git.oky.sh/gitadmin/autonomous-build-test.git
cd autonomous-build-test

# Fix the syntax error in build.js
cat > build.js << 'EOF'
// Fixed version - proper closing braces
const express = require('express');
const app = express();

app.get('/', (req, res) => {
  res.send('Hello World!');  // Now properly closed
});

app.listen(3000, () => {
  console.log('Server running');
});
EOF

# Commit and push
git add .
git commit -m "Fix: Corrected syntax errors in build.js"
git push origin main
```

**Expected Flow:**
1. **1st Attempt:** Fails (syntax error)
2. **2nd Attempt:** OpenHands receives feedback, fixes error
3. **3rd Attempt:** Succeeds ✅

**Success Criteria:**
```json
{
  "status": "SUCCESS",
  "attempts": 3,
  "message": "Build completed successfully ✅",
  "gitea_status": "success"
}
```

### Test 3: Max Retries with Persistent Errors (45 min)

**Setup:**
```bash
# Force persistent error
cat > build.js << 'EOF'
// This will always fail
console.error('Intentional error - will not fix');
process.exit(1);
EOF

git add .
git commit -m "Test: Persistent error (should fail after 3 attempts)"
git push origin main
```

**Expected Flow:**
1. **1st Attempt:** Fails
2. **2nd Attempt:** Fails
3. **3rd Attempt:** Fails
4. **Stops:** Max retries exceeded ❌

**Success Criteria:**
```json
{
  "status": "FAILED",
  "attempts": 3,
  "max_attempts": 3,
  "message": "Build failed after 3 attempts ❌",
  "gitea_status": "failure"
}
```

### Test 4: Real Project Build (45 min)

**Setup:**
Use actual MVP project:
```bash
# Clone real project
git clone https://git.oky.sh/gitadmin/mvp-project.git
cd mvp-project

# Make a small change
echo "// Test change" >> README.md

# Commit and push
git add .
git commit -m "Test: Trigger autonomous build"
git push origin main
```

**Success Criteria:**
- OpenHands executes full build
- Dependencies installed correctly
- Build completes or fails with clear errors
- Retry logic works if needed

---

## 🚨 TROUBLESHOOTING GUIDE

### Issue 1: Retry Count Always 0

**Symptom:** Workflow doesn't retry, always shows attempt 1

**Solution:** Check Node 3 configuration
```javascript
// MUST initialize staticData
$workflow.staticData = $workflow.staticData || {};
$workflow.staticData.retry_count = ($workflow.staticData.retry_count || 0) + 1;
```

### Issue 2: Gitea Status Not Updating

**Symptom:** Commit status stays "pending" or doesn't appear

**Diagnosis:**
```bash
# Check token permissions
curl -H "X-Gitea-Token: {YOUR_TOKEN}" \
  https://git.oky.sh/api/v1/user

# Should return user info
```

**Solution:**
1. Ensure token has "repo" scope
2. Check URL format: `/api/v1/repos/{owner}/{repo}/statuses/{sha}`
3. Verify commit_sha is correct (40 character SHA)

### Issue 3: Workflow Hangs After OpenHands

**Symptom:** Workflow stops at Node 4 or Node 5

**Solution:**
1. **Check Wait Node:** Set to 10 seconds minimum
2. **Check SSH Timeout:** Set to 300000ms (5 minutes)
3. **Check SSH Authentication:** Test manually
```bash
ssh -i /home/bam/.ssh/n8n_key bam@localhost "echo 'SSH works'"
```

### Issue 4: OpenHands Not Using Feedback

**Symptom:** Retry attempts show same errors

**Solution:** Check Node 4 command generation
```javascript
// Must include previous error in task
if (retryCount > 0) {
  task += `

PREVIOUS BUILD FAILED:
${errorDetails}`;
}
```

### Issue 5: Data Lost in Retry Loop

**Symptom:** Repository info missing on retry

**Solution:** Check data preservation in Node 4
```javascript
return {
  ...repoData,              // ← CRITICAL: Preserve repo data
  code: sshOutput.code,
  stdout: sshOutput.stdout,
  stderr: sshOutput.stderr,
  status: 'SUCCESS'
};
```

---

## 📊 MONITORING & DEBUGGING

### Enable Workflow Debugging

**In n8n UI:**
1. Open workflow → Toggle "Save Manual Executions"
2. View execution history
3. Check each node's input/output

### Key Metrics to Track

- **Success Rate:** % of builds that succeed
- **Retry Efficiency:** Avg attempts per build
- **Build Time:** Duration from push to completion
- **Error Categories:** Type of failures

### Debug Commands

```bash
# Check OpenHands execution
tail -f /tmp/openhands_execution_*.log

# Test SSH manually
ssh -i /home/bam/.ssh/n8n_key bam@localhost \
  "sh /home/bam/openhands-sdk-wrapper-sh.sh 'Test task'"

# Check workflow status
curl -H "X-N8N-API-KEY: {TOKEN}" \
  https://n8n.oky.sh/api/v1/workflows/j1MmXaRhDjvkRSLa
```

---

## ⏱️ TIME-BOXED IMPLEMENTATION

### Session 1 (2 hours): Core Workflow
- [ ] Step 1: Setup test repository (20 min)
- [ ] Step 2: Configure Gitea webhook (15 min)
- [ ] Step 3: Add nodes 3, 7, 8, 9, 10 (45 min)
- [ ] Test success path (30 min)
- [ ] Buffer for issues (10 min)

### Session 2 (2 hours): Retry Logic
- [ ] Test failure path with fixable errors (45 min)
- [ ] Test max retries (45 min)
- [ ] Debug and fix issues (30 min)

### Session 3 (1 hour): Real Project & Documentation
- [ ] Test with real project (45 min)
- [ ] Document workflow (15 min)

---

## ✅ SUCCESS CRITERIA

**Must Have:**
- [ ] End-to-end workflow completes (push → build → response)
- [ ] OpenHands executes autonomously
- [ ] Retry counter prevents infinite loops (max 3)
- [ ] Error feedback improves retry attempts
- [ ] Gitea commit status updates (success/failure)
- [ ] Works with real projects

**Verification:**
```bash
# Test workflow manually
curl -X POST https://n8n.oky.sh/webhook/openhands-autonomous-build \
  -H "Content-Type: application/json" \
  -d '{
    "repository": {"name": "test-repo", "full_name": "gitadmin/test-repo"},
    "ref": "refs/heads/main",
    "after": "abc123def456"
  }'
```

**Expected Response:**
```json
{
  "status": "SUCCESS",
  "attempts": 1,
  "repo": "test-repo",
  "branch": "main",
  "gitea_status": "success"
}
```

---

## 📚 REFERENCE FILES

- **Phase 3 Plan:** `/home/bam/claude/mvp-factory/phase3.md`
- **SDK Wrapper:** `/home/bam/openhands-sdk-wrapper-sh.sh`
- **n8n API Docs:** `/home/bam/claude/mvp-factory/n8n-api.md`
- **Phase 2 Learnings:** `/home/bam/claude/mvp-factory/phase2.md`
- **Data Preservation:** `/home/bam/claude/mvp-factory/N8N_DATA_PRESERVATION_SOLUTION.md`

---

## 🎉 FINAL NOTES

**Key Reminders:**
1. **Data Preservation:** Always use `$node["Node Name"].json` pattern
2. **Retry Counter:** Initialize with `$workflow.staticData = $workflow.staticData || {}`
3. **SSH Timeout:** Set to 5 minutes minimum
4. **Wait Node:** 10 seconds between nodes
5. **Gitea Token:** Must have "repo" scope for status updates

**Workflow ID:** `j1MmXaRhDjvkRSLa`
**Target Webhook:** `https://n8n.oky.sh/webhook/openhands-autonomous-build`

---

**Implementation Ready ✅**
**Estimated Total Time:** 4-5 hours
**Can proceed immediately**