595 lines
22 KiB
Markdown
595 lines
22 KiB
Markdown
# AI Trading Bot - System Architecture
|
||
|
||
## 1. High-Level System Design
|
||
|
||
The AI Trading Bot is a full-stack application designed for automated grid trading with AI optimization capabilities. The system follows a microservices architecture with clear separation between the backend API, real-time WebSocket connections, asynchronous task processing, and the frontend dashboard.
|
||
|
||
### System Components
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────┐
|
||
│ FRONTEND (Vue 3) │
|
||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||
│ │ Dashboard │ │ Config │ │ Charts │ │
|
||
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
|
||
└─────────┼────────────────┼─────────────────┼──────────────────────
|
||
│ │ │
|
||
└────────────────┼─────────────────┘
|
||
│
|
||
┌──────────────────────────┼──────────────────────────────────────┐
|
||
│ BACKEND (FastAPI) │
|
||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||
│ │ REST API │ │ WebSocket │ │ Celery │ │
|
||
│ │ Endpoints │ │ Handler │ │ Worker │ │
|
||
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
|
||
└─────────┼──────────────────┼──────────────────┼───────────────────<EFBFBD>
|
||
│ │ │
|
||
│ ┌────────▼────────┐ │
|
||
│ │ Redis │ │
|
||
│ │ (Caching + │ │
|
||
│ │ Queue) │ │
|
||
│ └────────┬────────┘ │
|
||
┌─────────┼──────────────────┼──────────────────┼───────────────────┐
|
||
│ │ │ │ │
|
||
│ ┌───────▼──────┐ ┌────────▼────────┐ ┌─────▼─────┐ │
|
||
│ │ Binance │ │ PostgreSQL │ │ Claude │ │
|
||
│ │ Exchange │ │ + TimescaleDB │ │ AI │ │
|
||
│ └──────────────┘ └─────────────────┘ └───────────┘ │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### Core Workflows
|
||
|
||
**1. Trading Flow**
|
||
- User configures grid parameters via UI → API → Bot record created
|
||
- Celery worker reads bot config → Connects to Binance → Places grid orders
|
||
- WebSocket sends real-time updates to UI
|
||
- Order fills trigger grid rebalancing
|
||
|
||
**2. AI Optimization Flow**
|
||
- User requests AI optimization → API → Claude API call with market data
|
||
- Claude returns grid parameters → User accepts/rejects → Config updated
|
||
|
||
**3. Real-time Data Flow**
|
||
- Binance WebSocket → Price data → Redis cache → WebSocket → UI
|
||
- Bot status changes → WebSocket → Dashboard updates
|
||
|
||
## 2. Technology Choices and Rationale
|
||
|
||
### Backend Technologies
|
||
|
||
**FastAPI**
|
||
- **Rationale**: High-performance async web framework with automatic API documentation
|
||
- **Benefits**: Type hints, automatic validation, built-in WebSocket support, excellent for real-time trading applications
|
||
- **Alternatives Considered**: Django (too heavyweight), Flask (requires more manual work)
|
||
|
||
**PostgreSQL + TimescaleDB**
|
||
- **Rationale**: Time-series data optimization for price history
|
||
- **Benefits**: ACID compliance, TimescaleDB provides automatic data partitioning and time-based queries
|
||
- **Database Choice**: PostgreSQL chosen for reliability and TimescaleDB extension for price data
|
||
|
||
**Redis**
|
||
- **Rationale**: In-memory data store for caching and message queuing
|
||
- **Benefits**: Sub-millisecond latency for price data caching, session storage, Celery broker
|
||
- **Alternatives Considered**: Memcached (no persistence), RabbitMQ (heavier for simple use case)
|
||
|
||
**Celery**
|
||
- **Rationale**: Distributed task queue for background processing
|
||
- **Benefits**: Handles long-running trading operations without blocking API, retry logic, monitoring
|
||
- **Use Cases**: Order placement, grid rebalancing, AI optimization calls
|
||
|
||
**ccxt**
|
||
- **Rationale**: Unified cryptocurrency exchange library
|
||
- **Benefits**: Single API for multiple exchanges, handles WebSocket connections, rate limiting
|
||
- **Critical For**: Binance integration, order management
|
||
|
||
**Anthropic (Claude)**
|
||
- **Rationale**: AI optimization of trading parameters
|
||
- **Benefits**: Natural language reasoning, structured JSON responses
|
||
- **Integration**: Market data → AI → Parameter suggestions
|
||
|
||
### Frontend Technologies
|
||
|
||
**Vue 3**
|
||
- **Rationale**: Progressive framework with Composition API
|
||
- **Benefits**: Reactive data binding, component-based architecture, excellent TypeScript support
|
||
- **Choice**: Chosen over React for simpler learning curve and excellent integration with Vite
|
||
|
||
**Vite**
|
||
- **Rationale**: Next-generation frontend build tool
|
||
- **Benefits**: Fast HMR, optimized builds, excellent developer experience
|
||
- **Performance**: Sub-second cold starts crucial for rapid development
|
||
|
||
**TailwindCSS**
|
||
- **Rationale**: Utility-first CSS framework
|
||
- **Benefits**: Rapid UI development, consistent design system, minimal CSS bundle size
|
||
- **Efficiency**: Perfect for trading dashboard with many similar components
|
||
|
||
**Lightweight Charts**
|
||
- **Rationale**: Financial charting library by TradingView
|
||
- **Benefits**: Real-time candlestick charts, overlay support for grid levels
|
||
- **Features**: Candlestick, volume, indicators
|
||
|
||
**Pinia**
|
||
- **Rationale**: State management for Vue 3
|
||
- **Benefits**: Type-safe, simple API, devtools integration
|
||
- **Use Case**: Bot configuration, real-time price updates, user state
|
||
|
||
### Deployment & Infrastructure
|
||
|
||
**Docker Compose**
|
||
- **Rationale**: Container orchestration for development and production
|
||
- **Benefits**: Consistent environment, easy deployment, service isolation
|
||
- **Services**: Backend, Frontend, PostgreSQL, Redis, Celery worker
|
||
|
||
## 3. Database Schema
|
||
|
||
```sql
|
||
-- Users table
|
||
CREATE TABLE users (
|
||
id SERIAL PRIMARY KEY,
|
||
email VARCHAR(255) UNIQUE NOT NULL,
|
||
api_key_encrypted TEXT, -- AES-256 encrypted Binance API key
|
||
api_secret_encrypted TEXT, -- AES-256 encrypted Binance API secret
|
||
created_at TIMESTAMP DEFAULT NOW()
|
||
);
|
||
|
||
-- Bots table
|
||
CREATE TABLE bots (
|
||
id SERIAL PRIMARY KEY,
|
||
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
|
||
strategy VARCHAR(50) NOT NULL, -- 'grid' for now, extensible
|
||
pair VARCHAR(20) NOT NULL, -- 'BTC/USDT'
|
||
status VARCHAR(20) NOT NULL, -- 'running', 'stopped', 'error'
|
||
config JSONB NOT NULL, -- Grid parameters, risk settings
|
||
created_at TIMESTAMP DEFAULT NOW(),
|
||
updated_at TIMESTAMP DEFAULT NOW()
|
||
);
|
||
|
||
-- Orders table
|
||
CREATE TABLE orders (
|
||
id SERIAL PRIMARY KEY,
|
||
bot_id INTEGER REFERENCES bots(id) ON DELETE CASCADE,
|
||
exchange_order_id VARCHAR(100), -- Binance order ID
|
||
side VARCHAR(10) NOT NULL, -- 'buy', 'sell'
|
||
price DECIMAL(20, 8) NOT NULL,
|
||
amount DECIMAL(20, 8) NOT NULL,
|
||
status VARCHAR(20) NOT NULL, -- 'open', 'filled', 'cancelled'
|
||
filled_at TIMESTAMP,
|
||
created_at TIMESTAMP DEFAULT NOW()
|
||
);
|
||
|
||
-- Trades table (filled order pairs)
|
||
CREATE TABLE trades (
|
||
id SERIAL PRIMARY KEY,
|
||
bot_id INTEGER REFERENCES bots(id) ON DELETE CASCADE,
|
||
buy_order_id INTEGER REFERENCES orders(id),
|
||
sell_order_id INTEGER REFERENCES orders(id),
|
||
profit DECIMAL(20, 8), -- Realized profit/loss
|
||
created_at TIMESTAMP DEFAULT NOW()
|
||
);
|
||
|
||
-- Price data table (TimescaleDB hypertable)
|
||
CREATE TABLE prices (
|
||
time TIMESTAMPTZ NOT NULL,
|
||
symbol VARCHAR(20) NOT NULL,
|
||
open DECIMAL(20, 8) NOT NULL,
|
||
high DECIMAL(20, 8) NOT NULL,
|
||
low DECIMAL(20, 8) NOT NULL,
|
||
close DECIMAL(20, 8) NOT NULL,
|
||
volume DECIMAL(20, 8) NOT NULL,
|
||
PRIMARY KEY (time, symbol)
|
||
);
|
||
|
||
-- Create TimescaleDB hypertable
|
||
SELECT create_hypertable('prices', 'time');
|
||
```
|
||
|
||
### Database Design Rationale
|
||
|
||
**Normalization Strategy**
|
||
- Users → Bots → Orders → Trades follows proper foreign key relationships
|
||
- JSONB config allows flexible bot parameters without schema changes
|
||
- TimescaleDB enables efficient time-series queries for historical price analysis
|
||
|
||
**Performance Optimizations**
|
||
- Indexes on frequently queried columns (bot_id, user_id, symbol, time)
|
||
- TimescaleDB automatic data partitioning by time
|
||
- Redis caching for frequently accessed bot status and current prices
|
||
|
||
**Data Integrity**
|
||
- ON DELETE CASCADE ensures proper cleanup
|
||
- CHECK constraints on ENUM values (side, status, strategy)
|
||
- Foreign key constraints maintain referential integrity
|
||
|
||
## 4. API Endpoints
|
||
|
||
### Authentication Endpoints
|
||
```python
|
||
POST /api/auth/register
|
||
Body: { email, password }
|
||
Response: { token, user_id }
|
||
|
||
POST /api/auth/login
|
||
Body: { email, password }
|
||
Response: { token, user_id }
|
||
```
|
||
|
||
### Bot Management Endpoints
|
||
```python
|
||
GET /api/bots
|
||
Response: [ { id, strategy, pair, status, config, created_at } ]
|
||
|
||
POST /api/bots
|
||
Body: { strategy, pair, config }
|
||
Response: { id, strategy, pair, status, config }
|
||
|
||
GET /api/bots/{id}
|
||
Response: { id, strategy, pair, status, config, created_at, updated_at }
|
||
|
||
PUT /api/bots/{id}
|
||
Body: { config } # Update configuration
|
||
Response: { id, updated_config }
|
||
|
||
POST /api/bots/{id}/start
|
||
Response: { status: "started" }
|
||
|
||
POST /api/bots/{id}/stop
|
||
Response: { status: "stopped" }
|
||
|
||
DELETE /api/bots/{id}
|
||
Response: { status: "deleted" }
|
||
```
|
||
|
||
### Trading Data Endpoints
|
||
```python
|
||
GET /api/bots/{id}/orders
|
||
Query: ?status=open # optional filter
|
||
Response: [ { id, side, price, amount, status, created_at } ]
|
||
|
||
GET /api/bots/{id}/trades
|
||
Query: ?limit=50&offset=0 # pagination
|
||
Response: [ { id, buy_price, sell_price, profit, created_at } ]
|
||
|
||
GET /api/bots/{id}/balance
|
||
Response: { free, locked, total }
|
||
|
||
GET /api/bots/{id}/stats
|
||
Response: { total_trades, win_rate, total_profit, max_drawdown }
|
||
```
|
||
|
||
### AI Optimization Endpoints
|
||
```python
|
||
POST /api/ai/optimize
|
||
Body: {
|
||
pair: "BTC/USDT",
|
||
capital: 10000,
|
||
current_price: 97500,
|
||
lower_bound: 95000,
|
||
upper_bound: 105000,
|
||
volatility: 0.02,
|
||
volume_24h: 1000000
|
||
}
|
||
Response: {
|
||
grid_lower: 96000,
|
||
grid_upper: 104000,
|
||
num_levels: 25,
|
||
confidence: 78,
|
||
reasoning: "...",
|
||
risk_level: "medium"
|
||
}
|
||
```
|
||
|
||
### Market Data Endpoints
|
||
```python
|
||
GET /api/market/price/{symbol}
|
||
Response: { symbol: "BTC/USDT", price: 97500, timestamp: "..." }
|
||
|
||
GET /api/market/indicators/{symbol}
|
||
Response: {
|
||
rsi: 65.5,
|
||
volatility: 0.02,
|
||
support: 97000,
|
||
resistance: 98000,
|
||
volume_24h: 1500000000
|
||
}
|
||
|
||
GET /api/market/history/{symbol}
|
||
Query: ?interval=1h&limit=100
|
||
Response: [ { time, open, high, low, close, volume } ]
|
||
```
|
||
|
||
### WebSocket Endpoints
|
||
```python
|
||
# Real-time bot status updates
|
||
WS /ws/bot/{id}
|
||
Message: {
|
||
type: "status_update",
|
||
data: { status, open_orders, realized_pnl, unrealized_pnl }
|
||
}
|
||
|
||
# Real-time price feed
|
||
WS /ws/market/{symbol}
|
||
Message: {
|
||
type: "price_update",
|
||
data: { symbol, price, timestamp }
|
||
}
|
||
|
||
# Real-time order updates
|
||
WS /ws/bot/{id}/orders
|
||
Message: {
|
||
type: "order_update",
|
||
data: { order_id, status, filled_price, filled_amount }
|
||
}
|
||
```
|
||
|
||
### Request/Response Validation
|
||
|
||
All endpoints use Pydantic models for request/response validation:
|
||
|
||
```python
|
||
# Example: Create Bot Request
|
||
class CreateBotRequest(BaseModel):
|
||
strategy: Literal["grid"]
|
||
pair: str # e.g., "BTC/USDT"
|
||
config: BotConfig
|
||
|
||
class BotConfig(BaseModel):
|
||
capital: float = Field(gt=0, le=1000000)
|
||
lower_bound: float = Field(gt=0)
|
||
upper_bound: float = Field(gt=0)
|
||
grid_levels: int = Field(ge=10, le=50)
|
||
stop_loss_pct: float = Field(ge=0, le=10)
|
||
max_daily_loss: float = Field(ge=0)
|
||
```
|
||
|
||
## 5. File Structure
|
||
|
||
```
|
||
/home/bam/openhands/workspace/aibo
|
||
├── docker-compose.yml # Container orchestration
|
||
├── .env.example # Environment variables template
|
||
├── README.md # Setup instructions
|
||
│
|
||
├── backend/ # FastAPI backend
|
||
│ ├── Dockerfile
|
||
│ ├── requirements.txt
|
||
│ ├── main.py # FastAPI app entry point
|
||
│ │
|
||
│ └── app/
|
||
│ ├── __init__.py
|
||
│ ├── config.py # Settings, environment variables
|
||
│ ├── database.py # Database connection, models
|
||
│ ├── celery.py # Celery configuration
|
||
│ ├── security.py # Auth, encryption utilities
|
||
│ │
|
||
│ ├── api/ # FastAPI routers
|
||
│ │ ├── __init__.py
|
||
│ │ ├── deps.py # Dependencies (auth, db)
|
||
│ │ ├── auth.py # Authentication endpoints
|
||
│ │ ├── bots.py # Bot management endpoints
|
||
│ │ ├── trading.py # Trading data endpoints
|
||
│ │ ├── market.py # Market data endpoints
|
||
│ │ └── ai.py # AI optimization endpoints
|
||
│ │
|
||
│ ├── models/ # Database models
|
||
│ │ ├── __init__.py
|
||
│ │ ├── user.py # User model
|
||
│ │ ├── bot.py # Bot model
|
||
│ │ ├── order.py # Order model
|
||
│ │ └── trade.py # Trade model
|
||
│ │
|
||
│ ├── schemas/ # Pydantic schemas
|
||
│ │ ├── __init__.py
|
||
│ │ ├── user.py # User schemas
|
||
│ │ ├── bot.py # Bot schemas
|
||
│ │ ├── order.py # Order schemas
|
||
│ │ └── trade.py # Trade schemas
|
||
│ │
|
||
│ ├── services/ # Business logic
|
||
│ │ ├── __init__.py
|
||
│ │ ├── binance.py # Binance API integration
|
||
│ │ ├── ai.py # Claude AI integration
|
||
│ │ ├── grid.py # Grid trading strategy
|
||
│ │ ├── websocket.py # WebSocket connection manager
|
||
│ │ └── risk.py # Risk management
|
||
│ │
|
||
│ ├── strategies/ # Trading strategies
|
||
│ │ ├── __init__.py
|
||
│ │ ├── base.py # Strategy interface
|
||
│ │ └── grid.py # Grid strategy implementation
|
||
│ │
|
||
│ └── tasks/ # Celery tasks
|
||
│ ├── __init__.py
|
||
│ ├── trading.py # Trading operations
|
||
│ ├── grid.py # Grid management
|
||
│ └── ai_optimization.py # AI parameter optimization
|
||
│
|
||
└── frontend/ # Vue 3 frontend
|
||
├── Dockerfile
|
||
├── package.json
|
||
├── vite.config.js
|
||
├── index.html
|
||
│
|
||
└── src/
|
||
├── main.js # Vue app entry point
|
||
├── App.vue # Root component
|
||
├── router/ # Vue Router
|
||
│ └── index.js
|
||
│
|
||
├── stores/ # Pinia stores
|
||
│ ├── auth.js # Authentication store
|
||
│ ├── bot.js # Bot management store
|
||
│ └── market.js # Market data store
|
||
│
|
||
├── views/ # Page components
|
||
│ ├── Dashboard.vue # Main trading dashboard
|
||
│ ├── Config.vue # Bot configuration page
|
||
│ ├── Login.vue # Login page
|
||
│ └── Register.vue # Registration page
|
||
│
|
||
├── components/ # Reusable components
|
||
│ ├── BotCard.vue # Bot status card
|
||
│ ├── PriceChart.vue # TradingView chart
|
||
│ ├── OrderBook.vue # Grid orders display
|
||
│ ├── TradeHistory.vue # Trade history table
|
||
│ ├── StatsPanel.vue # Performance statistics
|
||
│ ├── EmergencyStop.vue # Emergency stop button
|
||
│ └── AIDialog.vue # AI optimization dialog
|
||
│
|
||
├── composables/ # Vue composables
|
||
│ ├── useWebSocket.js # WebSocket connection
|
||
│ ├── useBinance.js # Binance data
|
||
│ └── useTrading.js # Trading utilities
|
||
│
|
||
└── utils/ # Helper functions
|
||
├── api.js # API client
|
||
├── formatters.js # Number/date formatting
|
||
└── grid.js # Grid calculations
|
||
```
|
||
|
||
### Directory Structure Rationale
|
||
|
||
**Backend Organization**
|
||
- **API Layer**: FastAPI routers handle HTTP requests/responses
|
||
- **Services Layer**: Business logic and external integrations
|
||
- **Models Layer**: Database models using SQLAlchemy
|
||
- **Strategies Layer**: Trading strategy implementations
|
||
- **Tasks Layer**: Celery background tasks
|
||
|
||
**Frontend Organization**
|
||
- **Views**: Page-level components (routed components)
|
||
- **Components**: Reusable UI components
|
||
- **Stores**: Pinia state management
|
||
- **Composables**: Vue 3 Composition API utilities
|
||
- **Utils**: Helper functions and API client
|
||
|
||
## 6. Component Interactions
|
||
|
||
### Backend Component Interactions
|
||
|
||
**1. Bot Lifecycle Flow**
|
||
```
|
||
User (UI) → POST /api/bots → API Router → Bot Model → Database
|
||
↓
|
||
Celery Task → Grid Service
|
||
↓
|
||
Binance Service → Exchange API
|
||
```
|
||
|
||
**2. Order Management Flow**
|
||
```
|
||
Binance WebSocket → WebSocket Service → Redis Cache → WebSocket API → UI
|
||
↓
|
||
Order Model → Database
|
||
```
|
||
|
||
**3. AI Optimization Flow**
|
||
```
|
||
User Request → AI Endpoint → Claude Service → Grid Service → Bot Config
|
||
↓
|
||
Redis Cache → WebSocket → UI
|
||
```
|
||
|
||
### Frontend Component Interactions
|
||
|
||
**1. Dashboard State Flow**
|
||
```
|
||
Page Load → Market Store → API GET /market/price → WebSocket Connect
|
||
↓
|
||
Bot Store ← Real-time Updates
|
||
↓
|
||
Components Re-render
|
||
```
|
||
|
||
**2. Bot Control Flow**
|
||
```
|
||
User Action (Start) → Bot Store → API POST /bots/{id}/start
|
||
↓
|
||
WebSocket ← Status Update
|
||
↓
|
||
Dashboard Components Update
|
||
```
|
||
|
||
### Cross-System Interactions
|
||
|
||
**1. Real-time Price Updates**
|
||
```
|
||
Binance WebSocket → Backend WebSocket Handler → Redis Pub/Sub →
|
||
Frontend WebSocket → Market Store → Chart Components
|
||
```
|
||
|
||
**2. Bot Status Synchronization**
|
||
```
|
||
Celery Worker → Database Update → Redis Cache → WebSocket Broadcast →
|
||
Frontend Store Update → UI Re-render
|
||
```
|
||
|
||
**3. Risk Management Flow**
|
||
```
|
||
Trading Task → Risk Service → Database Check →
|
||
If Limit Exceeded → Stop Bot → WebSocket Notification
|
||
```
|
||
|
||
### Message Flow Patterns
|
||
|
||
**1. Publish-Subscribe (Redis)**
|
||
- Price updates published to Redis channels
|
||
- WebSocket service subscribes and forwards to clients
|
||
- Decouples data producers from consumers
|
||
|
||
**2. Event-Driven Architecture**
|
||
- Order fills trigger grid rebalancing events
|
||
- WebSocket events broadcast to connected clients
|
||
- Celery tasks triggered by database events
|
||
|
||
**3. Observer Pattern**
|
||
- Bot status changes notify all subscribers
|
||
- Market data changes trigger chart updates
|
||
- Risk events trigger emergency stop procedures
|
||
|
||
## 7. Security Architecture
|
||
|
||
**Authentication & Authorization**
|
||
- JWT tokens for API authentication
|
||
- Role-based access control (user owns their bots)
|
||
- Password hashing with bcrypt
|
||
|
||
**Data Protection**
|
||
- API keys encrypted with AES-256 before database storage
|
||
- TLS encryption for all external communications
|
||
- Environment variables for sensitive configuration
|
||
|
||
**Trading Safety**
|
||
- Mandatory paper trading for new users
|
||
- Confirmation dialogs for live mode activation
|
||
- Daily loss limits enforced at application level
|
||
- Emergency stop capability from UI
|
||
|
||
**Rate Limiting**
|
||
- API endpoints rate limited by user/IP
|
||
- Binance API rate limiting compliance
|
||
- WebSocket connection limits
|
||
|
||
## 8. Scalability Considerations
|
||
|
||
**Horizontal Scaling**
|
||
- Stateless FastAPI servers behind load balancer
|
||
- Celery workers can be scaled independently
|
||
- Redis cluster for high availability
|
||
|
||
**Database Optimization**
|
||
- TimescaleDB automatic partitioning by time
|
||
- Read replicas for market data queries
|
||
- Connection pooling with async SQLAlchemy
|
||
|
||
**Caching Strategy**
|
||
- Redis for hot price data (1-second TTL)
|
||
- Application-level caching for bot configurations
|
||
- Database query result caching
|
||
|
||
**Monitoring & Observability**
|
||
- Health check endpoints for all services
|
||
- Structured logging with correlation IDs
|
||
- Metrics collection for trading performance
|