5bb78bd9be
- Create project structure with FastAPI - Add database models for blocks, transactions, arbitrages, pools, and DEXes - Implement Solana RPC client for fetching blockchain data - Create arbitrage detection algorithm - Implement comprehensive API endpoints for analytics - Set up database migrations with Alembic - Add detailed project documentation generated with BackendIM... (backend.im) Co-Authored-By: Claude <noreply@anthropic.com>
194 lines
6.7 KiB
Markdown
194 lines
6.7 KiB
Markdown
# Solana Arbitrage Analytics Backend
|
|
|
|
A FastAPI-based backend for tracking, analyzing, and visualizing arbitrage transactions on the Solana blockchain. This application provides detailed insights into arbitrage opportunities, performance, and patterns across various DEXes and liquidity pools.
|
|
|
|
## Features
|
|
|
|
- **Blockchain Data Fetching**: Connects to Solana RPC endpoints to fetch and store blocks and transactions
|
|
- **Arbitrage Detection**: Automatically identifies and analyzes arbitrage transactions
|
|
- **DEX & Pool Tracking**: Maintains data on DEXes and liquidity pools involved in arbitrages
|
|
- **Comprehensive Analytics**: Detailed metrics on arbitrage profitability, volume, and patterns
|
|
- **REST API**: Well-documented API endpoints for integrating with frontends and other services
|
|
- **Background Workers**: Continuously fetches new blockchain data and analyzes it for arbitrages
|
|
- **SQLite Database**: Lightweight database for storing all analytics data
|
|
|
|
## System Architecture
|
|
|
|
The system consists of several key components:
|
|
|
|
1. **Blockchain Fetcher**: Retrieves block and transaction data from Solana RPC nodes
|
|
2. **Arbitrage Detector**: Analyzes transactions to identify and record arbitrage activities
|
|
3. **Database Models**: SQLAlchemy ORM models for storing blockchain and arbitrage data
|
|
4. **API Endpoints**: FastAPI routes that provide access to the analytics data
|
|
5. **Background Workers**: Services that run continuous fetching and analysis operations
|
|
|
|
## API Endpoints
|
|
|
|
The API is organized into the following categories:
|
|
|
|
### Blocks
|
|
- `GET /api/v1/blocks`: List blocks with pagination
|
|
- `GET /api/v1/blocks/latest`: Get the latest blocks
|
|
- `GET /api/v1/blocks/{block_id}`: Get block by ID
|
|
- `GET /api/v1/blocks/height/{height}`: Get block by height
|
|
- `GET /api/v1/blocks/hash/{block_hash}`: Get block by hash
|
|
- `GET /api/v1/blocks/{block_id}/transactions`: Get transactions in a block
|
|
|
|
### Transactions
|
|
- `GET /api/v1/transactions`: List transactions with pagination
|
|
- `GET /api/v1/transactions/{transaction_id}`: Get transaction by ID
|
|
- `GET /api/v1/transactions/hash/{tx_hash}`: Get transaction by hash
|
|
- `GET /api/v1/transactions/signature/{signature}`: Get transaction by signature
|
|
- `GET /api/v1/transactions/program/{program_id}`: Get transactions involving a program
|
|
- `GET /api/v1/transactions/account/{account}`: Get transactions involving an account
|
|
|
|
### Arbitrages
|
|
- `GET /api/v1/arbitrages`: List arbitrages with pagination
|
|
- `GET /api/v1/arbitrages/stats`: Get overall arbitrage statistics
|
|
- `GET /api/v1/arbitrages/{arbitrage_id}`: Get detailed arbitrage information
|
|
- `GET /api/v1/arbitrages/initiator/{initiator_address}`: Get arbitrages by initiator
|
|
- `GET /api/v1/arbitrages/token/{token_address}`: Get arbitrages involving a token
|
|
- `GET /api/v1/arbitrages/profit-range`: Get arbitrages within a profit range
|
|
- `GET /api/v1/arbitrages/time-range`: Get arbitrages within a time range
|
|
- `GET /api/v1/arbitrages/dex/{dex_id}`: Get arbitrages involving a DEX
|
|
|
|
### DEXes
|
|
- `GET /api/v1/dexes`: List DEXes with pagination
|
|
- `GET /api/v1/dexes/{dex_id}`: Get DEX by ID
|
|
- `GET /api/v1/dexes/address/{address}`: Get DEX by address
|
|
- `GET /api/v1/dexes/name/{name}`: Get DEX by name
|
|
- `GET /api/v1/dexes/search`: Search DEXes by name
|
|
- `GET /api/v1/dexes/{dex_id}/pools`: Get pools for a DEX
|
|
|
|
### Pools
|
|
- `GET /api/v1/pools`: List pools with pagination
|
|
- `GET /api/v1/pools/active`: Get most active pools by volume
|
|
- `GET /api/v1/pools/{pool_id}`: Get pool by ID
|
|
- `GET /api/v1/pools/address/{address}`: Get pool by address
|
|
- `GET /api/v1/pools/token/{token_address}`: Get pools containing a token
|
|
- `GET /api/v1/pools/pair`: Get pools for a token pair
|
|
- `GET /api/v1/pools/tvl-range`: Get pools within a TVL range
|
|
|
|
### Stats
|
|
- `GET /api/v1/stats`: Get overall system statistics
|
|
|
|
## Setup & Installation
|
|
|
|
### Prerequisites
|
|
- Python 3.8+
|
|
- Access to a Solana RPC endpoint (public or private)
|
|
|
|
### Installation Steps
|
|
|
|
1. Clone the repository:
|
|
```bash
|
|
git clone <repository-url>
|
|
cd solana-arbitrage-analytics-backend
|
|
```
|
|
|
|
2. Create a virtual environment:
|
|
```bash
|
|
python -m venv venv
|
|
source venv/bin/activate # On Windows: venv\Scripts\activate
|
|
```
|
|
|
|
3. Install dependencies:
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
4. Set up environment variables (optional):
|
|
Create a `.env` file in the project root:
|
|
```
|
|
SOLANA_RPC_URL=https://your-solana-rpc-endpoint.com
|
|
```
|
|
|
|
5. Apply database migrations:
|
|
```bash
|
|
alembic upgrade head
|
|
```
|
|
|
|
6. Start the application:
|
|
```bash
|
|
uvicorn main:app --reload
|
|
```
|
|
|
|
7. Access the API documentation at:
|
|
```
|
|
http://localhost:8000/docs
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Starting the Background Workers
|
|
|
|
The background workers need to be started to begin fetching and analyzing blockchain data:
|
|
|
|
1. Start the blockchain fetcher:
|
|
```
|
|
POST /api/v1/stats/worker/start-fetcher
|
|
```
|
|
|
|
2. Start the arbitrage detector:
|
|
```
|
|
POST /api/v1/stats/worker/start-detector
|
|
```
|
|
|
|
3. To stop the workers:
|
|
```
|
|
POST /api/v1/stats/worker/stop
|
|
```
|
|
|
|
### Accessing Analytics
|
|
|
|
Once data has been collected, you can access various analytics through the API endpoints. The Swagger documentation provides detailed information on each endpoint and its parameters.
|
|
|
|
## Development
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
├── alembic.ini # Alembic configuration
|
|
├── app/ # Application package
|
|
│ ├── api/ # API endpoints
|
|
│ │ └── v1/ # API version 1
|
|
│ ├── core/ # Core configuration
|
|
│ ├── crud/ # Database CRUD operations
|
|
│ ├── db/ # Database setup
|
|
│ ├── models/ # SQLAlchemy models
|
|
│ ├── schemas/ # Pydantic schemas
|
|
│ ├── services/ # Business logic services
|
|
│ └── utils/ # Utility functions
|
|
├── main.py # Application entry point
|
|
├── migrations/ # Database migrations
|
|
│ └── versions/ # Migration versions
|
|
└── requirements.txt # Python dependencies
|
|
```
|
|
|
|
### Adding New Features
|
|
|
|
To add new features or models:
|
|
|
|
1. Create or update SQLAlchemy models in `app/models/`
|
|
2. Create or update Pydantic schemas in `app/schemas/`
|
|
3. Create or update CRUD operations in `app/crud/`
|
|
4. Create or update API endpoints in `app/api/v1/`
|
|
5. Generate a new migration:
|
|
```bash
|
|
alembic revision --autogenerate -m "Description of changes"
|
|
```
|
|
6. Apply the migration:
|
|
```bash
|
|
alembic upgrade head
|
|
```
|
|
|
|
## License
|
|
|
|
MIT License
|
|
|
|
## Acknowledgements
|
|
|
|
- Solana blockchain and its ecosystem
|
|
- FastAPI for the web framework
|
|
- SQLAlchemy for ORM
|
|
- Alembic for database migrations |