System Architecture Overview

Introduction

cpm (Git Repository Management) is a distributed system for centralized git repository management, synchronization, and access control. This document provides a comprehensive overview of the system architecture, component interactions, data flows, and design decisions.

Architecture Principles

Core Design Goals

1. Simplicity: Minimal dependencies, straightforward deployment
2. Efficiency: Optimized transfer protocols, incremental synchronization
3. Scalability: Support for multiple servers and repositories
4. Security: SSH-based authentication, role-based access control
5. Reliability: Data integrity, conflict detection, robust error handling

Architectural Style

cpm follows a hybrid architecture combining:

• Client-Server: Central repository management with remote servers
• Peer-to-Peer: Neighbor discovery and direct synchronization
• Command-Line Interface: Direct CLI access for automation and scripting

High-Level Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        cpm CLI Client                          │
│                                                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │
│  │   Commands   │  │ Configuration│  │  Database    │        │
│  │   (Cobra)    │  │   (Viper)    │  │  (SQLite)    │        │
│  └──────────────┘  └──────────────┘  └──────────────┘        │
│                                                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │
│  │ Repository   │  │     SSH      │  │   Server     │        │
│  │  Management  │  │  Operations  │  │  Management  │        │
│  └──────────────┘  └──────────────┘  └──────────────┘        │
│                                                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │
│  │Organization  │  │    User      │  │   Neighbor   │        │
│  │  Management  │  │  Management  │  │   Discovery  │        │
│  └──────────────┘  └──────────────┘  └──────────────┘        │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
         ┌────────────────────────────────────────────┐
         │         Network Layer (SSH/rsync)          │
         └────────────────────────────────────────────┘
                              │
         ┌────────────────────┴────────────────────────┐
         │                                              │
         ▼                                              ▼
┌─────────────────┐                           ┌─────────────────┐
│  Main Server    │◄──────────────────────────┤  Backup Server  │
│  (Primary)      │                           │  (Secondary)    │
│                 │                           │                 │
│  - Repositories │                           │  - Repositories │
│  - SSH Keys     │                           │  - SSH Keys     │
│  - git daemon   │                           │  - git daemon   │
└─────────────────┘                           └─────────────────┘
         │                                              │
         │            ┌────────────────┐               │
         └───────────►│   Neighbors    │◄──────────────┘
                      │  (P2P Network) │
                      │                │
                      │  - Auto-       │
                      │    discovery   │
                      │  - Direct sync │
                      └────────────────┘

Component Architecture

1. Command Layer

Technology: Cobra (github.com/spf13/cobra)

Responsibilities:

• Parse command-line arguments and flags
• Validate user input
• Route commands to appropriate handlers
• Format and display output
• Handle errors and exit codes

Key Components:

cmd/
├── root.go           # Root command and global flags
├── init.go           # Repository initialization
├── list.go           # Repository listing
├── push.go           # Push operations
├── pull.go           # Pull operations
├── merge.go          # Branch merging
├── org.go            # Organization management
├── user.go           # User management
├── sshkey.go         # SSH key management
├── servers.go        # Server management
├── neighbors.go      # Neighbor discovery
└── config.go         # Configuration management

2. Configuration Layer

Technology: Viper (github.com/spf13/viper)

Responsibilities:

• Load configuration from YAML files
• Provide configuration access to components
• Support environment variable overrides
• Validate configuration values
• Manage configuration updates

Configuration Schema:

main_server: string    # Main server address
data_dir: path         # Local repository storage
ssh_key_path: path     # Default SSH key
database_path: path    # SQLite database location

3. Database Layer

Technology: SQLite3 (github.com/mattn/go-sqlite3)

Responsibilities:

• Persist repository metadata
• Store user and organization data
• Track SSH keys and servers
• Manage access control relationships
• Maintain audit trails

Schema: See Database Architecture

4. Repository Management

Location: internal/repo/

Responsibilities:

• Initialize bare git repositories
• Validate repository structure
• List and discover repositories
• Transfer repositories (push/pull)
• Merge branch operations
• Repository metadata management

Key Functions:

// Repository initialization
func InitRepo(name, path string, orgID int64) error

// Repository existence check
func RepoExists(path string) (bool, error)

// Repository listing
func ListLocal(dataDir string) ([]Repository, error)
func ListRemote(server string) ([]Repository, error)

// Transfer operations
func PushRepo(repo, server string) error
func PullRepo(repo, server string) error

5. SSH Operations

Location: internal/ssh/

Responsibilities:

• Generate SSH key pairs (ed25519)
• Deploy keys to remote servers
• Retrieve keys from servers
• Key validation and fingerprinting
• SSH authentication

Architecture: See SSH Architecture

6. Server Management

Location: internal/server/

Responsibilities:

• Register and configure servers
• Health monitoring and status checks
• Server connectivity testing
• Synchronization coordination
• Neighbor discovery

Key Components:

• server.go: Server registry and CRUD operations
• sync.go: Repository synchronization
• neighbor.go: Peer discovery and management
• health.go: Health checks and monitoring

7. Organization Management

Location: internal/org/

Responsibilities:

• Create and manage organizations
• Role-based member management
• Repository association
• Access control enforcement
• Organization metadata

Roles:

• Owner: Full control including deletion
• Admin: Manage members and repositories
• Member: Read access to organization repositories

8. User Management

Location: internal/db/ (CRUD operations)

Responsibilities:

• User registration and authentication
• SSH public key storage
• Repository access tracking
• User lifecycle management

Data Flow Patterns

Repository Initialization Flow

User Command
    │
    ▼
┌─────────────────┐
│ cmd/init.go     │
│ - Parse args    │
│ - Validate name │
└─────────────────┘
    │
    ▼
┌─────────────────┐
│ repo.InitRepo() │
│ - Create bare   │
│ - Set perms     │
└─────────────────┘
    │
    ▼
┌─────────────────┐
│ db.CreateRepo() │
│ - Insert record │
│ - Return ID     │
└─────────────────┘
    │
    ▼
Success Response

Repository Push Flow

User Command: cpm push myrepo
    │
    ▼
┌──────────────────┐
│ cmd/push.go      │
│ - Get repo name  │
│ - Get target     │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Validate Repo    │
│ - Exists locally │
│ - Get metadata   │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Get Server Config│
│ - Load from DB   │
│ - Get SSH key    │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Transfer (rsync) │
│ - SSH connect    │
│ - Incremental    │
│ - Verify         │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Update DB        │
│ - Sync timestamp │
│ - Statistics     │
└──────────────────┘
    │
    ▼
Success Response

Neighbor Discovery Flow

User Command: cpm neighbors discover
    │
    ▼
┌──────────────────┐
│ Detect Network   │
│ - Get local IPs  │
│ - Parse CIDR     │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Generate Hosts   │
│ - All IPs in net │
│ - Port 9418      │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Concurrent Scan  │
│ - 50 parallel    │
│ - 2s timeout     │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Filter Results   │
│ - Reachable only │
│ - cpm servers   │
└──────────────────┘
    │
    ▼
┌──────────────────┐
│ Optional Add     │
│ - Register in DB │
│ - Mark as        │
│   neighbor       │
└──────────────────┘
    │
    ▼
Display Results

Communication Protocols

SSH-based Communication

All remote operations use SSH for security:

cpm Client
    │
    │ SSH (Port 22/custom)
    │ - Key authentication
    │ - Encrypted channel
    │
    ▼
Remote Server

rsync Protocol

Repository transfers use rsync over SSH:

rsync -avz --delete -e ssh <source> <destination>

Benefits:

• Incremental transfers (only changed files)
• Compression during transfer
• Integrity verification (checksums)
• Bandwidth optimization
• Progress tracking

Fallback: tar + scp if rsync unavailable

Neighbor Protocol (P2P)

Direct peer-to-peer communication on port 9418:

Neighbor 1 ◄──────► Neighbor 2
           TCP 9418
           - Discovery
           - Status check
           - Direct sync

Technology Stack

Core Technologies

External Dependencies

module cpm

require (
    github.com/spf13/cobra v1.8.0
    github.com/spf13/viper v1.18.0
    github.com/mattn/go-sqlite3 v1.14.22
    golang.org/x/crypto v0.18.0
)

System Requirements

Server Requirements:

• Linux/Unix operating system
• SSH server (OpenSSH recommended)
• rsync utility
• Git 2.x or later
• Disk space for repositories

Client Requirements:

• Linux/Unix/macOS operating system
• SSH client
• rsync utility
• SQLite3 support

Security Architecture

Authentication

1. SSH Key-based: All server communication uses SSH keys
2. No Password Storage: Passwords never stored or transmitted
3. Key Rotation: Support for regular key rotation

Authorization

1. Role-based Access: Organization roles (owner, admin, member)
2. Repository Permissions: Read, write, admin levels
3. Database-tracked: All permissions in SQLite

Data Security

1. Encrypted Transit: SSH encryption for all transfers
2. File Permissions: Strict permissions on keys and data
3. Secure Key Storage: 0600 for private keys, 0644 for public

Network Security

1. Firewall-friendly: Standard ports (22, 9418)
2. Local Network: Neighbor discovery on trusted networks
3. No Internet Requirement: Can operate on isolated networks

Scalability Considerations

Horizontal Scaling

• Multiple server support
• Neighbor-based distribution
• Independent server operations

Performance Optimization

• rsync for incremental transfers
• Concurrent neighbor scanning
• Database indexing on key fields
• Batch operations where possible

Resource Usage

• Memory: Minimal (10-50 MB typical)
• Disk: Proportional to repository count
• CPU: Low (peaks during transfers)
• Network: Efficient (incremental only)

Deployment Patterns

Single Server

Client ◄───► Main Server

Simple deployment for small teams or personal use.

Multi-Server

Client ◄──┬──► Main Server
          ├──► Backup Server
          └──► Dev Server

Production deployment with redundancy.

Peer Network

Client A ◄───► Neighbor A
   │              │
   ├──────────────┼────► Neighbor B
   │              │
   └──────────────┴────► Neighbor C

Distributed deployment for high availability.

Extensibility

Plugin Points

Future extensibility through:

• Custom transfer protocols
• Alternative database backends
• Plugin-based authentication
• External notification systems

API Potential

Internal packages designed for:

• Library embedding
• HTTP API wrapper
• GraphQL interface
• WebSocket notifications

Related Documentation

• Database Architecture
• SSH Operations
• Sync Architecture
• API Reference
• Configuration Guide