Skip to content

Backend Overview

Architecture

The BidScript backend is built with NestJS, a progressive Node.js framework that provides a robust foundation for building scalable server-side applications. It follows a modular architecture using dependency injection and TypeScript to create maintainable and testable code.

Backend Architecture Overview

graph TB
    Client[Frontend Client] --> Gateway[API Gateway]
    Gateway --> Auth[Auth Module]
    Gateway --> Controllers[Controllers]

    Controllers --> Services[Services]
    Services --> Database[Database Layer]
    Services --> Cache[Redis Cache]
    Services --> Queue[BullMQ Queues]
    Services --> AI[AI Services]

    subgraph "Core Modules"
        Auth
        Chat[Chat Module]
        ResourceHub[ResourceHub Module]
        Editor[Editor Module]
        Theme[Theme Module]
    end

    subgraph "External Integrations"
        Azure[Azure Services]
        Anthropic[Anthropic AI]
        MongoDB[MongoDB]
        PostgreSQL[PostgreSQL]
        Vector[Vector Store]
    end

    subgraph "Infrastructure"
        Cache
        Queue
        Grafana[Grafana Monitoring]
        Config[Configuration]
    end

    AI --> Azure
    AI --> Anthropic
    Database --> MongoDB
    Database --> PostgreSQL
    Services --> Vector

    Queue --> Background[Background Jobs]
    Background --> Processing[Document Processing]
    Background --> Analysis[AI Analysis]

    style Client fill:#e3f2fd
    style Gateway fill:#bbdefb
    style Auth fill:#90caf9
    style Controllers fill:#64b5f6
    style Services fill:#42a5f5
    style Database fill:#2196f3,color:#ffffff
    style Cache fill:#1976d2,color:#ffffff
    style Queue fill:#1565c0,color:#ffffff
    style AI fill:#0d47a1,color:#ffffff
    style Chat fill:#e8eaf6
    style ResourceHub fill:#c5cae9
    style Editor fill:#9fa8da
    style Theme fill:#7986cb
    style Azure fill:#5c6bc0,color:#ffffff
    style Anthropic fill:#3f51b5,color:#ffffff
    style MongoDB fill:#303f9f,color:#ffffff
    style PostgreSQL fill:#283593,color:#ffffff
    style Vector fill:#1a237e,color:#ffffff
    style Grafana fill:#e3f2fd
    style Config fill:#bbdefb
    style Background fill:#90caf9
    style Processing fill:#64b5f6
    style Analysis fill:#42a5f5

Module Dependency Graph

graph TD
    App[App Module] --> Core[Core Modules]
    App --> Infrastructure[Infrastructure]
    App --> External[External Services]

    Core --> Auth[Auth Module]
    Core --> Chat[Chat Module]
    Core --> ResourceHub[ResourceHub Module]
    Core --> Editor[Editor Module]
    Core --> Collaboration[Collaboration]

    Infrastructure --> Database[Database Module]
    Infrastructure --> Redis[Redis Module]
    Infrastructure --> BullMQ[Queue Module]
    Infrastructure --> Config[Config Module]

    External --> Azure[Azure Module]
    External --> Anthropic[Anthropic Module]
    External --> MongoDB[MongoDB Module]
    External --> Langchain[🔗 Langchain Module]

    ResourceHub --> TenderParsing[📄 Tender Parsing]
    TenderParsing --> AI[🤖 AI Processing]
    TenderParsing --> DocumentDB[📂 Document Storage]

    Chat --> Vector[🔍 Vector Store]
    Chat --> RAG[📚 RAG Service]

    Auth --> JWT[🎫 JWT Strategy]
    Auth --> Guards[🛡️ Guards]

    Editor --> DocumentProcessor[📝 Document Processor]
    Editor --> Conversion[🔄 File Conversion]

    subgraph "Data Layer"
        Database
        Redis
        MongoDB
        DocumentDB
        Vector
    end

    subgraph "AI Processing"
        AI
        RAG
        Anthropic
        Azure
    end

    subgraph "Security"
        JWT
        Guards
    end
    style App fill:#1976d2,color:#ffffff
    style Core fill:#42a5f5
    style Infrastructure fill:#64b5f6
    style External fill:#90caf9
    style Auth fill:#bbdefb
    style Chat fill:#e3f2fd
    style ResourceHub fill:#e8eaf6
    style Editor fill:#c5cae9
    style Collaboration fill:#9fa8da
    style Database fill:#7986cb,color:#ffffff
    style Redis fill:#5c6bc0,color:#ffffff
    style BullMQ fill:#3f51b5,color:#ffffff
    style Config fill:#303f9f,color:#ffffff
    style Azure fill:#283593,color:#ffffff
    style Anthropic fill:#1a237e,color:#ffffff
    style MongoDB fill:#0d47a1,color:#ffffff
    style Langchain fill:#e3f2fd
    style TenderParsing fill:#bbdefb
    style AI fill:#90caf9
    style DocumentDB fill:#64b5f6
    style Vector fill:#42a5f5
    style RAG fill:#2196f3,color:#ffffff
    style JWT fill:#1976d2,color:#ffffff
    style Guards fill:#1565c0,color:#ffffff
    style DocumentProcessor fill:#0d47a1,color:#ffffff
    style Conversion fill:#e8eaf6

Repository Structure

bidssappserver-nestjs/
├── src/                  # Source code
│   ├── main.ts           # Application bootstrap
│   ├── app.module.ts     # Main application module
│   ├── app.controller.ts # Main controller
│   ├── app.service.ts    # Main service
│   ├── anthropic/        # Anthropic AI models integration
│   ├── auth/             # Authentication and authorization
│   ├── azure/            # Azure cloud services integration
│   │   ├── azure.module.ts
│   │   ├── azure.service.ts
│   │   ├── blob.service.ts
│   │   ├── azure-sql.service.ts
│   │   ├── document-parse.service.ts
│   │   └── openai/       # Azure OpenAI integration
│   ├── bullMQ/           # BullMQ queue management for background jobs
│   ├── chat/             # Chat functionality
│   │   ├── chat.module.ts
│   │   ├── chat.controller.ts
│   │   ├── chat.service.ts
│   │   └── chat.dto.ts
│   ├── collaboration/    # Real-time collaboration features
│   ├── config/           # Application configuration
│   ├── conversation/     # Conversation management
│   ├── conversion/       # Document conversion utilities
│   ├── db/               # Database utilities
│   ├── drizzle/          # Drizzle ORM integration
│   ├── editor/           # Document editor
│   ├── grafana/          # Grafana integration for monitoring
│   ├── integration/      # External service integrations
│   ├── langchain/        # LangChain integration
│   ├── mongodb/          # MongoDB database integration
│   ├── notification/     # Notification system
│   ├── rag/              # Retrieval Augmented Generation
│   ├── redis/            # Redis cache management
│   ├── projectHub/      # Project management
│   │   └── tender-parsing/ # Tender document parsing
│   ├── theme/            # UI theming
│   ├── types/            # TypeScript type definitions
│   └── vectorstore/      # Vector database integration
├── dist/                 # Compiled JavaScript output
├── node_modules/         # Dependencies
├── .env                  # Environment variables
├── package.json          # Project metadata and dependencies
├── tsconfig.json         # TypeScript configuration
└── nest-cli.json         # NestJS CLI configuration

Key Technologies

  • NestJS: Progressive Node.js framework
  • TypeScript: Type-safe development
  • Socket.IO: Real-time bidirectional communication
  • BullMQ & Bull: Redis-based queue for background jobs
  • LangChain: Framework for LLM application development
  • Anthropic Claude: AI model integration
  • Azure OpenAI: AI services integration
  • Azure Blob Storage: Document storage
  • Azure Form Recognizer: Document processing
  • Pinecone: Vector database for semantic search
  • Redis: In-memory data store and caching
  • MongoDB: NoSQL database
  • PostgreSQL: Relational database
  • Drizzle ORM: Database ORM
  • Mongoose: MongoDB ODM
  • Yjs & Y-Websocket: Real-time collaboration
  • JWT Authentication: Secure API access

For the full list of dependencies, see Dependencies.

Core Modules

Anthropic Module

Provides integration with Anthropic's Claude models:

  • Configuration for Claude models
  • API integration
  • AI request handling

Auth Module

Handles authentication and authorization:

  • JWT token generation and validation
  • User authentication
  • Role-based access control

Azure Module

Handles all Azure cloud services integration including:

  • Blob storage for document management
  • Azure OpenAI for AI functionalities
  • Document parsing and processing
  • Azure SQL database connections
  • Azure Form Recognizer integration

BullMQ Module

Manages background job queues:

  • Job scheduling and processing
  • Linked references processing
  • Asynchronous task management

Chat Module

Manages conversational interfaces:

  • Chat history tracking
  • AI response generation
  • Context management
  • User interaction handling

Collaboration Module

Enables real-time collaboration features:

  • Document co-editing
  • Real-time updates
  • Conflict resolution
  • Using Yjs for CRDT (Conflict-free Replicated Data Type)

Config Module

Manages application configuration:

  • Environment variables
  • Feature flags
  • Service configuration

Conversation Module

Manages conversation history and context:

  • Conversation storage
  • Message history
  • Context management

Conversion Module

Handles document format conversion:

  • PDF to text
  • DOCX to text
  • Format normalisation

Drizzle Module

Provides ORM functionality for SQL databases:

  • Schema definitions
  • Query building
  • Database migrations

Editor Module

Provides collaborative document editing capabilities:

  • Real-time synchronisation
  • Document versioning
  • Collaborative annotations
  • Format conversions

Grafana Module

Integrates with Grafana for monitoring:

  • Metrics collection
  • Dashboard configuration
  • Alert management

MongoDB Module

Manages MongoDB database connections:

  • Schema definitions
  • Repository patterns
  • Data access methods

Notification Module

Handles user notifications:

  • Event-based notifications
  • Email notifications
  • In-app alerts

RAG (Retrieval Augmented Generation) Module

Enhances AI responses by retrieving relevant information from documents:

  • Document ingestion and chunking
  • Vector embeddings creation
  • Semantic search capabilities
  • Context-aware responses

Redis Module

Manages Redis cache and data storage:

  • Cache management
  • Pub/sub functionality
  • Session storage

ProjectHub Module

Manages shared resources and utilities:

  • Tender parsing services
  • Document processing
  • Data extraction

Theme Module

Handles UI theming:

  • Theme configuration
  • Style management

Vectorstore Module

Manages vector database operations:

  • Vector embedding storage
  • Similarity search
  • Integration with Pinecone

API and Communication

The backend exposes RESTful APIs and WebSocket endpoints:

  • RESTful APIs for CRUD operations
  • WebSockets for real-time updates
  • Authentication via JWT tokens
  • Rate limiting and security features

For API details, see API Documentation. For WebSocket information, see WebSocket Documentation. For environment variables, see Environment Variables. For security practices, see Security Documentation.

Development and Deployment

The backend uses Docker for containerisation and supports various deployment environments.

For setup instructions, see Setup Guide.