Server & API — Distributed Architecture & Data Engine

1. Project Introduction

At the center of any multi-app platform is the data engine: the single authoritative source of truth that enforces business rules, guarantees data consistency, and guards against unauthorized mutations.

The CIVA Backend API is a standalone Node.js and Express 5 service. It serves both the public customer storefront (civa-storefront) and the private operations dashboard (civa-admin) from a single authoritative MongoDB 8 database cluster.

Project:      CIVA Backend API & Data Core
Role:         Backend Architecture, Database Schema, ACID Transactions & Security
Stack:        Express 5, Node.js, MongoDB 8, Mongoose, Arcjet, JWT, OTPAuth
Status:       Active Build / Staging

2. The Situation

Modern full-stack tutorials frequently push developers toward serverless backend functions embedded directly within frontend meta-frameworks.

While serverless routes work well for simple contact forms, an e-commerce platform with concurrent inventory reservations, multi-document catalog mutations, two-factor authentication handshakes, and automated audit logging requires persistent database connection pooling and centralized transaction lifecycles.

We needed a robust, standalone backend engine that treated all frontend consumers as untrusted clients.

3. The Problem

Building a centralized hardware e-commerce backend involves severe data integrity and security challenges:

1. Multi-Document Mutation Atomicity: Creating a product involves writing to Product, inserting eight ProductVariant records, updating category product counters, and appending to ActivityLog. If an operation fails halfway through, the database ends up with corrupt, orphaned documents.
2. 2FA State Handshake Leaks: Issuing full JWT access tokens before 6-digit TOTP verification is completed renders two-factor authentication useless. But without a token, the frontend cannot prove it passed password verification.
3. Automated Scraping & Brute Force: Public catalog endpoints and login routes are prime targets for competitor scrapers and credential stuffing attacks.

4. My First Assumption

I initially thought standard Mongoose validation hooks and basic Express middleware would be enough to keep the catalog clean:

// Naive approach: individual writes without transaction sessions
const product = await Product.create(req.body);
await ProductVariant.insertMany(variants);
await ActivityLog.create({ action: 'CREATED' });

I assumed MongoDB would rarely fail mid-request, making transactional rollback logic unnecessary overkill for an initial build.

5. What I Discovered

During stress-testing and simulation of network timeouts during catalog uploads, the cracks appeared immediately:

• Orphaned Ghost Records: When an image upload timed out during variant creation, the parent Product document remained in the database with zero active variants, causing customer storefront PDP pages to crash with unhandled null pointer exceptions.
• Corrupted Inventory Counters: If an activity log write failed due to a validation constraint, the product was saved, but the category count remained unchanged, causing search filter counts to mismatch real catalog inventory.
• Token Exposure: Storing authentication tokens in frontend localStorage left sessions vulnerable to XSS extraction.

6. The Turning Point

The turning point was committing to a strict ACID Transaction Architecture and an isolated Dual-Token Handshake Protocol.

We re-architected all mutating operations to run inside native MongoDB session transactions:

Incoming API Request
  ↓
1. Start MongoDB Session & Transaction
2. Execute Parent Product Insert
3. Execute Child Variant Matrix InsertMany
4. Update Category Counters
5. Record Activity Audit Log Diff
  ↓
All Succeeded? → Commit Transaction (Data Persisted)
Any Failed?    → Abort Transaction (100% Clean Rollback)

If any single write fails, MongoDB rolls back every database modification executed in that session. Zero ghost records. Zero desynchronized counts.

7. Decisions & Experiments

Express 5 over Serverless Routes

We chose Express 5 for native asynchronous error handling, long-lived database connection pooling, and strict middleware pipelines. This eliminated cold-start connection latency and gave us complete control over rate-limiting windows.

Scoped Handshake Tokens for 2FA

To solve the intermediate 2FA verification problem, the login endpoint issues a restricted tempToken valid for exactly 300 seconds and scoped exclusively to /api/auth/verify-2fa. Full access tokens and HttpOnly refresh cookies are only minted after the TOTP code is cryptographically verified.

Layered Arcjet Defense

We integrated Arcjet middleware directly into request pipelines:

• Sliding-window rate limiting on /api/auth/* routes to block brute-force attacks.
• Real-time bot detection on public catalog endpoints to stop aggressive scraping.
• Disposable email validation on customer signup.

8. What Didn't Work

1. Monolithic Route Files with Inline Database Logic

In early iterations, controllers contained direct Mongoose queries mixed with HTTP response formatting. As business logic grew, testing business rules without mocking Express req and res objects became painful. We refactored into a clean Domain-Driven layout: Controller → Service → Model → Route.

2. Client-Accessible Refresh Tokens

Early token implementations stored refresh tokens in client memory. We replaced this with strict httpOnly, SameSite=Strict, Secure cookies with automatic token rotation on every refresh call.

9. The Final Solution

  ┌───────────────────────────┐         ┌───────────────────────────┐
  │   CIVA Admin Panel        │         │   CIVA Storefront         │
  │   (React 19 + Vite SPA)   │         │   (Next.js 16 App Router) │
  └─────────────┬─────────────┘         └─────────────┬─────────────┘
                │                                     │
                │ REST (Dual-Token + 2FA)             │ REST (Public Catalog)
                │                                     │
                └──────────────────┬──────────────────┘
                                   ▼
                ┌─────────────────────────────────────┐
                │          CIVA Backend API           │
                │     (Express 5 / Node.js Core)      │
                │   Arcjet Protection · ACID Sessions │
                └──────────────────┬──────────────────┘
                                   │
                                   ▼
                ┌─────────────────────────────────────┐
                │          MongoDB Database           │
                │     Compound Multikey Indexes       │
                └─────────────────────────────────────┘

The final backend engine provides a reliable, secure data foundation:

• ACID Guarantees: 100% of catalog mutations execute in atomic transaction sessions.
• Secure Auth: Dual-token rotation, TOTP handshakes, and HttpOnly cookie isolation.
• High-Speed Faceted Queries: Compound multikey indexes resolve complex hardware filters in under 4ms.

10. Result

• Zero Orphaned Documents: Multi-document rollbacks prevent database corruption during network timeouts.
• Sub-4ms Filter Queries: Indexed hardware specifications resolve instant faceted search results across thousands of parts.
• Airtight Security: 2FA handshakes and Arcjet rate-limiting completely block unauthorized access attempts.

11. What I Learned

Backend architecture is about planning for failure.

Hardware and networks will fail, requests will time out, and users will enter unexpected inputs. By wrapping mutations in atomic transactions and establishing strict security perimeters, you build a system that remains rock-solid no matter what happens on the client.