Building a RESTful API with Node.js and Express: What 3 Years of Production Taught Me

Last March, our startup's API went from handling 500,000 requests per day to over 10 million practically overnight. We'd landed a major client, and their integration team was already building against our endpoints. I remember sitting in our office at 2 AM on a Friday, watching our response times climb from 200ms to 8 seconds, then watching the entire API collapse under load.

That night changed everything I thought I knew about building APIs with Node.js and Express.

I'd built dozens of REST APIs before. I knew the basics—routes, middleware, controllers, all that stuff. But there's a massive gap between building an API that works on your laptop and building one that survives real production traffic. The tutorials don't tell you about connection pool exhaustion. The documentation doesn't mention that bodyParser has a default limit that'll bite you when clients start sending larger payloads. Nobody warns you that your error handling strategy will determine whether you spend weekends debugging or actually sleeping.

Here's what I learned building and scaling RESTful APIs that now serve over 50 million requests daily, support 200,000+ active users, and maintain 99.97% uptime. This isn't theory—these are the patterns, mistakes, and hard-won lessons from three years in the trenches.

The Problem with Most Express API Tutorials

Most Express tutorials follow the same pattern. They show you how to set up a basic server, add a few routes, maybe throw in some middleware, and call it done. You end up with something like this:

const express = require('express');
const app = express();

app.use(express.json());

app.get('/api/users', (req, res) => {
  res.json({ users: [] });
});

app.listen(3000);

This works fine for demos. It completely falls apart in production.

When we first deployed our API, we used this exact pattern. Within two weeks, we had our first major incident. A client accidentally sent a request with a 50MB JSON payload. Our server tried to parse it, ran out of memory, and crashed. The process manager restarted it, but the client retried immediately. Crash, restart, retry, crash. We were stuck in a crash loop for 20 minutes before I figured out what was happening.

The fix was simple—add a payload size limit:

app.use(express.json({ limit: '1mb' }));

But that incident taught me something crucial: production APIs need to be defensive from day one. You can't assume clients will behave nicely. You can't assume network conditions will be perfect. You can't assume your database will always respond quickly.

Starting with the Right Foundation

Let me show you how I structure Express APIs now. This is the foundation that's survived three years of production use, multiple scaling challenges, and enough edge cases to fill a book.

First, the project structure matters more than you think. Here's what we use:

src/
  config/
    database.js
    redis.js
    logger.js
  middleware/
    auth.js
    errorHandler.js
    rateLimiter.js
    requestLogger.js
    validator.js
  models/
    User.js
    Product.js
  routes/
    index.js
    users.js
    products.js
  services/
    UserService.js
    ProductService.js
  utils/
    errors.js
    response.js
  app.js
  server.js
tests/
  integration/
  unit/

This structure emerged after we hit 1 million users. Before that, we had everything in one giant index.js file. Bad idea. When you need to add caching, implement rate limiting, or debug a production issue at 3 AM, you'll thank yourself for organizing properly.

Here's the server.js that starts everything:

const app = require('./app');
const logger = require('./config/logger');
const config = require('./config');

const PORT = config.port || 3000;
const ENV = config.env || 'development';

const server = app.listen(PORT, () => {
  logger.info(`API server started on port ${PORT} in ${ENV} mode`);
  logger.info(`Worker process ${process.pid} is running`);
});

// Graceful shutdown handling - this saved us during deployments
process.on('SIGTERM', () => {
  logger.info('SIGTERM received, starting graceful shutdown');
  
  server.close(() => {
    logger.info('HTTP server closed');
    
    // Close database connections
    require('./config/database').close();
    
    // Close Redis connections
    require('./config/redis').quit();
    
    process.exit(0);
  });
  
  // Force shutdown after 30 seconds
  setTimeout(() => {
    logger.error('Forced shutdown after timeout');
    process.exit(1);
  }, 30000);
});

process.on('unhandledRejection', (reason, promise) => {
  logger.error('Unhandled Rejection at:', promise, 'reason:', reason);
  // Don't exit in production - log and continue
  if (ENV !== 'production') {
    process.exit(1);
  }
});

process.on('uncaughtException', (error) => {
  logger.error('Uncaught Exception:', error);
  // Exit process - let process manager restart it
  process.exit(1);
});

That graceful shutdown handling? We added it after a deployment caused active requests to fail mid-processing. Users were getting 500 errors during our deploy window. Now we wait for active requests to complete before shutting down. Our deploy error rate dropped from 2.3% to 0.01%.

The Middleware Stack That Actually Works in Production

Middleware order matters. I learned this the hard way when our authentication middleware was running after our rate limiter. Unauthenticated requests were consuming our rate limit quota. We were essentially DDoS-ing ourselves.

Here's the middleware stack we use now, in the exact order that matters:

const express = require('express');
const helmet = require('helmet');
const cors = require('cors');
const compression = require('compression');
const morgan = require('morgan');
const rateLimit = require('express-rate-limit');

const errorHandler = require('./middleware/errorHandler');
const requestLogger = require('./middleware/requestLogger');
const { authenticate } = require('./middleware/auth');

const app = express();

// 1. Security headers - ALWAYS FIRST
app.use(helmet({
  contentSecurityPolicy: {
    directives: {
      defaultSrc: ["'self'"],
      styleSrc: ["'self'", "'unsafe-inline'"]
    }
  },
  hsts: {
    maxAge: 31536000,
    includeSubDomains: true,
    preload: true
  }
}));

// 2. CORS - before any routes
app.use(cors({
  origin: process.env.ALLOWED_ORIGINS?.split(',') || '*',
  credentials: true,
  maxAge: 86400 // 24 hours
}));

// 3. Compression - early for all responses
app.use(compression({
  filter: (req, res) => {
    if (req.headers['x-no-compression']) {
      return false;
    }
    return compression.filter(req, res);
  },
  threshold: 1024 // Only compress responses > 1KB
}));

// 4. Body parsing with limits
app.use(express.json({ 
  limit: '1mb',
  verify: (req, res, buf) => {
    req.rawBody = buf.toString('utf8');
  }
}));
app.use(express.urlencoded({ 
  extended: true, 
  limit: '1mb' 
}));

// 5. Request logging
app.use(requestLogger);

// 6. Rate limiting - BEFORE authentication
const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // limit each IP to 100 requests per windowMs
  message: 'Too many requests from this IP',
  standardHeaders: true,
  legacyHeaders: false,
  // Skip rate limiting for specific IPs
  skip: (req) => {
    const whitelist = process.env.RATE_LIMIT_WHITELIST?.split(',') || [];
    return whitelist.includes(req.ip);
  }
});

app.use('/api/', limiter);

// 7. Routes
app.use('/api', require('./routes'));

// 8. 404 handler
app.use((req, res) => {
  res.status(404).json({
    error: 'Not Found',
    message: `Route ${req.method} ${req.path} not found`,
    timestamp: new Date().toISOString()
  });
});

// 9. Error handler - ALWAYS LAST
app.use(errorHandler);

module.exports = app;

Let me explain why this order matters, because we got it wrong for six months.

Security headers first: We had a security audit in Q3 2024 that flagged missing security headers. Adding Helmet was easy, but we initially put it after body parsing. The auditor pointed out that certain attacks could exploit the parser before security headers were set. Moving Helmet to position #1 was a one-line change that closed several attack vectors.

CORS before routes: We spent two days debugging why our frontend couldn't access the API. Turns out our CORS middleware was after the routes, so preflight requests were hitting our 404 handler. Moving CORS up fixed it immediately.

Compression early: We added compression after noticing our bandwidth costs were climbing. Our API responses were averaging 50KB uncompressed. With compression, they dropped to 8KB—an 84% reduction. But we initially added compression late in the middleware stack, after authentication. This meant unauthenticated requests weren't being compressed. Moving it earlier saved us about $800/month in bandwidth costs.

Rate limiting before authentication: This was the big one. We were rate-limiting AFTER authentication, which meant attackers could hammer our database with authentication attempts before hitting rate limits. Moving rate limiting before auth reduced our database load by 40% during attack attempts.

Error Handling That Actually Helps You Debug

Here's something nobody tells you: your error handling strategy determines how fast you can fix production issues.

We used to have error handling like this:

app.use((err, req, res, next) => {
  console.error(err);
  res.status(500).json({ error: 'Something went wrong' });
});

This is useless in production. When something breaks at 2 AM, you need to know exactly what happened, where it happened, and what the user was doing. Generic error messages don't cut it.

Here's our production error handler:

// utils/errors.js
class AppError extends Error {
  constructor(message, statusCode, isOperational = true) {
    super(message);
    this.statusCode = statusCode;
    this.isOperational = isOperational;
    this.timestamp = new Date().toISOString();
    Error.captureStackTrace(this, this.constructor);
  }
}

class ValidationError extends AppError {
  constructor(message, errors = []) {
    super(message, 400);
    this.errors = errors;
  }
}

class AuthenticationError extends AppError {
  constructor(message = 'Authentication required') {
    super(message, 401);
  }
}

class AuthorizationError extends AppError {
  constructor(message = 'Insufficient permissions') {
    super(message, 403);
  }
}

class NotFoundError extends AppError {
  constructor(resource = 'Resource') {
    super(`${resource} not found`, 404);
  }
}

class ConflictError extends AppError {
  constructor(message) {
    super(message, 409);
  }
}

class RateLimitError extends AppError {
  constructor(message = 'Rate limit exceeded') {
    super(message, 429);
  }
}

module.exports = {
  AppError,
  ValidationError,
  AuthenticationError,
  AuthorizationError,
  NotFoundError,
  ConflictError,
  RateLimitError
};

And the error handler middleware:

// middleware/errorHandler.js
const logger = require('../config/logger');
const { AppError } = require('../utils/errors');

module.exports = (err, req, res, next) => {
  // Default to 500 if status code not set
  err.statusCode = err.statusCode || 500;
  err.status = err.status || 'error';

  // Log error details
  const errorLog = {
    timestamp: new Date().toISOString(),
    method: req.method,
    url: req.originalUrl,
    ip: req.ip,
    userId: req.user?.id,
    statusCode: err.statusCode,
    message: err.message,
    stack: err.stack,
    body: req.body,
    query: req.query,
    params: req.params
  };

  if (err.statusCode >= 500) {
    logger.error('Server Error:', errorLog);
  } else {
    logger.warn('Client Error:', errorLog);
  }

  // Don't leak error details in production
  if (process.env.NODE_ENV === 'production') {
    // Operational errors - safe to send to client
    if (err.isOperational) {
      return res.status(err.statusCode).json({
        status: err.status,
        message: err.message,
        ...(err.errors && { errors: err.errors })
      });
    }

    // Programming or unknown errors - don't leak details
    logger.error('Non-operational error:', err);
    return res.status(500).json({
      status: 'error',
      message: 'Internal server error'
    });
  }

  // Development - send full error details
  res.status(err.statusCode).json({
    status: err.status,
    message: err.message,
    stack: err.stack,
    error: err,
    request: {
      body: req.body,
      query: req.query,
      params: req.params
    }
  });
};

This error handling saved us countless hours. When we had an issue where certain product searches were failing, the logs showed us:

{
  "timestamp": "2024-03-15T14:23:45.123Z",
  "method": "GET",
  "url": "/api/products/search",
  "ip": "192.168.1.100",
  "userId": "user_12345",
  "statusCode": 500,
  "message": "Invalid regular expression",
  "query": { "q": "laptop [unclosed" },
  "stack": "Error: Invalid regular expression..."
}

We immediately saw the issue: users were entering search queries with unescaped regex characters. We added input sanitization and solved it in 15 minutes. With generic error handling, we'd have spent hours reproducing the issue.

Database Integration That Doesn't Fall Over

Our first production database integration was a disaster. We were using a simple connection pattern:

// DON'T DO THIS
const { Pool } = require('pg');
const pool = new Pool({
  connectionString: process.env.DATABASE_URL
});

app.get('/api/users', async (req, res) => {
  const result = await pool.query('SELECT * FROM users');
  res.json(result.rows);
});

This worked fine in development. In production, it failed spectacularly.

The problem? Connection pool exhaustion. Under load, we'd open connections faster than we could close them. Eventually, we'd hit the connection limit and new requests would hang indefinitely. Users would see timeouts. Our monitoring showed connection pool utilization at 100%.

Here's what actually works:

// config/database.js
const { Pool } = require('pg');
const logger = require('./logger');

const pool = new Pool({
  host: process.env.DB_HOST,
  port: process.env.DB_PORT || 5432,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  
  // Connection pool settings - these numbers came from load testing
  max: 20, // Maximum connections
  min: 5,  // Minimum connections
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 5000, // Fail fast if can't get connection
  
  // Statement timeout - kill queries after 30s
  statement_timeout: 30000,
  
  // Query timeout - kill queries after 30s
  query_timeout: 30000
});

// Monitor pool health
pool.on('connect', (client) => {
  logger.debug('New database client connected');
});

pool.on('error', (err, client) => {
  logger.error('Unexpected database error:', err);
});

pool.on('remove', (client) => {
  logger.debug('Database client removed from pool');
});

// Health check query
const healthCheck = async () => {
  try {
    const result = await pool.query('SELECT NOW()');
    return { healthy: true, timestamp: result.rows[0].now };
  } catch (error) {
    logger.error('Database health check failed:', error);
    return { healthy: false, error: error.message };
  }
};

// Graceful shutdown
const close = async () => {
  logger.info('Closing database pool');
  await pool.end();
  logger.info('Database pool closed');
};

module.exports = {
  query: (text, params) => pool.query(text, params),
  getClient: () => pool.connect(),
  healthCheck,
  close
};

Those pool settings came from painful experience. Initially, we had max: 100 because we thought "more is better." Wrong. Our database server had a hard limit of 100 connections, and we were running 4 API servers. Do the math: 4 servers × 100 connections = 400 connections, but the database only allowed 100. Connections would hang waiting for availability.

We load-tested different pool sizes and found that max: 20 gave us the best throughput without overwhelming the database. With 4 servers, that's 80 total connections—well under the database limit with room for other services.

The statement timeout is crucial. We had a bug where a poorly written query would scan the entire users table (2 million rows) without an index. The query would run for 5+ minutes, holding a connection and blocking other requests. Adding statement timeouts meant these queries would fail fast instead of degrading the entire API.

Here's how we use this in routes:

// routes/users.js
const express = require('express');
const router = express.Router();
const db = require('../config/database');
const { NotFoundError, ValidationError } = require('../utils/errors');
const { authenticate } = require('../middleware/auth');

// Get user by ID
router.get('/:id', authenticate, async (req, res, next) => {
  try {
    const { id } = req.params;
    
    // Validate ID format
    if (!id.match(/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i)) {
      throw new ValidationError('Invalid user ID format');
    }
    
    const result = await db.query(
      'SELECT id, email, name, created_at FROM users WHERE id = $1',
      [id]
    );
    
    if (result.rows.length === 0) {
      throw new NotFoundError('User');
    }
    
    res.json({
      status: 'success',
      data: result.rows[0]
    });
  } catch (error) {
    next(error);
  }
});

// Create user with transaction
router.post('/', authenticate, async (req, res, next) => {
  const client = await db.getClient();
  
  try {
    const { email, name, password } = req.body;
    
    // Validate input
    if (!email || !name || !password) {
      throw new ValidationError('Missing required fields', [
        { field: 'email', message: 'Email is required' },
        { field: 'name', message: 'Name is required' },
        { field: 'password', message: 'Password is required' }
      ]);
    }
    
    await client.query('BEGIN');
    
    // Check if user exists
    const existingUser = await client.query(
      'SELECT id FROM users WHERE email = $1',
      [email]
    );
    
    if (existingUser.rows.length > 0) {
      throw new ConflictError('User with this email already exists');
    }
    
    // Hash password (using bcrypt in real app)
    const hashedPassword = await hashPassword(password);
    
    // Insert user
    const result = await client.query(
      `INSERT INTO users (email, name, password_hash, created_at) 
       VALUES ($1, $2, $3, NOW()) 
       RETURNING id, email, name, created_at`,
      [email, name, hashedPassword]
    );
    
    // Create user profile
    await client.query(
      'INSERT INTO user_profiles (user_id, bio, avatar_url) VALUES ($1, $2, $3)',
      [result.rows[0].id, '', null]
    );
    
    await client.query('COMMIT');
    
    res.status(201).json({
      status: 'success',
      data: result.rows[0]
    });
  } catch (error) {
    await client.query('ROLLBACK');
    next(error);
  } finally {
    client.release();
  }
});

module.exports = router;

Notice the transaction handling? We learned this after a bug where user creation would succeed but profile creation would fail, leaving orphaned user records. Wrapping both operations in a transaction ensures they either both succeed or both fail.

Also notice we're always releasing the client in the finally block. We had a leak where errors would prevent client release, slowly exhausting the connection pool. The finally block ensures cleanup happens even if an error occurs.

Authentication and Authorization That Scales

We started with session-based authentication using express-session. It worked until we scaled to multiple servers. Sessions were stored in memory, so users would be logged in on server A but not server B. Load balancer would route them to different servers randomly. Nightmare.

We switched to JWT tokens and haven't looked back:

// middleware/auth.js
const jwt = require('jsonwebtoken');
const { AuthenticationError, AuthorizationError } = require('../utils/errors');
const redis = require('../config/redis');

const JWT_SECRET = process.env.JWT_SECRET;
const JWT_EXPIRY = process.env.JWT_EXPIRY || '24h';

// Generate JWT token
const generateToken = (user) => {
  return jwt.sign(
    {
      id: user.id,
      email: user.email,
      role: user.role
    },
    JWT_SECRET,
    {
      expiresIn: JWT_EXPIRY,
      issuer: 'api.yourapp.com',
      audience: 'yourapp.com'
    }
  );
};

// Verify JWT token
const verifyToken = (token) => {
  try {
    return jwt.verify(token, JWT_SECRET, {
      issuer: '