How Tomcat Handles Multiple Requests in Spring Boot: Threads, Connections, and Performance
Learn how Apache Tomcat handles multiple simultaneous requests in a Spring Boot application, from the HTTP connector and worker thread pool to shared singleton beans and thread safety. This guide explains the differences between maxThreads, maxConnections, and acceptCount, why timeouts and request size limits matter, and how keep-alive, NIO, and HTTP/2 fit in, along with practical lessons for diagnosing slow APIs and tuning performance based on evidence.
When multiple users call the same Spring Boot API simultaneously, how does the application handle them? Does it create a new thread, controller, and service for every user?
To understand this, we need to look at Apache Tomcat, the embedded web server commonly used by Spring Boot applications.
Tomcat manages incoming HTTP connections and request processing. Spring MVC routes requests to controllers, while your application executes business logic and accesses databases or external services.
Understanding this process helps backend engineers diagnose slow APIs, thread exhaustion, connection limits, and performance problems.
Table of Contents
- What Is the Tomcat HTTP Connector?
- How Tomcat Processes an HTTP Request
- How Tomcat Handles Multiple Requests
- maxThreads vs. maxConnections vs. acceptCount
- Why Timeouts Matter
- Request Size Limits and Security
- HTTP Keep-Alive, NIO, and HTTP/2
- Practical Performance Lessons
- Conclusion
What Is the Tomcat HTTP Connector?
The HTTP Connector is the Tomcat component responsible for accepting HTTP connections on a configured TCP port and forwarding requests for processing.
In a typical Spring Boot application, the architecture looks like this:
For example, when a client calls GET /api/bookings, Tomcat receives the request and makes it available for processing by the application. Spring MVC routes it to the appropriate controller.
Important: Tomcat does not create a new Spring Boot application for every incoming request. The application remains running, and its Spring-managed singleton beans are generally reused across requests.
Learn more in the official Tomcat HTTP Connector documentation.
How Tomcat Processes an HTTP Request
In a conventional Spring MVC application, request processing follows these steps:
- The client establishes or reuses a network connection.
- Tomcat receives the HTTP request.
- A request-processing thread handles the synchronous request.
- Spring Security filters process authentication and authorization, if configured.
DispatcherServletidentifies the appropriate controller.- The controller invokes the service, which may access the database.
- The response is returned to the client.
Each request has its own processing flow, but multiple requests can use the same Spring controller and service instances.
For example, two users might call the same endpoint:
Their requests can be processed concurrently by different threads while invoking the same singleton BookingService bean.
This is why backend developers must understand both concurrency and thread safety. A shared service should not store one user's temporary request data in mutable instance fields.
For more details, see the Spring MVC DispatcherServlet documentation.
How Tomcat Handles Multiple Requests
Tomcat uses a pool of request-processing threads. Instead of creating a new thread for every request, it can reuse existing worker threads.
Consider a simplified example:
Thread 1 and Thread 2 can execute the same service method simultaneously.
Each method invocation has its own local variables and execution state. However, mutable fields on a shared singleton bean are accessible to both threads and can cause race conditions if modified without proper synchronization.
What happens when all worker threads are busy?
Suppose all available request-processing threads are handling slow requests.
New requests may need to wait for processing capacity. If connection limits are also reached, additional connections can be queued by the operating system or eventually refused or timed out.
This creates an important distinction:
- Concurrency: How much work can be processed at the same time.
- Connections: How many network connections the server can handle.
- Throughput: How many requests the system completes over a period.
- Latency: How long an individual request takes to complete.
These metrics are related, but they are not interchangeable.
maxThreads vs. maxConnections vs. acceptCount
These three Tomcat settings are particularly important for understanding server capacity.
| Setting | Purpose |
|---|---|
maxThreads | Limits request-processing threads when using Tomcat's internal executor. |
maxConnections | Limits the number of connections Tomcat will process or keep track of, subject to connector behavior. |
acceptCount | Controls the operating-system connection queue size when the connection limit is reached. |
For Tomcat 10.1's standard NIO/NIO2 HTTP connectors, the documented defaults are:
maxThreads:200maxConnections:8192acceptCount:100
These are documented defaults, not universal production recommendations. An explicitly configured executor can change which thread settings apply.
A practical example
Imagine an application configured with:
If all 200 worker threads are busy, Tomcat cannot immediately execute unlimited additional synchronous requests. Other connections may wait for a worker to become available.
If the connection limit is reached, the operating system may queue further connection attempts. If that queue fills, new connection attempts may fail or time out.
Increasing maxConnections does not automatically increase the number of requests that can execute simultaneously. The application still needs enough processing capacity to handle its workload.
Always check the official connector reference before changing these settings, because defaults and behavior can vary by connector and configuration.
Why Timeouts Matter
Timeouts prevent clients and servers from waiting indefinitely.
Tomcat provides several timeout settings, including:
connectionTimeout: Controls how long the connector waits for the request URI after accepting a connection. Related request-body behavior also depends on configuration.keepAliveTimeout: Controls how long Tomcat waits for another request on a persistent connection.asyncTimeout: Sets the default timeout for asynchronous requests.
These settings solve different problems.
For example, keepAliveTimeout helps determine how long an idle persistent connection remains open while Tomcat waits for another request.
However, Tomcat's connection timeout is not a universal timeout for your entire API operation. A database query, external HTTP call, reverse proxy, and application-level operation may each have separate timeout settings.
For production systems, configure appropriate timeouts at each layer and ensure they work together.
Request Size Limits and Security
Tomcat provides limits that help control the size and complexity of incoming requests.
| Setting | Purpose |
|---|---|
maxHttpRequestHeaderSize | Limits the size of the HTTP request line and headers. |
maxHeaderCount | Limits the number of request headers. |
maxParameterCount | Limits parameters parsed from the query string and applicable request bodies. |
maxPartCount | Limits the number of parts in multipart requests. |
maxPostSize | Limits certain request-body data converted into request parameters; it is not a universal POST body limit. |
These limits matter because oversized headers and multipart requests can consume significant resources.
For example, an endpoint that accepts a small JSON booking request has different requirements from an endpoint that accepts large file uploads.
Avoid increasing limits blindly. First identify which limit was exceeded, determine whether the request is legitimate, and assess the resource implications of allowing it.
See the Tomcat HTTP Connector configuration reference for the exact behavior of each setting.
HTTP Keep-Alive, NIO, and HTTP/2
Three additional concepts help explain how Tomcat handles network traffic.
HTTP Keep-Alive
Keep-alive allows multiple HTTP requests to reuse a persistent connection instead of establishing a new TCP connection for every request.
This can reduce connection setup overhead.
Java NIO
Tomcat's standard HTTP/1.1 connector uses Java NIO by default.
Non-blocking network I/O allows Tomcat to manage network connections efficiently. However, this does not mean that a conventional Spring MVC controller executes all business logic without blocking.
A synchronous request can still occupy a worker thread while waiting for a database query or an external service.
HTTP/2
HTTP/2 supports multiple concurrent streams over a single connection.
This improves connection utilization, but it does not automatically eliminate application thread limits, database bottlenecks, or slow business logic.
Remember: Network connection handling and application request processing are related but separate concerns.
For more information, consult the Tomcat HTTP/2 documentation.
Practical Performance Lessons
Understanding connector configuration is useful, but changing values without measuring the application can make performance worse.
When investigating a slow Spring Boot API, follow this sequence:
- Measure latency: Identify which endpoints are slow and whether the problem occurs under load.
- Inspect application threads: Check whether Tomcat worker threads are frequently busy or waiting.
- Investigate database operations: Look for slow queries, lock contention, and exhausted database connection pools.
- Review external calls: Ensure email services and other downstream dependencies have suitable timeouts.
- Inspect connection and queue limits: Determine whether requests are waiting for available capacity.
- Tune based on evidence: Change thread counts, connection limits, or timeouts only after identifying the bottleneck.
For example, increasing Tomcat's worker thread limit may not help if the database connection pool is already exhausted. It could instead allow more requests to compete for the same limited database resources.
If the application runs in multiple containers or Kubernetes pods, remember that each application instance has its own Tomcat connector, thread pool, and Spring application context. Capacity planning must consider the entire deployment, not just one instance.
Conclusion
Apache Tomcat is a fundamental part of how a typical Spring Boot web application receives and processes HTTP requests.
As a backend engineer, you should understand how the connector accepts connections, how worker threads process synchronous requests, how connection and queue limits affect capacity, and how timeouts and request limits protect resources.
The most important takeaway is that connection capacity, request-processing capacity, and application performance are different things. Understanding their relationship helps you diagnose real production issues instead of attempting random configuration changes.
You do not need to memorize every Tomcat configuration property. Learn the core concepts, measure your application's behavior, and consult the official documentation when tuning a specific setting.
