How Node.js handles many requests with one thread

The problem, in numbers

Say you have one endpoint. It saves a user to the database. It checks the input, writes one row, sends a response back. One request takes 100 ms from start to finish.

Of those 100 ms, the database call takes 90. The other 10 ms is your JavaScript.

So for 90 ms out of every 100, your CPU has nothing to do. It is waiting for a machine somewhere else to answer. If Node.js stopped and waited too, this server would handle 10 requests per second and spend 90% of its life idle.

It does not stop and wait. One thread runs your JavaScript, and that same server can handle hundreds of these requests at the same time. This post is about what happens in those 90 ms.

I wrote about the event loop and its phases in an earlier post. That post picks up when a callback is already sitting in a queue. This one is about everything that happens before it gets there.

Node.js is more than JavaScript

When people say Node.js is single-threaded, they mean one specific thing. Your JavaScript runs on one thread, called the main thread. Two lines of your code never run at the same moment.

But Node.js is not only V8, the engine that runs the JavaScript. A large part of it is C and C++. The piece that matters here is libuv, a C library that does the input and output work. Reading files, opening sockets, looking up host names. libuv has threads. It also talks to the operating system, and the operating system has plenty of threads of its own.

So the honest answer to "is Node.js single-threaded" is that your code is, and the process is not.

Asynchronous does not mean parallel

This is the idea most people get wrong, and everything else depends on it.

An asynchronous operation is not an operation that runs somewhere else at the same time. It is an operation that will not answer now, and will answer later.

When you write await db.save(user), nothing about that sentence promises a second worker. It promises that your function will pause and that something will wake it up when the answer arrives. Whether anyone is actually working during the pause is a separate question, and it has two different answers.

Two kinds of waiting

Some work needs someone to sit and wait for it. Some does not.

blocking                              non-blocking
────────                              ────────────

libuv takes a thread from             libuv asks the operating system
its pool and that thread              to watch the socket, then walks
sits there until the work             away. Nobody waits. The OS taps
is finished.                          Node.js on the shoulder later.

costs one thread                      costs no thread
pool has 4 by default                 limited only by memory

Reading a file is the blocking kind. There is no way to ask the disk to call you back, so a thread has to do the read and block while it happens. Waiting for a TCP socket is the non-blocking kind. Every operating system has a way to watch thousands of sockets at once and report which ones are ready.

Most of what a web server waits for is the second kind. That is why one thread is enough.

Three short programs make this concrete. You can run all of them yourself. The numbers below come from a machine with Node.js v24.16.0 and libuv 1.52.1 on an Apple M4 Pro, so your own numbers will differ. The pattern will not.

Experiment one: the thread pool is real

pbkdf2 is a password hashing function. Its asynchronous form is one of the rare Node.js calls that is both asynchronous and blocking, so it takes a thread from the pool and holds it.

import { pbkdf2 } from 'node:crypto';

const start = Date.now();

for (const run of [1, 2]) {
  pbkdf2('secret', 'salt', 1e7, 64, 'sha512', () => {
    console.log(`run ${run} finished after ${(Date.now() - start) / 1000}s`);
  });
}

UV_THREADPOOL_SIZE sets how many threads libuv starts. Run the file with one thread, then with two.

$ UV_THREADPOOL_SIZE=1 node blocking.mjs
run 1 finished after 1.728s
run 2 finished after 3.452s

$ UV_THREADPOOL_SIZE=2 node blocking.mjs
run 1 finished after 1.794s
run 2 finished after 1.803s

With one thread the two hashes run one after the other, and the second one finishes at double the time. With two threads they run together. The pool exists, and it is the thing deciding the order.

Experiment two: network work ignores the pool

Now the same shape of test, but with two HTTP requests instead of two hashes.

const start = Date.now();

for (const run of [1, 2]) {
  fetch('https://example.com').then(() => {
    console.log(`fetch ${run} returned after ${(Date.now() - start) / 1000}s`);
  });
}

$ UV_THREADPOOL_SIZE=1 node non-blocking.mjs
fetch 1 returned after 0.071s
fetch 2 returned after 0.072s

$ UV_THREADPOOL_SIZE=8 node non-blocking.mjs
fetch 1 returned after 0.075s
fetch 2 returned after 0.076s

Eight times the threads, same result. I ran each size three times and every run landed between 0.069s and 0.076s, apart from two slower runs around 0.13s, where both fetches slowed down together.

Giving libuv more threads does nothing for network work, because network work never asked for a thread.

Experiment three: what actually competes with what

The first two experiments each show one behaviour. This one shows the whole picture in a single program, and it is where the surprise lives.

The default pool has 4 threads. So start 4 slow hashes to fill every one of them. Then, at the same moment, do three things that all look like waiting: read a small file, open a TCP connection to an IP address, and run a fetch.

import { pbkdf2 } from 'node:crypto';
import { readFile } from 'node:fs';
import { connect } from 'node:net';

const busy = Number(process.argv[2] ?? 0);
const start = Date.now();
const since = () => `${(Date.now() - start) / 1000}s`;

// Fill every thread in the pool with slow CPU work.
for (let i = 0; i < busy; i += 1) {
  pbkdf2('secret', 'salt', 1e7, 64, 'sha512', () => {});
}

readFile(new URL(import.meta.url), () =>
  console.log(`  file read   done at ${since()}`),
);

connect({ host: '1.1.1.1', port: 443 }, function onConnect() {
  console.log(`  tcp connect done at ${since()}`);
  this.destroy();
});

fetch('https://example.com').then(() =>
  console.log(`  fetch       done at ${since()}`),
);

With an empty pool, everything is quick.

$ node competition.mjs 0
  file read   done at 0.015s
  tcp connect done at 0.027s
  fetch       done at 0.071s

Now fill the pool.

$ node competition.mjs 4
  tcp connect done at 0.027s
  file read   done at 2.065s
  fetch       done at 2.115s

Read that output slowly, because there are three separate lessons in it.

The TCP connection finished at 0.027s in both runs. Four busy threads changed nothing, because opening a socket never needed a thread. The operating system was watching it.

The file read went from 0.015s to 2.065s. It waited two full seconds for a thread to come free. Nothing was wrong with the disk. The pool was simply full.

And fetch went from 0.071s to 2.115s, which is the part people do not expect. fetch is network work, so it should behave like the TCP connection. It does, except for the first step. Before it can open a socket it has to turn example.com into an IP address, and Node.js does that with dns.lookup, which uses the thread pool. The lookup got stuck in the queue, and the whole request got stuck behind it.

That is the practical lesson of the whole post. A CPU-heavy call in one part of your server can delay file reads and outgoing HTTP requests in a completely unrelated part, and the stack trace will tell you nothing about why.

Only four kinds of work use the pool

Node.js sends these to the libuv thread pool.

Everything in fs, except the Sync versions
dns.lookup, which is also what fetch, http and most database drivers use to resolve host names
Some of crypto, namely pbkdf2, scrypt, randomBytes, randomFill and generateKeyPair
All of zlib

dns.resolve is the exception worth remembering. It sends a real DNS query over the network, so it skips the pool. dns.lookup asks the operating system, and the call that does that blocks, so it needs a thread.

Everything else in Node.js either runs on the main thread or is handed to the operating system.

So who waits for the network?

If no thread is waiting for a socket, something has to be. That something is the kernel. Getting good at this took operating systems about thirty years, and the story is short enough to tell.

One thread per connection, the old way

The first way to write a server was to give every connection its own thread. A connection arrives, you start a thread, that thread reads and writes until the client goes away.

This works. It also means that 20,000 connections cost 20,000 threads, and each thread costs memory and scheduler time even while it is doing nothing but waiting. That does not scale.

The insight that fixed it is that a waiting connection is not much data. It is a small number and a note about what to do when something happens on it.

What that small number is

The kernel keeps a table for every process. Each row in that table describes one open thing, such as a file, a socket or a device. The row holds the type, the current read position, the permissions, and a pointer to the kernel's own data for it.

A file descriptor is the row number. That is all it is. An integer index into that table. When you open a file and the kernel hands back 7, it means row 7 of your table.

On Windows the same idea exists under the name handle. The mechanics differ, the concept does not.

So instead of one thread per connection, a server can hold a list of integers and ask the kernel one question. Which of these are ready?

select, and why it stopped being enough

select was the first widely used answer. You pass it a list of descriptors, it blocks until at least one is ready, and then you walk the list to find out which.

You walk the list. Every time. With 10,000 descriptors you check 10,000 of them to find the 3 that are ready, and you do it again on the next turn. The cost grows with the number of connections you are watching, not with the number that actually did something. select also had a hard limit on how many descriptors it could handle at all.

epoll, kqueue and IOCP

The fix is to tell the kernel once which descriptors you care about, and let the kernel hand back only the ready ones.

On Linux this is epoll. You create an instance with epoll_create, register descriptors with epoll_ctl, and then call epoll_wait. The registration happens once. The wait returns only what is ready, so the cost now tracks the number of events instead of the number of connections. epoll_wait also takes a timeout, which is exactly what the event loop needs so it can wake up in time for the next setTimeout.

macOS and the BSDs have kqueue. Windows has I/O completion ports. libuv uses whichever one it finds and hides the difference from you. This is most of what libuv is for.

The event loop is a loop around that one call. Ask the kernel what is ready, run the callbacks for those things, ask again. When nothing is left to wait for, the loop ends and the process exits.

io_uring, and where it stands today

io_uring is the newer Linux design. It does not make the old question faster, it removes the question. Instead of asking what is ready and then acting, the process puts requests into a queue that it shares with the kernel, and the kernel puts results into a second shared queue. Two rings, one for work going in and one for results coming out. No asking.

Here is the part that is worth knowing, because a lot of articles still get it wrong.

That is a good reason to trust experiment three over any article, including this one. Run it on your own machine and see what your own Node.js does.

The whole path of one request

Putting it together, here is what happens between the client connecting and the client getting bytes back.

   your JavaScript              libuv                    operating system
   ───────────────              ─────                    ────────────────

   app.post('/users')
         │
         │  db.save(user)
         ├──────────────────────▶  is this call
         │                          blocking?
         │                              │
         │                    no ───────┴─────── yes
         │                     │                  │
         │                     ▼                  ▼
         │             register the         take a thread
         │             socket with     ◀──   from the pool
         │             epoll / kqueue        and block on it
         │                     │                  │
         ▼                     │                  │
   returns straight away       │                  │
   main thread is free to      │                  │
   run the next request        │                  │
                               ▼                  ▼
                        the kernel says     the thread finishes
                        the socket is       and reports back
                        ready               to the loop
                               │                  │
                               └────────┬─────────┘
                                        ▼
                             the callback joins a queue
                                        │
                                        ▼
                             the event loop runs it on
                             the main thread, in its turn

The callback queues and the order they run in are the subject of the event loop post.

Two requests at the same time

Now the original question has a boring answer, which is the sign that the explanation worked.

Two requests arrive. The kernel accepts both sockets and tells the event loop about both. The loop runs the first route handler on the main thread. That handler reaches db.save(user), which hands a socket to the kernel and returns immediately. The handler is now paused and the main thread is free, so the loop runs the second route handler. It does the same.

Both requests are now waiting. No thread belongs to either of them. Whichever database answer arrives first gets its callback queued first, and the loop runs them in the order they arrive.

Node.js never ran the two handlers at the same instant. It did not need to. It only needed to never be the one doing the waiting.

What to do with this

The rule that follows is short. Never make the main thread wait.

Every readFileSync, every JSON.parse of a 40 MB payload, every long loop over a big array is time when your server answers nobody. Not the request it is working on, and not the other 200. The 10 ms of CPU in the example at the top of this post is your whole budget per request. Spend 500 ms and every other user feels it.

For real CPU work, worker_threads gives you separate threads that run JavaScript in parallel. It has been in Node.js since v12 and there is nothing experimental about it now. I wrote about it in Node.js CPU intensive operations. Workers are for computation. They will not make your I/O faster, because your I/O was never the slow part.

And if file reads or outgoing requests go mysteriously slow under load, check whether something is holding the pool. UV_THREADPOOL_SIZE goes up to 1024, but a bigger pool usually just hides the call that should not have been there.