Build a worker pool

Why would an interviewer ask me to build a thread pool when ThreadPoolExecutor exists?

Because every hard part of concurrency is inside it, and building one is the shortest way to make you say all of them out loud. A pool is shared mutable state (the queue), a handoff between threads (the job), a result travelling backwards (the future), an exception crossing a thread boundary, a bound that has to push back on the producer, and a termination protocol. Name the executor in your first minute, show it in five lines so nobody thinks you have forgotten it, then say that you will build the parts and that you expect the interesting questions to be about shutdown. That framing is worth a mark on its own.

How many workers should the pool have?

It depends on what the jobs wait for. Work that spends its time waiting on a network or a disk can run at a concurrency far above the core count, because the waiting threads are not using a core, and the number is then set by what the thing at the other end can take rather than by your machine. Work that spends its time computing in Python gets nothing from more threads, because the global interpreter lock lets one thread run bytecode at a time, so the honest answer there is a process pool. Say which of the two the workload is before you say a number.

What should submit do when the queue is full?

There are four answers and the interviewer wants to hear that you know all of them. Block the caller, which is backpressure and the right default when the caller is a loop you control. Raise, which suits a server that would rather shed load than let a request queue grow. Drop the oldest or the newest, which suits telemetry where a fresh sample beats a complete record. Or run the job on the calling thread, which throttles the producer by making it do the work. Pick one, say why, and say what the queue size is buying: burst tolerance, against memory and against how stale the oldest queued job is by the time it runs.

Why a sentinel rather than a flag for shutdown?

Because a worker blocked in queue.get() is not running any of your code, so it cannot check a flag. It wakes when something arrives on the queue, and the only reliable way to wake it is to put something there. A sentinel is that something: a value the worker recognises as "no more work", which arrives behind every job already queued, so the queued work finishes first. One sentinel per worker, because each one stops exactly one worker and is then gone. A flag still has a use, checked between jobs, if you want a cancel that skips the remaining queue.

What does the pool do with an exception raised inside a job?

It catches it in the worker, stores it on that job's future, and re-raises it in whichever thread calls result(). Two things go wrong if you skip that. Let the exception escape the worker loop and the thread dies, so the pool silently shrinks by one and eventually has no workers at all, with nothing but a traceback on stderr to say so. Swallow it without recording it and the caller waits for ever on a future nobody will complete. The rule is that every job either completes its future or fails it, and no job may end the worker that ran it.