Build a log flusher

Why must the flusher not hold the lock while it writes?

Because the write is the slow step, and a lock held across it makes every caller who logs during a write wait for the disk. The writer takes the lock only to swap the full list for an empty one, which costs the same whatever the list holds, and calls the write after letting go. A test can prove it by passing a write function that stops inside its first call and then logging from another thread: with the lock released, that call returns while the write is still held.

Why close a batch on both a line count and a time limit?

Because each alone fails in the case the other covers. With only a count, a system that goes quiet leaves its last few lines in memory for as long as the quiet lasts. With only a timer, a busy system writes a small batch every interval and loses the saving batching was for. The writer waits on a condition with a timeout computed from the age of the batch's first line, and a caller notifies it when the batch reaches its size, so whichever comes first wins.

How do you test a time limit when Condition.wait waits in real seconds?

Inject the clock, and keep the gap small. The class computes each timeout from the injected clock and hands it to a real wait, so on a frozen fake clock the writer wakes every gap real seconds, finds no fake time has passed, and sleeps again. The test checks that nothing is written, moves the fake clock to exactly the deadline, and the writer's next lap writes the batch. The decision is tested exactly; that wait really returns after its timeout is the standard library's contract.

What exactly does a crash lose?

Everything in memory that has not been written and fsynced: the lines waiting in the buffer and the batch being written. write() returning does not mean the bytes are on the disk; it means they are in the operating system's page cache, so durability needs flush() and then os.fsync(). A caller that must know its line survived calls flush() with such a write behind it and waits for the fsync, which is the wait the design otherwise keeps off the common path.

Why not use logging.handlers.QueueHandler and QueueListener?

In an application, do: they move handler work onto a background thread, which keeps request threads off the disk. What they do not do is batch. The listener hands records over one at a time, and MemoryHandler, which does batch, flushes inside the caller that filled it and has no time limit. Write your own when one fsync per batch rather than per record is the point, or when you need flush, a drop counter, and the other guarantees as tested behaviour.