How Ctrl-C Reaches a Running Query: The Client's Cancel Packet

TL;DR: The client’s signal handler does nothing but decrement an atomic counter — no I/O, no allocation, no locks. The receive loop is the thing that notices the counter, writes a one-byte Cancel packet on the same socket it is reading results from, and then keeps reading until the server’s EndOfStream so the connection stays in sync for the next query.

Pressing Ctrl-C in clickhouse-client has to satisfy four constraints that pull in opposite directions:

The server must stop working on the query now, not when the client finishes reading what it already has.
The connection must be left in a state where the next query works — no leftover result blocks, no half-read packet, no lost session state.
The rows the server already produced must not be silently dropped from the output, unless the user asked for that.
The whole thing must work when the client is blocked in a read() on a socket, in a signal handler, or in the middle of formatting output.

The shape of the answer:

text
   user presses Ctrl-C
            |
            v
   SIGINT handler  ---- or ----  0x03 byte read by the keystroke thread
            |                              |
            +-------------+----------------+
                          v
          exit_after_signals.fetch_sub(1)   <-- the ONLY thing the interrupt does
                          |
        +-----------------+-----------------+
        |                                   |
   tryStop() == true                  cancelled() == true
   (the process exits)               (the receive loop sends Cancel)
                                            |
                                            v
                              read and discard the rest of the
                              result until the server says EndOfStream

Most implementations solve this by making the client disconnect: send a RST, throw away the socket, reconnect for the next query. That satisfies (1) and (2) — a dead socket is trivially in sync — but it costs a round trip, it loses session state (temporary tables, the current database, SET values), and it produces an ugly error.

ClickHouse does not disconnect. It sends a packet, and then it does the hard part: it keeps the conversation going to the end.

The signal handler does nothing

Here is the whole of the client’s interrupt handling, in src/Client/ClientApplicationBase.cpp:

cpp
// src/Client/ClientApplicationBase.cpp:49
/// This signal handler is set only for SIGINT and SIGQUIT.
void interruptSignalHandler(int signum)
{
    /// Signal handler might be called even before the setup is fully finished
    /// and client application started to process the query.
    /// Because of that we have to manually check it.
    if (auto * instance = ClientApplicationBase::instanceRawPtr(); instance)
        if (auto * base = dynamic_cast<ClientApplicationBase *>(instance); base)
            if (base->tryStopQuery())
                /// No leak check: in signal context it deadlocks if the interrupt landed under
                /// the allocator lock. Quiet, because here stderr is program output, not a log.
                safeExit(128 + signum, LeakCheck::SkipQuietly);
}

Read it again: there is no write(), no send(), no throw. The handler calls tryStopQuery(), which is an atomic decrement, and if that says “this was the last interrupt” it exits the process. The cancel packet is not sent from here — it cannot be, because sending on a socket from a signal handler is not async-signal-safe: send() may take a lock, and the interrupt may have landed while that lock was held.

So the handler’s only job is to move a number. The number is a counter of outstanding interrupts, and it lives in a small class that exists for exactly this purpose:

cpp
// src/Client/ClientBase.h:339
    class QueryInterruptHandler : private boost::noncopyable
    {
    public:
        /// Store how much interrupt signals can be before stopping the query
        /// by default stop after the first interrupt signal.
        void start(Int32 signals_before_stop = 1) { exit_after_signals.store(signals_before_stop); }

        /// Set value not greater then 0 to mark the query as stopped.
        void stop() { exit_after_signals.store(0); }

        /// Return true if the query was stopped.
        /// Query was stopped if it received at least "signals_before_stop" interrupt signals.
        bool tryStop() { return exit_after_signals.fetch_sub(1) <= 0; }
        bool cancelled() { return exit_after_signals.load() <= 0; }

        /// Return how much interrupt signals remain before stop.
        Int32 cancelled_status() { return exit_after_signals.load(); }

    private:
        std::atomic<Int32> exit_after_signals = 0;
    };

Two methods, and the difference between them is the whole trick:

tryStop() decrements and reports whether the counter has reached zero. It is called from the signal handler. It is the “should the process die?” question, and it is answered exactly once per interrupt.
cancelled() reads the counter. It is called from the receive loop. It is the “should the server be told to stop?” question, and it is answered thousands of times per second.

Because tryStop() decrements and cancelled() reads, a single Ctrl-C does both jobs at once: the counter goes from 1 to 0, so cancelled() becomes true (the server should be told to stop) while tryStop() returns false (the process should not die yet — the query owns that first interrupt). We will come back to exactly why, because it is the key to the two-stage cancel.

The other path: Ctrl-C as a byte, not a signal

There is a second, more interesting way Ctrl-C can reach the client. When a client runs interactively with the progress table enabled, it puts the terminal into raw mode and reads keystrokes itself:

cpp
// src/Client/TerminalKeystrokeInterceptor.cpp:67
    /// Set terminal to the raw terminal mode.
    struct termios raw = *orig_termios;
    raw.c_lflag &= ~(ECHO | ICANON);
    raw.c_cc[VMIN] = 0;
    raw.c_cc[VTIME] = 1;
    if (tcsetattr(fd, TCSAFLUSH, &raw))
        throw DB::ErrnoException(
            DB::ErrorCodes::SYSTEM_ERROR, "Cannot set terminal to the raw mode for the terminal referred to by file descriptor '{}'", fd);

    intercept_thread = std::make_unique<std::thread>(&TerminalKeystrokeInterceptor::run, this, callbacks);

In raw mode the terminal driver does not translate ^C into SIGINT — the byte 0x03 is simply delivered to the reader. The interceptor’s thread polls the terminal with ioctl(FIONREAD) and dispatches on the byte:

cpp
// src/Client/TerminalKeystrokeInterceptor.cpp:125
void TerminalKeystrokeInterceptor::runImpl(const DB::TerminalKeystrokeInterceptor::CallbackMap & map) const
{
    char ch = 0;

    int available = 0;
    if (ioctl(fd, FIONREAD, &available) < 0)
        throw DB::ErrnoException(DB::ErrorCodes::SYSTEM_ERROR, "ioctl({}, FIONREAD)", fd);

    if (available <= 0)
        return;

    if (read(fd, &ch, 1) > 0)  /// NOLINT(clang-analyzer-unix.BlockInCriticalSection)
    {
        auto it = map.find(ch);
        if (it != map.end())
        {
            auto fn = it->second;
            fn();
        }
    }
}

and the client registers the keys it cares about — the space bar that toggles the progress table, and, in the embedded client only, 0x03:

cpp
// src/Client/ClientBase.cpp:1541
void ClientBase::initKeystrokeInterceptor()
{
    if (is_interactive && need_render_progress_table && progress_table_toggle_enabled)
    {
        keystroke_interceptor = std::make_unique<TerminalKeystrokeInterceptor>(stdin_fd, error_stream);
        keystroke_interceptor->registerCallback(' ', [this]() { progress_table_toggle_on = !progress_table_toggle_on; });

        if (isEmbeeddedClient())
            keystroke_interceptor->registerCallback(0x03, [this]() { tryStopQuery(); });
    }
}

So in the embedded client, Ctrl-C is not a signal at all: it is a byte read by a dedicated thread, which calls the very same tryStopQuery() — one code path for both, and no signal-handler-safety questions at all. In the ordinary clickhouse-client the interceptor only steals the space bar, and Ctrl-C remains a SIGINT.

The receive loop: where the cancel actually happens

The counter is checked in ClientBase::receiveResult, the loop that reads the server’s answer:

cpp
// src/Client/ClientBase.cpp:1968
    while (true)
    {
        Stopwatch receive_watch(CLOCK_MONOTONIC_COARSE);

        while (true)
        {
            /// Has the Ctrl+C been pressed and thus the query should be cancelled?
            /// If this is the case, inform the server about it and receive the remaining packets
            /// to avoid losing sync.
            if (!cancelled)
            {
                if (partial_result_on_first_cancel && query_interrupt_handler.cancelled_status() == signals_before_stop - 1)
                {
                    sendCancel();
                    /// First cancel reading request was sent. Next requests will only be with a full cancel
                    partial_result_on_first_cancel = false;
                }
                else if (query_interrupt_handler.cancelled())
                {
                    cancelQuery();
                }
                else
                {
                    double elapsed = receive_watch.elapsedSeconds();
                    if (break_on_timeout && elapsed > receive_timeout.totalSeconds())
                    {
                        PreformattedMessage error_message = PreformattedMessage::create
                        (
                            "Timeout exceeded while receiving data from server. Waited for {} seconds, timeout is {} seconds.",
                            static_cast<size_t>(elapsed),
                            receive_timeout.totalSeconds()
                        );

                        client_exception = std::make_unique<Exception>(error_message, ErrorCodes::TIMEOUT_EXCEEDED);
                        have_error = true;

                        cancelQuery();
                    }
                }
            }

Three things to notice.

The check is inside the poll loop, before the poll. The loop is:

cpp
            /// Poll for changes after a cancellation check, otherwise it never reached
            /// because of progress updates from server.

            if (connection->poll(poll_interval))
                break;

so the client wakes up at most every poll_interval microseconds, checks the counter, and only then blocks in poll(). The poll interval is derived from the receive timeout and clamped:

cpp
// src/Client/ClientBase.cpp:1955
    const auto receive_timeout = connection_parameters.timeouts.receive_timeout;
    constexpr size_t default_poll_interval = 1000000; /// in microseconds
    constexpr size_t min_poll_interval = 5000; /// in microseconds
    const size_t poll_interval
        = std::max(min_poll_interval, std::min<size_t>(receive_timeout.totalMicroseconds(), default_poll_interval));

Between 5 ms and 1 s. That is the client-side cancellation latency — the time between the user pressing Ctrl-C and the server being told. The server-side article covers the other half of that latency.

The “keep receiving to avoid losing sync” comment is the whole design. The client does not stop reading after it sends the cancel. It keeps reading until the server says the query is over.

The same machinery cancels for two other reasons. A receive timeout (TIMEOUT_EXCEEDED) and a broken output pipe (a LocalFormatError from the output formatter) both call sendCancel and then keep receiving:

cpp
// src/Client/ClientBase.cpp:2040
        catch (const LocalFormatError &)
        {
            /// Remember the first exception.
            if (!local_format_error)
                local_format_error = std::current_exception();
            sendCancel(std::current_exception());
        }

which is why head -1 on a clickhouse-client pipe does not leave the server burning CPU, and does not leave the connection unusable.

The packet itself

The cancel is one VarUInt on the same socket:

cpp
// src/Client/Connection.cpp:1236
void Connection::sendCancel()
{
    /// If we already disconnected.
    if (!out)
        throw Exception(ErrorCodes::NETWORK_ERROR, "Connection to {} terminated", getDescription());

    writeVarUInt(Protocol::Client::Cancel, *out);
    out->finishChunk();
    out->next();
}

Protocol::Client::Cancel is 3:

cpp
// src/Core/Protocol.h:123
            Cancel = 3,                     /// Cancel the query execution.

That is the entire wire format of a cancellation: one byte, 0x03, plus the chunked-protocol framing. The TCP socket is full-duplex, so the client can write it while it is in the middle of reading result blocks — no second connection, no out-of-band channel, no kill -SIGINT on the server.

The client’s sendCancel wrapper handles the one case where this is impossible:

cpp
// src/Client/ClientBase.cpp:2949
bool ClientBase::sendCancel(std::exception_ptr exception_ptr)
{
    if (!connection->isConnected())
    {
        error_stream << "Cannot send Cancel due to connection is lost";
        if (exception_ptr)
        {
            error_stream << ": ";
            error_stream << getExceptionMessage(exception_ptr, /*with_stacktrace=*/ true);
        }
        error_stream << '\n';
        return false;
    }
    else
    {
        connection->sendCancel();
        return true;
    }
}

and the user-visible counterpart clears the progress display and says so:

cpp
// src/Client/ClientBase.cpp:2969
void ClientBase::cancelQuery()
{
    sendCancel();

    stopKeystrokeInterceptorIfExists();

    if (need_render_progress && tty_buf)
    {
        std::unique_lock lock(tty_mutex);
        progress_indication.clearProgressOutput(*tty_buf, lock);
    }
    if (need_render_progress_table && tty_buf)
    {
        std::unique_lock lock(tty_mutex);
        progress_table.clearTableOutput(*tty_buf, lock);
    }

    if (is_interactive)
        output_stream << "Cancelling query." << std::endl;

    cancelled = true;
}

Staying in sync: the part everyone forgets

After the cancel, the server may still have blocks in flight. The client keeps reading them and discards them, which is what keeps the protocol aligned:

cpp
// src/Client/ClientBase.cpp:2060
bool ClientBase::receiveAndProcessPacket(ASTPtr parsed_query, bool cancelled_)
{
    Packet packet = connection->receivePacket();

    switch (packet.type)
    {
        case Protocol::Server::Data:
            if (!cancelled_)
                onData(packet.block, parsed_query);
            return true;

        case Protocol::Server::Progress:
            onProgress(packet.progress);
            return true;

Data, Totals and Extremes are dropped when cancelled_ is true; Progress, Log and ProfileEvents are still processed (the progress bar should keep updating until the server actually stops). The loop ends when the server sends EndOfStream:

cpp
// src/Client/ClientBase.cpp:2097
        case Protocol::Server::EndOfStream:
            onEndOfStream();
            return false;

and onEndOfStream is where the client explicitly declares the session clean:

cpp
// src/Client/ClientBase.cpp:2149
void ClientBase::onEndOfStream()
{
    /// `EndOfStream` terminates the query exchange with the protocol in sync, whatever happens
    /// below or afterwards. This also covers the failures that are recovered by draining the
    /// exchange: a `LocalFormatError` in the middle of a result (`receiveResult` sends `Cancel`
    /// and keeps receiving until the end of the stream) and a client-side failure while sending
    /// the data of an `INSERT` (`processInsertQuery` sends `Cancel` and calls
    /// `receiveEndOfQueryForInsert`). Such a session must not resynchronize the connection with
    /// a round trip: a `Pong` that does not arrive within `sync_request_timeout` is
    /// indistinguishable from a closed connection, and the client would silently reconnect,
    /// losing its temporary tables, current database and session settings.
    connection_needs_resynchronization = false;

This is the payoff of the whole design. Because the client drained the exchange, it knows the protocol is in sync, and it therefore does not have to resynchronize — which would cost a round trip and, if the Pong did not come back, would silently reconnect and lose the session. Cancelling a query costs the user nothing but the query.

One Ctrl-C or two: the partial-result trick

The counter is armed with a number that depends on a setting:

cpp
// src/Client/ClientBase.cpp:1849
    const auto & settings = client_context->getSettingsRef();
    const Int32 signals_before_stop = settings[Setting::partial_result_on_first_cancel] ? 2 : 1;

and it is armed (and disarmed) around the query itself:

cpp
// src/Client/ClientBase.cpp:1864
        try
        {
            query_interrupt_handler.start(signals_before_stop);
            SCOPE_EXIT({ query_interrupt_handler.stop(); });

partial_result_on_first_cancel defaults to false:

cpp
// src/Core/Settings.cpp:3311
    DECLARE(Bool, partial_result_on_first_cancel, false, R"(
Allows query to return a partial result after cancel.
)", 0) \

so by default signals_before_stop is 1 and the counter starts at 1. Now look closely at tryStop() again:

cpp
// src/Client/ClientBase.h:351
        bool tryStop() { return exit_after_signals.fetch_sub(1) <= 0; }

fetch_sub returns the value before the subtraction. With the counter at 1, the first Ctrl-C makes fetch_sub(1) return 1, and 1 <= 0 is false — so tryStop() returns false and the process does not exit. But the counter is now 0, so cancelled() is true and the query gets cancelled.

In other words: start(1) means “one interrupt cancels the query, a second one kills the client”. The first Ctrl-C is consumed by the query, not by the process. Only when the query is over does the SCOPE_EXIT reset the counter to 0, at which point the next Ctrl-C’s fetch_sub returns 0, 0 <= 0 is true, and the process exits with 128 + SIGINT.

That is the design in one sentence: the counter is shared between “stop the query” and “stop the process”, and the query gets the first interrupt.

The two-stage cancel works the same way. With partial_result_on_first_cancel enabled, the counter is armed at 2 and the receive loop distinguishes the two interrupts by their residual count:

cpp
// src/Client/ClientBase.cpp:1979
                if (partial_result_on_first_cancel && query_interrupt_handler.cancelled_status() == signals_before_stop - 1)
                {
                    sendCancel();
                    /// First cancel reading request was sent. Next requests will only be with a full cancel
                    partial_result_on_first_cancel = false;
                }
                else if (query_interrupt_handler.cancelled())
                {
                    cancelQuery();
                }

After the first Ctrl-C the residual count is 1, which equals signals_before_stop - 1 (2 - 1), so the client sends a soft cancel. After the second Ctrl-C the residual count is 0, cancelled() is true, and the client sends the hard cancel.

The server is what makes the two cancels different. The first one is interpreted as “stop working, but send me what you have”:

cpp
// src/Server/TCPHandler.cpp:3155
void TCPHandler::processCancel(QueryState & state)
{
    if (state.allow_partial_result_on_first_cancel && !state.stop_read_return_partial_result)
    {
        state.stop_read_return_partial_result = true;
        LOG_INFO(log, "Received 'Cancel' packet from the client, returning partial result.");
            return;
    }

    state.read_all_data = true;
    state.stop_query = true;

    throw Exception(ErrorCodes::QUERY_WAS_CANCELLED_BY_CLIENT, "Received 'Cancel' packet from the client, canceling the query.");
}

The first Cancel flips a flag and returns; the second one sets stop_query and throws. The client’s choice of which cancel to send is therefore a two-way handshake encoded in a counter.

The local connection

Not every client talks to a server over TCP. clickhouse-local runs the server in-process, and the “cancel” is a different mechanism with the same interface:

cpp
// src/Client/LocalConnection.cpp:113
LocalConnection::~LocalConnection()
{
    /// Last query may have not been finished or cancelled due to exception on client side.
    if (state && !state->is_finished && !state->is_cancelled)
    {
        try
        {
            LocalConnection::sendCancel();
        }
        catch (...) // NOLINT(bugprone-empty-catch)
        {
            /// Just ignore any exception.
        }
    }
    state.reset();
}

Note the destructor: even if the client is destroyed mid-query, it makes a best-effort attempt to cancel the query it abandoned.

Trade-offs

What the design buys.

The session survives. No reconnect, no lost temporary tables, no lost SET values, no lost current database. Cancelling a query in the middle of a session is free.
The latency is bounded. The client checks the flag at least every 5 ms, and the server checks the socket at least every interactive_delay (100 ms by default). The end-to-end cancellation latency is therefore tens of milliseconds, not “whenever the result set finishes”.
One mechanism, four uses. User interrupt, receive timeout, broken output pipe, and client-side failure during an INSERT all funnel into sendCancel + drain. There is exactly one place where the “am I still in sync?” question is answered.
The signal handler is trivially safe. It touches one atomic integer. There is no write() in a signal handler, no allocation, and no lock.

What it costs.

The connection must be drained. After a cancel, the client keeps reading until EndOfStream. If the server is in the middle of sending a 10 GB result, the client still reads (and discards) all of it. The cancellation saves the server’s compute, not the server’s output bandwidth. On a fast local socket with a huge result, cancelling can feel like it takes a while.
The poll interval is a trade-off. 5 ms of polling is a wake-up every 5 ms for the life of the query; 1 s would make Ctrl-C feel sluggish. The client picks min(receive_timeout, 1 s) and floors it at 5 ms.
The two-stage cancel is stateful and subtle. The residual counter has to mean “one more interrupt” in one place and “the last interrupt” in another, and the server has to remember whether it has already sent a partial result. A mistake in either direction shows up as “the first Ctrl-C returned partial results when I did not ask for that” or “the second Ctrl-C did nothing”.
The exit code is confusing. Because tryStop() decrements before comparing, a counter armed at 1 means “one Ctrl-C cancels the query, two kill the client”. Users expect Ctrl-C to exit the process, and it does — but only on the second press, and the first press is now a “cancel the query” instead. This is a behaviour change from the mysql client, where Ctrl-C always kills the process.
The terminal must be in raw mode for the byte path. The keystroke interceptor only exists when the progress table is enabled and the client is interactive, and the 0x03 callback is registered only in the embedded client; in every other configuration Ctrl-C is a SIGINT and the signal path is used.

The one thing to remember: the client is not “killing” the query, it is asking the server to stop, over the same connection, in one byte, and then finishing the conversation politely. That is why the connection survives.

The part people get wrong: cancelling does not free the server’s output bandwidth. The client drains the rest of the result stream, it just does not print it. If you are cancelling a query that is streaming hundreds of gigabytes to the client, the server will keep writing until the client has read it all.

What to go do about it: try SETTINGS partial_result_on_first_cancel = 1 on a long query and press Ctrl-C once. You get the rows that had already been computed; press it again and the query is killed outright.