How does GDB physically reach into the MCU to halt it and read memory?

Through a chain that ends at dedicated debug hardware built into the silicon. GDB itself speaks the GDB remote protocol over a TCP socket to a gdbserver such as OpenOCD; OpenOCD communicates over USB to a debug probe (ST-Link, SEGGER J-Link, CMSIS-DAP, etc.); the probe drives the chip's JTAG or SWD pins; and those pins connect to the MCU's debug port (on ARM, the Debug Access Port / DAP) and its associated debug logic, which can directly halt and run the core, read and write CPU registers, and access any memory or peripheral over the internal bus. So when you type print x or x/4xw 0x40020000, GDB sends a memory-read request that OpenOCD turns into SWD/JTAG transactions, the DAP performs the bus access on-chip, and the value comes back up the chain. The same path sets hardware breakpoints (programmed into debug comparators in the silicon) and programs flash (load). A powerful consequence is that this works independently of any running firmware, the debug logic can halt and inspect even a crashed or bricked chip, which is why on-chip debugging is so valuable. The key setup pieces are the right probe and the correct OpenOCD interface/target configuration so the protocol and chip are addressed correctly.