What are the I2C bus speed modes, and what limits how fast you can actually run the bus?

The spec defines a ladder of modes: Standard-mode at 100 kHz, Fast-mode at 400 kHz, Fast-mode Plus at 1 MHz, and High-speed mode at 3.4 MHz (there's also Ultra Fast-mode at 5 MHz, but it's a push-pull, write-only variant that drops ACK and multi-master arbitration, so it's not really the same bus). The modes are backward compatible in the sense that a faster master can talk to slower devices by clocking slower, and the practical rule is that the bus runs at the speed of its slowest participant, so one 100 kHz sensor caps the whole bus at 100 kHz unless you segment it with a mux or a second bus. What actually limits speed in hardware is the RC time constant formed by the pull-up resistance and total bus capacitance (traces, pads, and every device on the line), because open-drain lines are pulled low actively but rise passively. The spec bounds this by tightening the maximum rise time as you go up: roughly 1000 ns at Standard, 300 ns at Fast, 120 ns at Fast-mode Plus, against a 400 pF bus capacitance budget (550 pF for Fast+). So going faster means smaller pull-ups to speed up the edges, which raises the sink current every device must handle: Standard/Fast parts sink 3 mA, while Fast-mode Plus requires 20 mA drive, which is exactly why you can't just crank an existing 400 kHz design to 1 MHz without checking that every chip on it supports Fast+. High-speed mode is a different animal procedurally: the master starts in F/S mode, sends a reserved master code (00001xxx), which nobody ACKs, then issues a repeated START and switches into Hs, where it uses a current-source pull-up on SCL for fast rising edges, arbitration is disabled for the duration, and clock stretching is only allowed after the ACK bit. Hs is rarely implemented in practice; the realistic decision in most designs is 100 kHz versus 400 kHz, with Fast+ when both the MCU and every peripheral support it and the layout is short and low-capacitance.