The idea: a few calls on a socket
A network program does not build packets. It asks the kernel for a socket,
gets back a small number (a file descriptor, fd 3 for the first one), and then
calls a handful of functions on it. In Python they are methods of socket.socket,
and each one is a thin wrapper around the system call of the same name. Which calls you make, and
in which order, depends on two things: UDP or TCP, and server or client.
The canvas runs three real programs, one line per button press: server.py on the
right and the same client.py on two client hosts on the left (client A sends
b'hello', client B sends b'bye'). The server answers with
data.upper(). The bar on a program shows the next line to run; it turns orange
when the program is asleep inside a call. Under each program is its host's kernel socket table.
Both programs start with from socket import *, which is not drawn.
UDP: bind, recvfrom, sendto
s = socket.socket(AF_INET, SOCK_DGRAM)
s.bind(('0.0.0.0', 9000))
while True:
data, addr = s.recvfrom(1024)
s.sendto(data.upper(), addr)
AF_INET means IPv4 addresses and SOCK_DGRAM means datagrams, which is
UDP. bind() claims port 9000 on every local address ('0.0.0.0'). From then
on the kernel keeps every datagram for that port in the socket's queue, even while the program is
busy. recvfrom(1024) takes one whole datagram from the queue and also returns who sent
it, for example ('10.0.0.1', 50001). The server answers with
sendto(reply, addr). There is no connection: one socket talks to every client, and the
address goes with each datagram (Demo: UDP echo, one datagram each way).
The client never calls bind(). Its first sendto() makes the kernel pick a
free ephemeral port (50001 here; Linux picks from
net.ipv4.ip_local_port_range, 32768–60999). In Demo: UDP, two clients share one
server socket both datagrams wait in the same queue, and the server answers each one at the
address recvfrom() gave it.
TCP: listen, accept, connect
ls = socket.socket(AF_INET, SOCK_STREAM)
ls.bind(('0.0.0.0', 9000))
ls.listen(5)
while True:
conn, addr = ls.accept()
while data := conn.recv(1024):
conn.sendall(data.upper())
conn.close()
SOCK_STREAM is TCP. listen(5) turns the socket into a
listening socket: it never carries data, it only collects finished connections
(at most 5 waiting). The client's connect(('10.0.0.2', 9000)) picks an ephemeral port
and makes the kernels run the three-way handshake (see
TCP Handshake: States and the Two Listen Queues). The server's kernel creates a new socket for
the client during the handshake, before the program asks for it.
accept() takes that socket off the queue and returns it as a new socket
object, conn (fd 4), with the client's address. The program talks to this client
only through conn. ls stays open and keeps listening
(Demo: TCP echo, accept() returns a new socket). sendall() keeps sending until
every byte is handed to the kernel; plain send() may send only part.
The server on this page is iterative: it serves one client to the end before it calls
accept() again. In Demo: TCP, two clients get two sockets, client B connects while A
is being served. B's handshake still completes (the kernel does it), and B's b'bye' even lands
in B's socket before the server has accepted it. When the server comes back to accept() it gets
B at once, as fd 4 again: a closed fd number is free, and the kernel always hands out the lowest free one.
A real server serves clients at the same time with threads, processes or
epoll.
Blocking calls, and b'' at the end
recvfrom(), accept() and recv() block: if the kernel has nothing
for them, the program sleeps on that line. It wakes when a datagram arrives, a handshake finishes, or bytes
or a FIN arrive. Data that arrives earlier simply waits in the socket, so the order in which you press the
buttons changes when a call returns, never what it returns.
| call | returns when | returns |
|---|---|---|
s.recvfrom(1024) | a datagram is in the queue | (data, addr), one whole datagram |
ls.accept() | a connection is in the accept queue | (conn, addr), a new socket |
conn.recv(1024) | bytes are in the buffer, or the peer closed | up to 1024 bytes, or b'' |
c.connect(addr) | the handshake is done, or RST came back | None, or raises |
When the client calls close(), its kernel sends FIN. The server's recv() then
returns b'', an empty bytes object. It is false, so while data := conn.recv(1024)
stops, and the server closes its side too. An empty recv() always means "the other side
is done"; it never means "no data yet". The close states (CLOSE_WAIT, FIN_WAIT_2, TIME_WAIT) are explained
on A TCP Socket in Linux.
TCP is a byte stream: one recv() can return the bytes of two sendall() calls, or
half of one. Real protocols mark where a message ends (see TCP vs UDP).
Port demultiplexing
When a packet arrives, the kernel must pick the socket it belongs to. This is demultiplexing, and the canvas shows the key it uses under the server's sockets.
| lookup key | so | |
|---|---|---|
| UDP | (dst IP, dst port) | datagrams from every client to port 9000 go to the one bound socket |
| TCP, connected | (src IP, src port, dst IP, dst port) | each client's segments go to that client's own socket |
| TCP, new SYN | (dst IP, dst port) | the listening socket takes it and makes a new socket |
The protocol is part of the key too: UDP port 9000 and TCP port 9000 are different ports, and both
servers could run at once. Two TCP servers cannot bind() the same port: the second gets
OSError: [Errno 98] Address already in use.
When nothing listens
In Demo: UDP, no server the server host has no socket on UDP port 9000. Its kernel drops the
datagram and sends back ICMP "port unreachable". Linux does not report that error to an unconnected UDP
socket, so the client's recvfrom() would wait forever. That is why the client calls
settimeout(2.0): after 2 seconds recvfrom() raises
TimeoutError: timed out (socket.timeout before Python 3.10).
TCP finds out at once. A SYN to a port with no listening socket gets a RST, and connect()
raises ConnectionRefusedError: [Errno 111] Connection refused
(Demo: TCP, no server: connect() is refused).
Running it yourself
Save the programs as server.py and client.py, replace 10.0.0.2 with
127.0.0.1, and run them in two terminals. While the server runs, ss -uanp lists
UDP sockets and ss -tanp TCP sockets, with their addresses, states and owning process. Running
client.py twice shows two different ephemeral ports.
See also A TCP Socket in Linux (the kernel buffers behind these calls),
Linux Socket Backlog (what listen(5) limits) and
Unix Socket vs TCP.
What the page leaves out
TCP's data ACKs, sequence numbers and windows are not drawn, and every message fits in one packet.
Lost or reordered packets, IPv6 (AF_INET6), non-blocking sockets, select,
socketserver and asyncio, socket options such as SO_REUSEADDR,
shutdown(), and a connect()ed UDP socket (which does see ICMP errors) are left out.