Socket Programming

Network protocols

TCP/IP is a suite of protocols used by devices to communicate over the Internet and most local networks. TCP is more reliable, has extensive error checking, and requires more resources. It is used by services such as HTTP, SMTP, or FTP. UDP is much less reliable, has limited error checking, and requires less resources. It is used by services such as VoIP.

The socket.SOCK_STREAM is used to create a socket for TCP The socket.SOCK_DGRAM is used to create a socket for UDP.

Address families

When we create a socket, we have to specify its address family. Then we can only use addresses of that type with the socket.

• AF_UNIX, AF_LOCAL - Local communication

• AF_INET - IPv4 Internet protocols

• AF_INET6 - IPv6 Internet protocols

• AF_IPX - IPX - Novell protocols

• AF_BLUETOOTH - Wireless bluetooth protocols

• AF_PACKET - Low level packet interface

For the AF_INET address family, a pair (host, port) is specified. The host is a string representing either a hostname in Internet domain notation like google.com or an IPv4 address like 93.184.216.34, and port is an integer.

Get IP address

With gethostbyname, we get the IP address of the host.

#!/usr/bin/python3

import socket

ip = socket.gethostbyname('google.com')
print(ip)


Output:
172.217.21.14

UDP socket example

UDP is a communication protocol that transmits independent packets over the network with no guarantee of arrival and no guarantee of the order of delivery. One service that used UDP is the Quote of the Day (QOTD).

qotd_client.py

#!/usr/bin/env python

import socket

with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:

    message = b''
    addr = ("djxmmx.net", 17)

    s.sendto(message, addr)

    data, address = s.recvfrom(1024)
    print(data.decode())


Output:
"When a stupid man is doing something he is ashamed of, he always declares
 that it is his duty." George Bernard Shaw (1856-1950)

Explaining

print(data.decode())

UDP sockets use recvfrom to receive data. Its paremeter is the buffer size. The return value is a pair (data, address) where data is a byte string representing the data received and address is the address of the socket sending the data.

data, address = s.recvfrom(1024)

We send data with the sendto method.

s.sendto(message, addr)

We provide the address and the port.

addr = ("djxmmx.net", 17)

We send an empty message; the QOTD service works by sending arbitrary data to the socket; it simply responds with a quote. To communicate over TCP/UDP, we use binary strings.

message = b''

A datagram socket for IPv4 is created.

with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:

We import the socket module.

import socket

The example creates a client program that connects to a QOTD service.

TCP socket example

The are servers that provide current time. A client simply connects to the server with no commands, and the server responds with a current time.

#!/usr/bin/env python

import socket

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:

    host = "time.nist.gov"
    port = 13

    s.connect((host, port))
    s.sendall(b'')
    print(str(s.recv(4096), 'utf-8'))
    
    
    Output:
    59638 22-02-28 04:16:20 00 0 0 125.0 UTC(NIST) * 

The example determines the current time by connecting to a time server's TCP socket.

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:

A TCP socket for IPv4 is created.

host = "time.nist.gov"
port = 13

This is the host name and the port number of a working time server.

s.connect((host, port))

We connect to the remote socket with connect.

s.sendall(b'')

The sendall method sends data to the socket. The socket must be connected to a remote socket. It continues to send data from bytes until either all data has been sent or an error occurs.

print(str(s.recv(4096), 'utf-8'))

We print the received data. The recv method receives up to buffersize bytes from the socket. When no data is available, it blocks until at least one byte is available or until the remote end is closed. When the remote end is closed and all data is read, it returns an empty byte string.

Socket HEAD request

A HEAD request is a GET request without a message body. The header of a request/response contains metadata, such as HTTP protocol version or content type.

#!/usr/bin/python3

import socket

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:

    s.connect(("webcode.me" , 80))
    s.sendall(b"HEAD / HTTP/1.1\r\nHost: webcode.me\r\nAccept: text/html\r\n\r\n")
    print(str(s.recv(1024), 'utf-8'))
    
    
Output:

HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Mon, 28 Feb 2022 04:19:33 GMT
Content-Type: text/html
Content-Length: 394
Last-Modified: Sun, 23 Jan 2022 10:39:25 GMT
Connection: keep-alive
ETag: "61ed305d-18a"
Accept-Ranges: bytes

In the example, we send a HEAD request to webcode.me.

s.sendall(b"HEAD / HTTP/1.1\r\nHost: webcode.me\r\nAccept: text/html\r\n\r\n")

A head request is issued with the HEAD command followed by the resource URL and HTTP protocol version. Note that the \r are mandatory part of the communication process. The details are described in RFC 7231 document.

Socket GET request

The HTTP GET method requests a representation of the specified resource. Requests using GET should only retrieve data.

#!/usr/bin/python3

import socket

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:

    s.connect(("webcode.me" , 80))
    s.sendall(b"GET / HTTP/1.1\r\nHost: webcode.me\r\nAccept: text/html\r\nConnection: close\r\n\r\n")

    while True:

        data = s.recv(1024)

        if not data:
            break

        print(data.decode())
        
        
Output:

HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Mon, 28 Feb 2022 04:20:30 GMT
Content-Type: text/html
Content-Length: 394
Last-Modified: Sun, 23 Jan 2022 10:39:25 GMT
Connection: close
ETag: "61ed305d-18a"
Access-Control-Allow-Origin: *
Accept-Ranges: bytes

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="format.css">
    <title>My html page</title>
</head>
<body>

    <p>
        Today is a beautiful day. We go swimming and fishing.
    </p>
    
    <p>
         Hello there. How are you?
    </p>
    
</body>
</html>

The example reads the home page of the webcode.me using a GET request.

s.sendall(b"GET / HTTP/1.1\r\nHost: webcode.me\r\nAccept: text/html\r\nConnection: close\r\n\r\n")

For the HTTP 1.1 protocol, the connections may be persistent by default. This is why we send the Connection: close header.

while True:

    data = s.recv(1024)

    if not data:
        break

    print(data.decode())

We use a while loop to process the received data. If no error occurs, recv returns the bytes received. If the connection has been gracefully closed, the return value is an empty byte string. The recv is a blocking method that blocks until it is done, or a timeout is reached or another exception occurs.

$ ./get_request.py
HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Sun, 08 Sep 2019 11:39:34 GMT
Content-Type: text/html
Content-Length: 348
Last-Modified: Sat, 20 Jul 2019 11:49:25 GMT
Connection: keep-alive
ETag: "5d32ffc5-15c"
Access-Control-Allow-Origin: *
Accept-Ranges: bytes

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>My html page</title>
</head>
<body>

    <p>
        Today is a beautiful day. We go swimming and fishing.
    </p>

    <p>
         Hello there. How are you?
    </p>

</body>
</html>

Echo client server example

An echo server sends the message from the client back. It is a classic example used for testing and learning.

#!/usr/bin/python3
# SERVER

import socket
import time

with socket.socket() as s:

    host = 'localhost'
    port = 8001

    s.bind((host, port))
    print(f'socket binded to {port}')

    s.listen()

    con, addr = s.accept()

    with con:
   
        while True:

            data = con.recv(1024)

            if not data:
                break

            con.sendall(data)

The echo server sends the client message back to the client.

host = 'localhost'
port = 8001

The server runs on localhost on port 8001.

s.bind((host, port))

The bind method establishes the communication endpoint. It binds the socket to the specified address. The socket must not already be bound. (The format of address depends on the address family.)

s.listen()

The listen method enables a server to accept connections. The server can now listen for connections on a socket. The listen has a backlog parameter. It specifies the number of unaccepted connections that the system will allow before refusing new connections. The parameter is optional since Python 3.5. If not specified, a default backlog value is chosen.

con, addr = s.accept()

With accept, the server accepts a connection. It blocks and waits for an incoming connection. The socket must be bound to an address and listening for connections. The return value is a pair (con, addr) where con is a new socket object usable to send and receive data on the connection, and addr is the address bound to the socket on the other end of the connection.

Note that the accept creates a new socket for communication with a client, which is a different socket from the listening socket.

#!/usr/bin/python3
# CLIENT

import socket

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:

    host = "localhost"
    port = 8001

    s.connect((host, port))
    s.sendall(b'hello there')
    print(str(s.recv(4096), 'utf-8'))

The client sends a message to the echo server.

Asynchronous server example

In order to improve the performance of a server, we can use the asyncio module.

#!/usr/bin/python3

# from threading import current_thread

import asyncio


async def handle_client(reader, writer):

    data = (await reader.read(1024))

    writer.write(data)
    writer.close()


loop = asyncio.get_event_loop()
loop.create_task(asyncio.start_server(handle_client, 'localhost', 8001))
loop.run_forever()

We can now test the performance of the blocking and non-blocking servers.

$ ab -c 50 -n 1000 http://localhost:8001/

For instance, we can test the performance with the Apache benchmarking tool. In our case, the command sends 1000 requests, 50 at a time.

In this tutorial, we have showed how to create simple networking programs with sockets in Python.