Web servers

Web servers respond to Hypertext Transfer Protocol (HTTP) requests from clients and send back a response containing a status code and often content such as HTML, XML or JSON as well.

Why are web servers necessary?

Web servers are the ying to the web client's yang. The server and client speak the standardized language of the World Wide Web. This standard language is why an old Mozilla Netscape browser can still talk to a modern Apache or Nginx web server, even if it cannot properly render the page design like a modern web browser can.

The basic language of the Web with the request and response cycle from client to server then server back to client remains the same as it was when the Web was invented by Tim Berners-Lee at CERN in 1989. Modern browsers and web servers have simply extended the language of the Web to incorporate new standards.

Web server implementations

The conceptual web server idea can be implemented in various ways. The following web server implementations each have varying features, extensions and configurations.

The Apache HTTP Server has been the most commonly deployed web server on the Internet for 20+ years.
Nginx is the second most commonly used server for the top 100,000 websites and often serves as a reverse proxy for Python WSGI servers.
Caddy is a newcomer to the web server scene and is focused on serving the HTTP/2 protocol with HTTPS.
rwasa is a newer web server written in Assembly with no external dependencies that tuned to be faster than Nginx. The benchmarks are worth taking a look at to see if this server could fit your needs if you need the fastest performance trading off for as of yet untested web server.

Client requests

A client that sends a request to a web server is usually a browser such as Internet Explorer, Firefox, or Chrome, but it can also be a

headless browser, commonly use for testing, such as phantomjs
commandline utility, for example wget and cURL
text-based web browser such as Lynx
web crawler.

Web servers process requests from the above clients. The result of the web server's processing is a response code and commonly a content response. Some status codes, such as 204 (No content) and 403 (Forbidden), do not have content responses.

In a simple case, the client will request a static asset such as a picture or JavaScript file. The file sits on the file system in a location the web server is authorized to access and the web server sends the file to the client with a 200 status code. If the client already requested the file and the file has not changed, the web server will pass back a 304 "Not modified" response indicating the client already has the latest version of that file.

Web server and web browser request-response cycle

A web server sends files to a web browser based on the web browser's request. In the first request, the browser accessed the "www.fullstackpython.com" address and the server responded with the index.html HTML-formatted file. That HTML file contained references to other files, such as style.css and script.js that the browser then requested from the server.

Sending static assets (such as CSS and JavaScript files) can eat up a large amount of bandwidth which is why using a Content Delivery Network (CDN) to serve static assets is important when possible.

Building web servers

A Simple Web Server in less than 500 lines of code from the Architecture of Open Source book provides a great example with Python as the implementation language..
If you're looking to learn about web servers by building one, here's part one, part two and part three of a great tutorial that shows how to code a web server in Python.
Building a basic HTTP Server from scratch in Python (source code builds a very simple but insecure web server to show you how HTTP works.

Web server references

HTTP/1.1 and HTTP/2 specifications are the source for how web servers implement the modern web.
A reference with the full list of HTTP status codes is provided by W3C.
Usage of web servers broken down by ranking shows how popular Apache, Nginx and other websites are among the top million through the top 1,000 sites in the world.
Apache vs Nginx: practical considerations gives an an overview of each project, explains how each one handles connections, how it is configured and the differences between the two web servers in how it can use custom modules.
Optimizing web servers for high throughput and low latency is a wonderful read that shows how detailed knowledge at every layer of the stack is necessary to optimize web server connections at scale.
Implementing a tiny web server in a single printf call is an absurd C language hack that you would never want to use in any real project but still an amazing little application and wonderful explanation that you can learn a bit more about web servers by reading.
Top 5 open source web servers is a short overview of Apache, Nginx, Lighttpd and two programming language specific servers, Node.js for JavaScript and Tomcat for Java.

Web servers learning checklist

Choose a web server. Nginx is often recommended although Apache is also a great choice.
Create an SSL certificate via Let's Encrypt. You will need SSL for serving HTTPS traffic and preventing myriad security issues that occur with unencrypted user input.
Configure the web server to serve up static files such as CSS, JavaScript and images.
Once you set up the WSGI server you'll need to configure the web server as a pass through for dynamic content.