If you're seeing this message, it means we're having trouble loading external resources on our website. Show
If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked. This page is generated by a web of HTML, CSS, and Javascript, sent to you by Codecademy via the internet. The internet is made up of a bunch of resources hosted on different servers. The term “resource” corresponds to any entity on the web, including HTML files, stylesheets, images, videos, and scripts. To access content on the internet, your browser must ask these servers for the resources it wants, and then display these resources to you. This protocol of requests and responses enables you to view this page in your browser. This article focuses on one fundamental part of how the internet functions: HTTP. HTTP stands for Hypertext Transfer Protocol and is used to structure requests and responses over the internet. HTTP requires data to be transferred from one point to another over the network. The transfer of resources happens using TCP (Transmission Control Protocol). In viewing this webpage, TCP manages the channels between your browser and the server (in this case, codecademy.com). TCP is used to manage many types of internet connections in which one computer or device wants to send something to another. HTTP is the command language that the devices on both sides of the connection must follow in order to communicate. When you type an address such as www.codecademy.com into your browser, you are commanding it to open a TCP channel to the server that responds to that URL (or Uniform Resource Locator, which you can read more about on Wikipedia). A URL is like your home address or phone number because it describes how to reach you. In this situation, your computer, which is making the request, is called the client. The URL you are requesting is the address that belongs to the server. Once the TCP connection is established, the client sends a HTTP GET request to the server to retrieve the webpage it should display. After the server has sent the response, it closes the TCP connection. If you open the website in your browser again, or if your browser automatically requests something from the server, a new connection is opened which follows the same process described above. GET requests are one kind of HTTP method a client can call. You can learn more about the other common ones (POST, PUT and DELETE) in this article. Let’s explore an example of how GET requests (the most common type of request) are used to help your computer (the client) access resources on the web. Suppose you want to check out the latest course offerings from http://codecademy.com. After you type the URL into your browser, your browser will extract the http part and recognize that it is the name of the network protocol to use. Then, it takes the domain name from the URL, in this case “codecademy.com”, and asks the internet Domain Name Server to return an Internet Protocol (IP) address. Now the client knows the destination’s IP address. It then opens a connection to the server at that address, using the http protocol as specified. It will initiate a GET request to the server which contains the IP address of the host and optionally a data payload. The GET request contains the following text: GET / HTTP/1.1 Host: www.codecademy.comThis identifies the type of request, the path on www.codecademy.com (in this case, “/“) and the protocol “HTTP/1.1.” HTTP/1.1 is a revision of the first HTTP, which is now called HTTP/1.0. In HTTP/1.0, every resource request requires a separate connection to the server. HTTP/1.1 uses one connection more than once, so that additional content (like images or stylesheets) is retrieved even after the page has been retrieved. As a result, requests using HTTP/1.1 have less delay than those using HTTP/1.0. The second line of the request contains the address of the server which is "www.codecademy.com". There may be additional lines as well depending on what data your browser chooses to send. If the server is able to locate the path requested, the server might respond with the header: HTTP/1.1 200 OK Content-Type: text/htmlThis header is followed by the content requested, which in this case is the information needed to render www.codecademy.com. The first line of the header, HTTP/1.1 200 OK, is confirmation that the server understands the protocol that the client wants to communicate with (HTTP/1.1), and an HTTP status code signifying that the resource was found on the server. The third line, Content-Type: text/html, shows the type of content that it will be sending to the client. If the server is not able to locate the path requested by the client, it will respond with the header: HTTP/1.1 404 NOT FOUNDIn this case, the server identifies that it understands the HTTP protocol, but the 404 NOT FOUND status code signifies that the specific piece of content requested was not found. This might happen if the content was moved or if you typed in the URL path incorrectly or if the page was removed. You can read more about the 404 status code, commonly called a 404 error, here. It can be tricky to understand how HTTP functions because it’s difficult to examine what your browser is actually doing. (And perhaps also because we explained it using acronyms that may be new to you.) Let’s review what we learned by using an analogy that could be more familiar to you. Imagine the internet is a town. You are a client and your address determines where you can be reached. Businesses in town, such as Codecademy.com, serve requests that are sent to them. The other houses are filled with other clients like you that are making requests and expecting responses from these businesses in town. This town also has a crazy fast mail service, an army of mail delivery staff that can travel on trains that move at the speed of light. Suppose you want to read the morning newspaper. In order to retrieve it, you write down what you need in a language called HTTP and ask your local mail delivery staff agent to retrieve it from a specific business. The mail delivery person agrees and builds a railroad track (connection) between your house and the business nearly instantly, and rides the train car labeled “TCP” to the address of the business you provided. Upon arriving at the business, she asks the first of several free employees ready to fulfill the request. The employee searches for the page of the newspaper that you requested but cannot find it and communicates that back to the mail delivery person. The mail delivery person returns on the light speed train, ripping up the tracks on the way back, and tells you that there was a problem “404 Not Found.” After you check the spelling of what you had written, you realize that you misspelled the newspaper title. You correct it and provide the corrected title to the mail delivery person. This time the mail delivery person is able to retrieve it from the business. You can now read your newspaper in peace until you decide you want to read the next page, at which point, you would make another request and give it to the mail delivery person. Since your HTTP request can be read by anyone at certain network junctures, it might not be a good idea to deliver information such as your credit card or password using this protocol. Fortunately, many servers support HTTPS, short for HTTP Secure, which allows you to encrypt data that you send and receive. You can read more about HTTPS on Wikipedia. HTTPS is important to use when passing sensitive or personal information to and from websites. However, it is up to the businesses maintaining the servers to set it up. In order to support HTTPS, the business must apply for a certificate from a Certificate Authority.
Every day you open up your browser and navigate to your favorite websites — whether it be social media, news, or e-commerce sites. You go to this page by typing in a url or clicking on a link to the page. Have you ever thought about what happens behind the scenes? How does the news get to you when you press enter after typing in the URL? How did the images on this post show up in your browser? How does your Twitter feed and the tweet data show up in your browser securely? In this post, we’ll look at what happens when you type a URL into your browser and press enter. End to end, the process involves the browser, your computer’s operating system, your internet service provider, the server where you host the site, and services running on that server. It’s important to understand where things can go wrong, where to look for performance issues, and ensure you’re offering a secure experience for your users. First, we’ll take a look at the relationship between websites, servers, and IP addresses. Then, we’ll go through the steps your browser takes to:
Websites, servers, IP addresses, oh my!Websites are collections of files, often HTML, CSS, Javascript, and images, that tell your browser how to display the site, images, and data. They need to be accessible to anyone from anywhere at anytime, so hosting them on your computer at home isn’t be scalable or reliable. A powerful external computer connected to the Internet, called a server, stores these files. When you point your browser at a URL like https://jennapederson.dev/hello-world, your browser has to figure out which server on the Internet is hosting the site. It does this by looking up the domain, jennapederson.dev, to find the address. Each device on the Internet — servers, cell phones, your smart refrigerator — all have a unique address called an IP address. An IP address contains four numbered parts: 203.0.113.0 But numbers like this are hard to remember! That’s where domain names come in. jennapederson.dev is much easier to remember than 203.0.113.0, right? Imagine having to remember all the phone numbers of your contacts without having the Contacts app on your phone. Your Contacts app lets you look up phone numbers by name. We do the same on the Internet. The domain name system, or DNS, is like the Contacts app on our phone. DNS helps our browser (and us) find servers on the Internet. We can do a DNS lookup to find the IP address of the server based on the domain name, jennapederson.dev. You can read more about DNS here. Now that you know about the different parts and how they relate to one another, let’s look at each step of the process and how the browser communicates with the server when you type in a URL. Whether you typed in a URL or clicked on a linked URL from the current page, the process is the same. The processTo show how all these steps fit together, I’m going to use an Amazon Lightsail instance and a Lightsail DNS zone. I’m using Lightsail because it’s one of the simplest services to manage virtual private servers (VPS) and DNS in one place, but these concepts work for any VPS and DNS service. 1. You type https://jennapederson.dev/hello-world in your browser and press EnterLet’s break down the parts of this URL you typed in to get here. https://jennapederson.dev/hello-world Schemehttps:// is the scheme. HTTPS stands for Hypertext Transfer Protocol Secure. This scheme tells the browser to make a connection to the server using Transport Layer Security, or TLS. TLS is an encryption protocol to secure communications over the Internet. With HTTPS, the data exchanged between your browser and the server, like passwords or credit card info, is encrypted. You may have also seen ftp://, mailto://, or file://. These are other protocols that browsers know how to handle. Domainjennapederson.dev is the domain name of the site. It is the memorable address and points to a specific server’s IP address. If you look at the Lightsail DNS zone below, you can see a DNS A record pointing to the Lightsail instance, jennapedersondev-static-ip, which represents the static IP address of the Lightsail instance.  PathSometimes there is an additional path to the resource in the URL. For example, for this URL, https://jennapederson.dev/the-path-to/hello-world, the-path-to is the path on the server to the requested resource, hello-world. You can think of this like the directory structure of files and other directories on your computer. It’s a way to organize your resources, whether they are static HTML, CSS, Javascript, or image files, or dynamically generated content. Common examples of paths are blog, posts, tags, or images, each grouping different resources. ResourceWhen you typed this URL into your browser, hello-world is the name of the resource on the website you want to view. Sometimes you’ll see this with a file extension like .html which indicates this is a static file on the server with HTML content. Without an extension, like this URL, it usually indicates the server generated this content. For instance, a news site would show you customized, up to date, and local news content, which it can only do when it knows who you are or where the request came from. 2. Browser looks up IP address for the domainAfter you’ve typed the URL into your browser and pressed enter, the browser needs to figure out which server on the Internet to connect to. To do that, it needs to look up the IP address of the server hosting the website using the domain you typed in. It does this using a DNS lookup. We’ll go over the DNS lookup process at a high-level, but it is a complex topic beyond the scope of this post. You can read more about how DNS works here. Because DNS is complex and has to be blazingly fast, DNS data is cached at different layers between your browser and at various places across the Internet. Your browser checks its own cache, the operating system cache, a local network cache at your router, and a DNS server cache on your corporate network or at your internet service provider (ISP). If the browser cannot find the IP address at any of those cache layers, the DNS server on your corporate network or at your ISP does a recursive DNS lookup. A recursive DNS lookup asks multiple DNS servers around the Internet, which in turn ask more DNS servers for the DNS record until it is found. Once the browser gets the DNS record with the IP address, it’s time for it to find the server on the Internet and establish a connection. 3. Browser initiates TCP connection with the serverUsing the public Internet routing infrastructure, packets from a client browser request get routed through the router, the ISP, through an internet exchange to switch ISPs or networks, all using transmission control protocol, more commonly known as TCP, to find the server with the IP address to connect to. But this is a very roundabout way to get there and it’s not efficient. Instead, many sites use a content delivery network, or CDN, to cache static and dynamic content closer to the browser. In our example, I’ve set the Lightsail instance, jennapedersondev, as an origin for a CDN distribution. A CDN is a globally distributed network of caching servers that improve the performance of your site or app (the origin) by bringing the content closer to your users. The Lightsail CDN uses CloudFront, which is part of the AWS global network. Requests from the client browser get to take advantage of this private network that has ultra-low latency and high availability. To track the hops the request takes from my computer to jennapederson.dev, we can use traceroute. In the image below, the first hop (the first row) is to my router. Hops in box one are on my ISP’s network and hops in box two are on the AWS global network. Instead of relying on the public internet routing infrastructure and being subject to extra hops, redeliveries, and packet loss, the client browser request gets to take a ride on the AWS global network. The request is intelligently routed through the most performant location to deliver content to your browser. Once the browser finds the server on the Internet, it establishes a TCP connection with the server and if HTTPS is being used, a TLS handshake takes place to secure the communication. TCP and and TLS are extremely important topics, but we’ll cover them in another post. In image the below, this connection (Initial connection) took 130.30ms. Once the browser has established a connection with the server, the next step is to send the HTTP request to get the resource, or the page. 4. Browser sends the HTTP request to the serverNow that the browser has a connection to the server, it follows the rules of communication for the HTTP(s) protocol. It starts with the browser sending an HTTP request to the server to request the contents of the page. The HTTP request contains a request line, headers (or metadata about the request), and a body. The request line contains information that the server can use to determine what the client (in this case, your browser) wants to do. The request line contains the following:
The request line for the URL request looks like this: GET /hello-world HTTP/1.1 The request line tells the server that you want to GET resource at /hello-world and to communicate with HTTP/1.1. Remember that HTTP verbs express the semantic intent of your request. You could also use the POST, PUT, or PATCH methods to send data to the server for storage, either to create new data or update existing data at the request path. The DELETE method would delete the resource at the given path. However, it’s important to know that servers don’t have to support all verbs. A server could respond with a 200 OK status to a DELETE method and not do anything with the resource. The next part of the request is the request headers. Headers pass extra information along from the client that help route the request, indicate what type of client is making the request (the user agent), and can be used for supporting A/B testing, blue/green deployments, and canary deployments. Headers are key-value pairs like this: The last part of the request is the body. The body is (usually) empty for a GET request like ours. For a request that manipulates resources, like POST, PUT, or PATCH, the body will contain the data from the client to create or update. Once the server has received the request from the client, the server processes it and sends back a response. 5. Server processes request and sends back a responseThe server takes the request and based on the info in the request line, headers, and body, decides how to process the request. For the request, GET /hello-world/ HTTP/1.1, the server gets the content at this path, constructs the response and sends it back to the client. The response contains the following:
The status line contains both the HTTP version and a numeric and text representation of the status of the request. The response looks like this: HTTP/1.1 200 OK Date: Tue, 25 May 2021 19:40:59 GMT Server: Apache X-Frame-Options: SAMEORIGIN X-Powered-By: Express Cache-Control: max-age=0, no-cache Content-Type: text/html; charset=utf-8 Vary: Accept-Encoding X-Mod-Pagespeed: 1.13.35.2-0 Content-Encoding: br Keep-Alive: timeout=1, max=100 Connection: Keep-Alive Transfer-Encoding: chunked <!DOCTYPE html> <html lang="en"> <head> THE REST OF THE HTMLThe browser considers a status code in the 200s to be successful. The response was 200, so the browser knows to render the response. 6. Browser renders the contentOnce the browser has received the response from the server, it inspects the response headers for information on how to render the resource. The Content-Type header above tells the browser it received an HTML resource in the response body. Lucky for us, the browser knows what to do with HTML! The first GET request returns HTML, the structure of the page. But if you inspect the HTML of the page (or any web page) with your browser’s dev tools, you’ll see it references other Javascript, CSS, image resources and requests additional data in order to render the web page as designed. As the browser is parsing and rendering the HTML, it is making additional requests to get Javascript, CSS, images, and data. It can do much of this in parallel, but not always and that’s a story for a different post. In the image above, you can see the HTML references a CSS resource. The browser makes a subsequent request to the server to get this CSS resource to style the page. The Content-Type header of the request for the CSS resource tells the browser to render CSS. If the browser requests an image resource, the Content-Type header tells the browser it is non-text data and to render it accordingly. Wrapping UpYou did it! You traced a URL request from the browser all the way to the server hosting it and it’s response back to the browser to be rendered. We covered the relationship between websites, servers, IP addresses and stepped through each of the steps that your browser goes through when you type a URL into your browser and press enter. For review, here are those six steps:
Knowing what happens when you type a URL into your browser can help you figure out where things go wrong, where to look for performance issues with your website, and to offer a secure experience for your users. If you’d like to try this out for yourself, you can build your own virtual private server, setup a CDN, and manage domains with Amazon Lightsail. Get started now with this tutorial and check out the latest pricing promotion so you can get started even quicker. |
What exactly is the request asking for what does your browser expect to receive as a response?
Related Posts
Copyright © 2024 SignalDuo Inc.
