Requests in Python is a great module that allows you to use Python to send HTTP\/1.1 requests to web pages.<\/p>\n
Python 2.7 and 3.5+ are both officially supported. Keep \u2013 Alive, Connection Pooling, Sessions with permanent cookies, and Browser Style SSL verification make it the preferred solution for developers.<\/p>\n
In this post, we’ll go over some of these features in greater detail and show you how to get started with the Python Requests module to construct web requests.<\/p>\n
Installing requests in Python is simple and straightforward. There are various ways to install a module in Python. However, in this tutorial, we will demonstrate how to use it using the pip module.<\/p>\n
Open your terminal or command prompt (if you’re using Windows) and enter the following command.<\/p>\n
pip install requests \r\n#Alternatively, if the first command does not work, use the below:\r\npip3 install requests<\/pre>\nIt should have installed the requests module successfully on your device.<\/p>\n
Python Requests<\/h4>\n
To understand how the requests module works, you\u00a0must first understand what happens when you\u00a0browse the web and how it instantaneously displays the data we were hoping to see.<\/p>\n
When you click a link, we make an HTTP (Hypertext Transfer Protocol) request to the server that hosts the requested page.<\/p>\n
When the server receives the request, it returns the requested content to us.<\/p>\n
The two most useful HTTP requests are GET<\/strong> and POST<\/strong>.<\/p>\n
Importing Requests:<\/strong><\/p>\n
import requests<\/pre>\nWhat is GET Request?<\/h4>\n
This approach indicates that we are requesting the contents of our chosen URL from the server. So, let’s suppose we want to retrieve Amazon’s<\/strong> homepage utilizing HTTP requests.<\/p>\n
Then Enter the below code:<\/p>\n
import requests\r\nmy_req = requests.get(\"http:\/\/amazon.com\")<\/pre>\nWhat the single line of code accomplishes here?<\/p>\n
It uses the get() method to send an HTTP GET request to Amazon’s homepage, with the URL as the argument. The response object is then saved in our ‘my_req ‘ variable.<\/p>\n
Our Response object instance further classifies the retained data and saves it in the appropriate attributes.<\/p>\n
Example:<\/strong><\/p>\n
import requests\r\nmy_req = requests.get(\"http:\/\/amazon.com\")\r\n# The result contains the url's status code.\r\n# The result of a successful full attempt is 200.\r\nprint(my_req.status_code)\r\n\r\n\r\n# This attribute returns a Python dictionary containing the\r\n# headers' key-value pairs.\r\nprint(my_req.headers)\r\n\r\n# It displays the server's response content or Static Source Code.\r\nprint(my_req.text)\r\n\r\n# we could also view or modify the encoding of the response content\r\n# using the Requests library.\r\nprint(my_req.encoding)\r\nmy_req.encoding = 'utf-8'\r\n<\/pre>\nOutput:<\/strong><\/p>\n
503\r\n{'Server': 'Server', 'Content-Type': 'text\/html', 'Content-Length': '1203', 'x-amz-rid': 'T4PK7QWJPVK62Z0FZM5M', 'Last-Modified': 'Fri, 03 Dec 2021 19:33:54 GMT', 'ETag': '\"a6f-5d242fcc50c80-gzip\"', 'Accept-Ranges': 'bytes', 'Content-Encoding': 'gzip', 'Strict-Transport-Security': 'max-age=47474747; includeSubDomains; preload', 'Permissions-Policy': 'interest-cohort=()', 'Date': 'Wed, 15 Dec 2021 02:21:45 GMT', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding'}\r\n<!--\r\n To discuss automated access to Amazon data please contact api-services-support@amazon.com.\r\n For information about migrating to our APIs refer to our Marketplace APIs at https:\/\/developer.amazonservices.com\/ref=rm_5_sv, or our Product Advertising API at https:\/\/affiliate-program.amazon.com\/gp\/advertising\/api\/detail\/main.html\/ref=rm_5_ac for advertising use cases.\r\n-->\r\n<!doctype html>\r\n<html>\r\n<head>\r\n <meta charset=\"utf-8\">\r\n <meta http-equiv=\"x-ua-compatible\" content=\"ie=edge\">\r\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\">\r\n <title>Sorry! Something went wrong!<\/title>\r\n <style>\r\n html, body {\r\n padding: 0;\r\n margin: 0\r\n }\r\n\r\n img {\r\n border: 0\r\n }\r\n\r\n #a {\r\n background: #232f3e;\r\n padding: 11px 11px 11px 192px\r\n }\r\n\r\n #b {\r\n position: absolute;\r\n left: 22px;\r\n top: 12px\r\n }\r\n\r\n #c {\r\n position: relative;\r\n max-width: 800px;\r\n padding: 0 40px 0 0\r\n }\r\n\r\n #e, #f {\r\n height: 35px;\r\n border: 0;\r\n font-size: 1em\r\n }\r\n\r\n #e {\r\n width: 100%;\r\n margin: 0;\r\n padding: 0 10px;\r\n border-radius: 4px 0 0 4px\r\n }\r\n\r\n #f {\r\n cursor: pointer;\r\n background: #febd69;\r\n font-weight: bold;\r\n border-radius: 0 4px 4px 0;\r\n -webkit-appearance: none;\r\n position: absolute;\r\n top: 0;\r\n right: 0;\r\n padding: 0 12px\r\n }\r\n\r\n @media (max-width: 500px) {\r\n #a {\r\n padding: 55px 10px 10px\r\n }\r\n\r\n #b {\r\n left: 6px\r\n }\r\n }\r\n\r\n #g {\r\n text-align: center;\r\n margin: 30px 0\r\n }\r\n\r\n #g img {\r\n max-width: 90%\r\n }\r\n\r\n #d {\r\n display: none\r\n }\r\n\r\n #d[src] {\r\n display: inline\r\n }\r\n <\/style>\r\n<\/head>\r\n<body>\r\n <a href=\"\/ref=cs_503_logo\"><img id=\"b\" src=\"https:\/\/images-na.ssl-images-amazon.com\/images\/G\/01\/error\/logo._TTD_.png\" alt=\"Amazon.com\"><\/a>\r\n <form id=\"a\" accept-charset=\"utf-8\" action=\"\/s\" method=\"GET\" role=\"search\">\r\n <div id=\"c\">\r\n <input id=\"e\" name=\"field-keywords\" placeholder=\"Search\">\r\n <input name=\"ref\" type=\"hidden\" value=\"cs_503_search\">\r\n <input id=\"f\" type=\"submit\" value=\"Go\">\r\n <\/div>\r\n <\/form>\r\n<div id=\"g\">\r\n <div><a href=\"\/ref=cs_503_link\"><img src=\"https:\/\/images-na.ssl-images-amazon.com\/images\/G\/01\/error\/500_503.png\"\r\n alt=\"Sorry! Something went wrong on our end. Please go back and try again or go to Amazon's home page.\"><\/a>\r\n <\/div>\r\n <a href=\"\/dogsofamazon\/ref=cs_503_d\" target=\"_blank\" rel=\"noopener noreferrer\"><img id=\"d\" alt=\"Dogs of Amazon\"><\/a>\r\n <script>document.getElementById(\"d\").src = \"https:\/\/images-na.ssl-images-amazon.com\/images\/G\/01\/error\/\" + (Math.floor(Math.random() * 43) + 1) + \"._TTD_.jpg\";<\/script>\r\n<\/div>\r\n<\/body>\r\n<\/html>\r\n\r\nISO-8859-1<\/pre>\nPassing Arguments with the GET Method<\/h4>\n
Usually, a single GET method does not allow us to get all of the information we require, so we must send additional parameters with our original get request.<\/p>\n
Parameters are mostly key-value pairs of data wrapped in a tuple or list. We can send it using the params parameter of our get() method.<\/p>\n
Example:<\/strong><\/p>\n
# Import requests using the import keyword\r\nimport requests \r\n# Give the dictionary as static input and store it in a variable.\r\npayload_val = {'key1': 'value1', 'key2': 'value2'}\r\n# Get the requests by passing some random URl and payload values as the arguments to it.\r\nmy_req = requests.get('http:\/\/httpbin.org\/get', params=payload_val)\r\n# print the above request\r\nprint(my_req.text)<\/pre>\nOutput:<\/strong><\/p>\n
{\r\n \"args\": {\r\n \"key1\": \"value1\", \r\n \"key2\": \"value2\"\r\n }, \r\n \"headers\": {\r\n \"Accept\": \"*\/*\", \r\n \"Accept-Encoding\": \"gzip, deflate\", \r\n \"Host\": \"httpbin.org\", \r\n \"User-Agent\": \"python-requests\/2.23.0\", \r\n \"X-Amzn-Trace-Id\": \"Root=1-61b95330-4018303804e1772e3f298164\"\r\n }, \r\n \"origin\": \"34.86.125.144\", \r\n \"url\": \"http:\/\/httpbin.org\/get?key1=value1&key2=value2\"\r\n}\r\n<\/pre>\nWhat is POST Request?<\/h3>\n
In contrast, to GET requests in Python, the POST Method in HTTP requires a payload to be sent along with it. Instead of retrieving data directly, this method is used to transfer it to a server. Using the post() method in our requests module, we can access POST.<\/p>\n
Example<\/strong><\/p>\n
# Import requests module using the import keyword\r\nimport requests \r\n# Give the payload dictionary as static input and store it in a variable\r\npayload_val = {'key_1': 'value_1', 'key_2': 'value_2'}\r\n# Pass the Url and above payload dictionary as arguments to the post() method\r\n# and store in another variable\r\nmy_req = requests.post(\"https:\/\/httpbin.org\/post\", data=payload_val)\r\n# print the above request\r\nprint(my_req .text)<\/pre>\nOutput:<\/strong><\/p>\n
{\r\n\"args\": {}, \r\n\"data\": \"\", \r\n\"files\": {}, \r\n\"form\": {\r\n\"key_1\": \"value_1\", \r\n\"key_2\": \"value_2\"\r\n}, \r\n\"headers\": {\r\n\"Accept\": \"*\/*\", \r\n\"Accept-Encoding\": \"gzip, deflate\", \r\n\"Content-Length\": \"27\", \r\n\"Content-Type\": \"application\/x-www-form-urlencoded\", \r\n\"Host\": \"httpbin.org\", \r\n\"User-Agent\": \"python-requests\/2.23.0\", \r\n\"X-Amzn-Trace-Id\": \"Root=1-61cf20c7-4887ec7502308d5a0671e1ae\"\r\n}, \r\n\"json\": null, \r\n\"origin\": \"35.245.205.160\", \r\n\"url\": \"https:\/\/httpbin.org\/post\"\r\n}<\/pre>\nPython Advanced Request Features<\/h4>\n
GET and POST are the most fundamental and important HTTP methods. However, the requests module allows a variety of such methods such as PUT, PATCH, DELETE, and so on.<\/p>\n
These are some of the main reasons why the ‘requests’ module is so popular among developers is because of advanced features such as:<\/p>\n