{"id":2205,"date":"2023-10-15T14:42:01","date_gmt":"2023-10-15T09:12:01","guid":{"rendered":"https:\/\/python-programs.com\/?p=2205"},"modified":"2023-11-10T11:41:35","modified_gmt":"2023-11-10T06:11:35","slug":"how-to-scrape-amazon-data-using-python-scrapy","status":"publish","type":"post","link":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/","title":{"rendered":"How To Scrape Amazon Data Using Python Scrapy"},"content":{"rendered":"<p>Will it not be good if all the information related to some product will be placed in only one table? I guess it will be really awesome and accessible if we can get the entire information at one place.<\/p>\n<p>Since, Amazon is a huge website containing millions of data so scraping the data is quite challenging. Amazon is a tough website to scrape for beginners and people often get blocked by Amazon\u2019s anti-scraping technology.<\/p>\n<p>In this blog, we will be aiming to provide the information about the scrapy and how to scrape the Amazon website using it.<\/p>\n<h2>What is Scrapy?<\/h2>\n<p>Scrapy is a free and open-source web-crawling Python\u2019s framework. It was originally designed for web scraping, extracting the data using API\u2019s and or general-purpose web crawler.<\/p>\n<p>This framework is used in data mining, information processing or historical archival. The applications of this framework is used widely in different industries and has been proven very useful. It not only scrapes the data from the website, but it is able to scrape the data from the web services also. For example, Amazon API, Facebook API, and many more.<\/p>\n<h3>How to install Scrapy?<\/h3>\n<p>Firstly, there are some third-party softwares which needs to be installed in order to install the Scrapy module.<\/p>\n<ul>\n<li><strong>Python:<\/strong> As Scrapy has the base of the Python language, one has to install it first.<\/li>\n<li><strong>pip:<\/strong> pip is a python package manager tool which maintains a package repository and installs python libraries, and its dependencies automatically. It is better to install pip according to system OS, and then try to follow the standard way of installing Scrapy.<\/li>\n<\/ul>\n<p>There are different ways in which we can download Scrapy globally as well as locally but the most standard way of downloading it is by using <em>pip.<\/em><\/p>\n<p>Run the below command to install Scrapy using <em>pip:<\/em><\/p>\n<pre><strong><em>Pip install scrapy<\/em><\/strong><\/pre>\n<h3>How to get started with Scrapy?<\/h3>\n<p>Since we know that Scrapy is an application framework and it provides multiple commands to create an application and use them. But before everything, we have to set up a new Scrapy project. Enter a directory where you\u2019d like to store your code and run:<\/p>\n<pre><strong><em>Scrapy startproject new_project<\/em><\/strong><\/pre>\n<p>This will create a directory:<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-2211\" src=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-get-started-with-Scrapy.png\" alt=\"\" width=\"400\" height=\"357\" srcset=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-get-started-with-Scrapy.png 400w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-get-started-with-Scrapy-300x268.png 300w\" sizes=\"(max-width: 400px) 100vw, 400px\" \/><\/p>\n<p>Scrapy is an application framework which follows object oriented programming style for the definition of items and spiders for overall applications.<\/p>\n<p>The project structure contains different the following files:<\/p>\n<ol>\n<li><strong>Scrapy.cfg<\/strong> : This file is a root directory of the project, which includes project name with the project settings.<\/li>\n<li><strong>Test_project<\/strong> : It is an application directory with many different files which actually make running and scraping responsible from the web URLs.<\/li>\n<li>I<strong>tems.py <\/strong>:Items are containers that will be loaded with the scraped data and they work like simple python dictionaries.Items provide additional prevention against typos and populating undeclared fields.<\/li>\n<li><strong>Pipelines.py<\/strong> :After the scraping of an item has been done by the spider, it is sent to the item pipeline which processes it through several components. Each class has to implement a method called process_item for the processing of scraped items.<\/li>\n<li><strong>Settings.py<\/strong> : It allows the customization of the behaviour of all scrapy components, including the core, extensions, pipelines, and spiders themselves.<\/li>\n<li><strong>Spiders<\/strong> : It is a directory which contains all spiders\/crawlers as python classes.<\/li>\n<\/ol>\n<h3>Scrape Amazon Data: How to Scrape an Amazon Web Page<\/h3>\n<p>For a better understanding of how the scrapy works, we will be scraping the product name, price, category, and it\u2019s availability from the Amazon.com website.<\/p>\n<div class=\"ast-oembed-container\"><iframe loading=\"lazy\" title=\"Python Scrapy Tutorial - 22 - Web Scraping Amazon\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/2vcp0fKq3aw?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/div>\n<p>Let\u2019s name this project amazon_pro. You can use the project name according to your choice.<\/p>\n<p>Start by writing the below code:<\/p>\n<p><strong><em>Scrapy startproject test_project<\/em><\/strong><\/p>\n<p>The directory will be created in the local folder by the name mentioned above.<\/p>\n<p>Now we need three things which will help in the scraping process.<\/p>\n<ol>\n<li>Update <strong><em>items.py <\/em><\/strong>field which we want to scrape. For example names, price, availability, and so on.<\/li>\n<li>We have to create a new spider with all the necessary elements in it like <strong><em>allowed domains<\/em><\/strong>, <strong><em>start_urls<\/em><\/strong> and <strong><em>parse<\/em><\/strong> method.<\/li>\n<li>For data processing, we have to update <strong><em>pipelines.py<\/em><\/strong> file.<\/li>\n<\/ol>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-2213\" src=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-1.png\" alt=\"\" width=\"1366\" height=\"479\" srcset=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-1.png 1366w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-1-300x105.png 300w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-1-1024x359.png 1024w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-1-768x269.png 768w\" sizes=\"(max-width: 1366px) 100vw, 1366px\" \/><\/p>\n<p>Now after creating the spider, follow thee further steps given in the terminal:<\/p>\n<ol>\n<li>\n<pre><strong><em>Scrapy genspider amazon amazon.com<\/em><\/strong><\/pre>\n<\/li>\n<\/ol>\n<p>Now, we need to define the name, URLs, and possible domains to scrape the data.<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-2214\" src=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-2.png\" alt=\"How to Scrape an Amazon Web Page 2\" width=\"1360\" height=\"662\" srcset=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-2.png 1360w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-2-300x146.png 300w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-2-1024x498.png 1024w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-2-768x374.png 768w\" sizes=\"(max-width: 1360px) 100vw, 1360px\" \/><\/p>\n<p>An item object is defined in the<strong> parse<\/strong> method and is filled with required information using the utility of <strong>XPath<\/strong> response object. It is a search function that is used to find elements in the HTML tree structure. Lastly let\u2019s yield the items object, so that scrapy can do further processing on it.<\/p>\n<p>Next, after scraping data, scrapy calls Item pipelines to process them. These are called <strong>Pipeline<\/strong> classes and we can use these classes to store data in a file or database or in any other way. It is a default class like Items that scrapy generates for users.<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-2215\" src=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-3.png\" alt=\"How to Scrape an Amazon Web Page 3\" width=\"1340\" height=\"490\" srcset=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-3.png 1340w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-3-300x110.png 300w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-3-1024x374.png 1024w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-3-768x281.png 768w\" sizes=\"(max-width: 1340px) 100vw, 1340px\" \/><\/p>\n<p>The <strong>process_item<\/strong> is implemented by the pipeline classes method and items are being yielded by a Spider each and every time. It takes the <strong>item<\/strong> and <strong>spider<\/strong> class as arguments and returns a <strong>dict<\/strong> object. So for this example, we are just returning item dict as it is.<\/p>\n<p>Now, we have to enable the <strong><em>ITEM_PIPELINES <\/em><\/strong>from settings.py file.<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-full wp-image-2216\" src=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-4.png\" alt=\"How to Scrape an Amazon Web Page 4\" width=\"1365\" height=\"487\" srcset=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-4.png 1365w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-4-300x107.png 300w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-4-1024x365.png 1024w, https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-to-Scrape-an-Amazon-Web-Page-4-768x274.png 768w\" sizes=\"(max-width: 1365px) 100vw, 1365px\" \/><\/p>\n<p>Now, after completing the entire code, we need to scrape the item by sending requests and accepting response objects.<\/p>\n<p>We will call a spider by its unique name and scrapy will easily search from it.<\/p>\n<pre><strong><em>Scrapy crawl amazon<\/em><\/strong><\/pre>\n<p>Now, after the items have been scraped, we can save it to different formats using their extensions. For example, .json, .csv, and many more.<\/p>\n<pre><strong><em>Scrapy crawl amazon -o data.csv<\/em><\/strong><\/pre>\n<p>The above command will save the scraped data in the csv format in <strong>data.csv<\/strong> file.<\/p>\n<p>Here is the output of the above code:<\/p>\n<pre>[ \r\n{\"product_category\": \"Electronics,Computers &amp; Accessories,Data Storage,External Hard Drives\", \"product_sale_price\": \"$949.95\", \"product_name\": \"G-Technology G-SPEED eS PRO High-Performance Fail-Safe RAID Solution for HD\/2K Production 8TB (0G01873)\", \"product_availability\": \"Only 1 left in stock.\"},\r\n{\"product_category\": \"Electronics,Computers &amp; Accessories,Data Storage,USB Flash Drives\", \"product_sale_price\": \"\", \"product_name\": \"G-Technology G-RAID with Removable Drives High-Performance Storage System 4TB (Gen7) (0G03240)\", \"product_availability\": \"Available from these sellers.\"},\r\n{\"product_category\": \"Electronics,Computers &amp; Accessories,Data Storage,USB Flash Drives\", \"product_sale_price\": \"$549.95\", \"product_name\": \"G-Technology G-RAID USB Removable Dual Drive Storage System 8TB (0G04069)\", \"product_availability\": \"Only 1 left in stock.\"},\r\n{\"product_category\": \"Electronics,Computers &amp; Accessories,Data Storage,External Hard Drives\", \"product_sale_price\": \"$89.95\", \"product_name\": \"G-Technology G-DRIVE ev USB 3.0 Hard Drive 500GB (0G02727)\", \"product_availability\": \"Only 1 left in stock.\"}\r\n]<\/pre>\n<p>We have successfully scraped the data from Amazon.com using scrapy.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Will it not be good if all the information related to some product will be placed in only one table? I guess it will be really awesome and accessible if we can get the entire information at one place. Since, Amazon is a huge website containing millions of data so scraping the data is quite &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/\"> <span class=\"screen-reader-text\">How To Scrape Amazon Data Using Python Scrapy<\/span> Read More &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":2206,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[5],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v18.9 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How To Scrape Amazon Data Using Python Scrapy - Python Programs<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How To Scrape Amazon Data Using Python Scrapy - Python Programs\" \/>\n<meta property=\"og:description\" content=\"Will it not be good if all the information related to some product will be placed in only one table? I guess it will be really awesome and accessible if we can get the entire information at one place. Since, Amazon is a huge website containing millions of data so scraping the data is quite &hellip; How To Scrape Amazon Data Using Python Scrapy Read More &raquo;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/\" \/>\n<meta property=\"og:site_name\" content=\"Python Programs\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/btechgeeks\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-15T09:12:01+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-11-10T06:11:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"720\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@btech_geeks\" \/>\n<meta name=\"twitter:site\" content=\"@btech_geeks\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"veer\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/python-programs.com\/#organization\",\"name\":\"BTech Geeks\",\"url\":\"https:\/\/python-programs.com\/\",\"sameAs\":[\"https:\/\/www.instagram.com\/btechgeeks\/\",\"https:\/\/www.linkedin.com\/in\/btechgeeks\",\"https:\/\/in.pinterest.com\/btechgeek\/\",\"https:\/\/www.youtube.com\/channel\/UC9MlCqdJ3lKqz2p5114SDIg\",\"https:\/\/www.facebook.com\/btechgeeks\",\"https:\/\/twitter.com\/btech_geeks\"],\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/python-programs.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/python-programs.com\/wp-content\/uploads\/2020\/11\/BTechGeeks.png\",\"contentUrl\":\"https:\/\/python-programs.com\/wp-content\/uploads\/2020\/11\/BTechGeeks.png\",\"width\":350,\"height\":70,\"caption\":\"BTech Geeks\"},\"image\":{\"@id\":\"https:\/\/python-programs.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/python-programs.com\/#website\",\"url\":\"https:\/\/python-programs.com\/\",\"name\":\"Python Programs\",\"description\":\"Python Programs with Examples, How To Guides on Python\",\"publisher\":{\"@id\":\"https:\/\/python-programs.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/python-programs.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#primaryimage\",\"url\":\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg\",\"contentUrl\":\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg\",\"width\":1280,\"height\":720,\"caption\":\"How To Scrape Amazon Data Using Python Scrapy\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#webpage\",\"url\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/\",\"name\":\"How To Scrape Amazon Data Using Python Scrapy - Python Programs\",\"isPartOf\":{\"@id\":\"https:\/\/python-programs.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#primaryimage\"},\"datePublished\":\"2023-10-15T09:12:01+00:00\",\"dateModified\":\"2023-11-10T06:11:35+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/python-programs.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How To Scrape Amazon Data Using Python Scrapy\"}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#webpage\"},\"author\":{\"@id\":\"https:\/\/python-programs.com\/#\/schema\/person\/9f9e30fd3f415217a11ac0d939213b7f\"},\"headline\":\"How To Scrape Amazon Data Using Python Scrapy\",\"datePublished\":\"2023-10-15T09:12:01+00:00\",\"dateModified\":\"2023-11-10T06:11:35+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#webpage\"},\"wordCount\":979,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/python-programs.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg\",\"articleSection\":[\"Python\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#respond\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/python-programs.com\/#\/schema\/person\/9f9e30fd3f415217a11ac0d939213b7f\",\"name\":\"veer\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/python-programs.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f1f6915d5328abaea9a64249313d1c55?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f1f6915d5328abaea9a64249313d1c55?s=96&d=mm&r=g\",\"caption\":\"veer\"},\"sameAs\":[\"https:\/\/python-programs.com\"],\"url\":\"https:\/\/python-programs.com\/author\/veer\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How To Scrape Amazon Data Using Python Scrapy - Python Programs","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/","og_locale":"en_US","og_type":"article","og_title":"How To Scrape Amazon Data Using Python Scrapy - Python Programs","og_description":"Will it not be good if all the information related to some product will be placed in only one table? I guess it will be really awesome and accessible if we can get the entire information at one place. Since, Amazon is a huge website containing millions of data so scraping the data is quite &hellip; How To Scrape Amazon Data Using Python Scrapy Read More &raquo;","og_url":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/","og_site_name":"Python Programs","article_publisher":"https:\/\/www.facebook.com\/btechgeeks","article_published_time":"2023-10-15T09:12:01+00:00","article_modified_time":"2023-11-10T06:11:35+00:00","og_image":[{"width":1280,"height":720,"url":"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_creator":"@btech_geeks","twitter_site":"@btech_geeks","twitter_misc":{"Written by":"veer","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/python-programs.com\/#organization","name":"BTech Geeks","url":"https:\/\/python-programs.com\/","sameAs":["https:\/\/www.instagram.com\/btechgeeks\/","https:\/\/www.linkedin.com\/in\/btechgeeks","https:\/\/in.pinterest.com\/btechgeek\/","https:\/\/www.youtube.com\/channel\/UC9MlCqdJ3lKqz2p5114SDIg","https:\/\/www.facebook.com\/btechgeeks","https:\/\/twitter.com\/btech_geeks"],"logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/python-programs.com\/#\/schema\/logo\/image\/","url":"https:\/\/python-programs.com\/wp-content\/uploads\/2020\/11\/BTechGeeks.png","contentUrl":"https:\/\/python-programs.com\/wp-content\/uploads\/2020\/11\/BTechGeeks.png","width":350,"height":70,"caption":"BTech Geeks"},"image":{"@id":"https:\/\/python-programs.com\/#\/schema\/logo\/image\/"}},{"@type":"WebSite","@id":"https:\/\/python-programs.com\/#website","url":"https:\/\/python-programs.com\/","name":"Python Programs","description":"Python Programs with Examples, How To Guides on Python","publisher":{"@id":"https:\/\/python-programs.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/python-programs.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#primaryimage","url":"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg","contentUrl":"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg","width":1280,"height":720,"caption":"How To Scrape Amazon Data Using Python Scrapy"},{"@type":"WebPage","@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#webpage","url":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/","name":"How To Scrape Amazon Data Using Python Scrapy - Python Programs","isPartOf":{"@id":"https:\/\/python-programs.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#primaryimage"},"datePublished":"2023-10-15T09:12:01+00:00","dateModified":"2023-11-10T06:11:35+00:00","breadcrumb":{"@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/python-programs.com\/"},{"@type":"ListItem","position":2,"name":"How To Scrape Amazon Data Using Python Scrapy"}]},{"@type":"Article","@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#article","isPartOf":{"@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#webpage"},"author":{"@id":"https:\/\/python-programs.com\/#\/schema\/person\/9f9e30fd3f415217a11ac0d939213b7f"},"headline":"How To Scrape Amazon Data Using Python Scrapy","datePublished":"2023-10-15T09:12:01+00:00","dateModified":"2023-11-10T06:11:35+00:00","mainEntityOfPage":{"@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#webpage"},"wordCount":979,"commentCount":0,"publisher":{"@id":"https:\/\/python-programs.com\/#organization"},"image":{"@id":"https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#primaryimage"},"thumbnailUrl":"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg","articleSection":["Python"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/python-programs.com\/how-to-scrape-amazon-data-using-python-scrapy\/#respond"]}]},{"@type":"Person","@id":"https:\/\/python-programs.com\/#\/schema\/person\/9f9e30fd3f415217a11ac0d939213b7f","name":"veer","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/python-programs.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f1f6915d5328abaea9a64249313d1c55?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f1f6915d5328abaea9a64249313d1c55?s=96&d=mm&r=g","caption":"veer"},"sameAs":["https:\/\/python-programs.com"],"url":"https:\/\/python-programs.com\/author\/veer\/"}]}},"jetpack_featured_media_url":"https:\/\/python-programs.com\/wp-content\/uploads\/2021\/04\/How-To-Scrape-Amazon-Data-Using-Python-Scrapy.jpg","_links":{"self":[{"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/posts\/2205"}],"collection":[{"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/comments?post=2205"}],"version-history":[{"count":5,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/posts\/2205\/revisions"}],"predecessor-version":[{"id":2217,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/posts\/2205\/revisions\/2217"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/media\/2206"}],"wp:attachment":[{"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/media?parent=2205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/categories?post=2205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/python-programs.com\/wp-json\/wp\/v2\/tags?post=2205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}