How To Install Scrapy The Recommended Way

How To Install Scrapy The Recommended Way

Posted on

This tutorial will guide you to install scrapy on windows the recommended way. If you have an error regarding installation about cannot install Twisted module, i will show you how to fix pip install scrapy error using this guide.

Scrapy is amazing tool you should learn for every data scraper.

Installation:

1. Miniconda: https://docs.conda.io/en/latest/miniconda.html

2. Scrapy: https://docs.scrapy.org/en/latest/intro/install.html

I made a video tutorial about how to install scrapy on Windows.

Error about Twisted module when Pip Install Scrapy

I got this error about Twisted cannot be installed. Pip install Scrapy error exists about Twisted package cannot be built.

This will help you solve how to fix This Pip install Scrapy error windows OS.

Twisted package download from this link: https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

Download Twisted latest version. i.e.: Twisted‑20.3.0‑cp39‑cp39‑win_amd64.whl

After downloading the file, open a cmd in that folder and run the command:

pip install [file .whl]

For example, if you downloaded Twisted‑20.3.0‑cp39‑cp39‑win_amd64.whl use:

pip install Twisted‑20.3.0‑cp39‑cp39‑win_amd64.whl

Try to install Scrapy again:

pip install Scrapy

Hope it helps.