我想按照这个链接设置scrapy集群 scrapy-cluster,在我运行这个命令之前一切正常:
I wanna set up scrapy cluster follow this link scrapy-cluster,Everything is ok before I run this command:
pip install -r requirements.txt
requirements.txt 看起来像:
The requirements.txt looks like:
cffi==1.2.1
characteristic==14.3.0
ConcurrentLogHandler>=0.9.1
cryptography==0.9.1
...
我猜上面的命令意思是在requirements.txt中安装包.但是我不想让它指定版本,所以我把它改成这样:
I guess the above command means to install packages in requirements.txt.But I don't want it to specify the version,So I change it to this:
cat requirements.txt | while read line; do pip install ${line%%[>=]*} --user;done
安装密码学时,它给了我错误:
When install cryptography,it gives me the error:
build/temp.linux-x86_64-2.7/_openssl.c:12:24:fatal error:pyconfig.h:No such file or directory
#include <pyconfig.h>
我不知道如何解决这个问题,我尝试了很多方法,但都失败了.我的系统是centos 7,python的版本是2.7.5(默认).此外,是否还有其他适用于大量 url 的 scrapy 框架.提前致谢
I don't know how to solved this , I have tried a lot of methods ,but failed. my system is centos 7, and the version of python is 2.7.5(default). Besides, Is there any other scrapy frame which is appliable for a large number of urls . Thanks in advance
对于 Ubuntu,python2
For Ubuntu, python2
apt-get install python-dev
对于 Ubuntu,python3
For Ubuntu, python3
apt-get install python3-dev
这篇关于在“pip install cryptography"期间缺少 pyconfig.h的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持跟版网!