问题描述
我有一个非常基本的蜘蛛,它看起来像来自 scrapy testpiders 的 followall 蜘蛛.
I have a very basic spider that looks like the followall spider from scrapy testspiders.
我从下面的脚本中调用这个蜘蛛.蜘蛛使用 CrawlRunner 类运行,当它获取一个项目时会发出一个信号作为 p.signals.connect ,然后调用方法 crawler_results 并打印被抓取的项目.
I am calling this spider from a script as below. The spider is run using the CrawlRunner class and when it fetches an item emits a signal as p.signals.connect which then calls the method crawler_results and prints item scraped.
据我了解,我无法将爬行移动到它自己的类中,因为那样信号将无法与 PyQt5 一起使用
As far as my understanding is I cannot move the crawling into it's own class because then the signal wont work with PyQt5
我有一个带有简单 QTableWidget 和按钮的布局
I have a layout with a simple QTableWidget and a pushbutton
当我按下按钮时,我可以看到爬虫正在运行并进入 crawler_results 方法,因为它打印了抓取的项目.蜘蛛将每个项目返回为以下值
When I hit the pushbutton I can see the crawler running and entering the crawler_results method as it prints the item scraped. The spider returns each item as the following value
页面只是我的scrapy项目
Page is simply my scrapy items
我的问题是如何将这些数据转换到 GUI 中并让它在蜘蛛运行时自动刷新.这意味着每次抓取一个项目时,GUI 都会更新,然后蜘蛛会继续.
My question is how do I translate this data into the GUI and have it auto refresh as long as the spider runs. This means that every time an item is scraped the GUI updates and then the spider continues.
到目前为止我已经探索过了
I have so far explored
- 使用scrapy deferred 运气不佳
- 插槽/信号,但无法更新 GUI.
- 每秒更新一次 GUI 的 Qtimer 函数,但同样不会产生任何结果.
非常感谢任何帮助
推荐答案
你必须安装一个兼容 Qt 事件循环的反应器,例如使用:
You have to install a reactor compatible with the Qt event loop, for example using:
qt5reactor
(python -m pip install qt5reactor
),qt-reactor
(python -m pip install qt-reactor
)
这篇关于根据来自scrapy的信号更新主线程内的PyQt5 Gui的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持跟版网!