Scrapy 项目配置

        找了个兼职做,是做爬虫相关的东西。对方给了个配置文件我:models.py,我搭建好django后,也创建一个scrapy的app,不过却运行不起来。

报错为:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 354, in execute
    django.setup()
  File "/usr/lib/python2.7/dist-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/usr/lib/python2.7/dist-packages/django/apps/config.py", line 202, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/root/pro/py/soo/sox/models.py", line 4, in <module>
    from scrapy.contrib.djangoitem import DjangoItem
ImportError: No module named djangoitem

看起来挺蛋疼的,于是我去了scrapy的contrib目录发现没有djangoitem 这个目录。然后检查了scrapy版本,是1.0.3,又看到教程的是0.28 

所以看起来应该是这个类被分离出去了。结果在scrapy的官网上确实发现了http://doc.scrapy.org/en/1.0/news.html?highlight=djangoitem

于是跑到了github下载了https://github.com/scrapy-plugins/scrapy-djangoitem,    安装,
改了from
结果是运行正确了。可是那个配置文件还是有问题,继续查去了。