mirror of
https://github.com/Mueller-Patrick/Betterzon.git
synced 2024-12-23 20:15:13 +00:00
26ba21156a
* BETTERZON-58: Basic Functionality with scrapy * Added independent crawler function, yielding price * moved logic to amazon.py * . * moved scrapy files to unused folder * Added basic amazon crawler using beautifulsoup4 * Connected Api to Crawler * Fixed string concatenation for sql statement in getProductLinksForProduct * BETTERZON-58: Fixing SQL insert * BETTERZON-58: Adding access key verification * BETTERZON-58: Fixing API endpoint of the crawler - The list of products in the API request was treated like a string and henceforth, only the first product has been crawled * Added another selector for price on amazon (does not work for books) Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain> Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com>
11 lines
256 B
INI
11 lines
256 B
INI
# Automatically created by: scrapy startproject
|
|
#
|
|
# For more information about the [deploy] section see:
|
|
# https://scrapyd.readthedocs.io/en/latest/deploy.html
|
|
|
|
[settings]
|
|
default = crawler.settings
|
|
|
|
[deploy]
|
|
#url = http://localhost:6800/
|
|
project = crawler |