8 Commits

Author	SHA1	Message	Date
henningxtro	26ba21156a	BETTERZON-58 (#53) ... * BETTERZON-58: Basic Functionality with scrapy * Added independent crawler function, yielding price * moved logic to amazon.py * . * moved scrapy files to unused folder * Added basic amazon crawler using beautifulsoup4 * Connected Api to Crawler * Fixed string concatenation for sql statement in getProductLinksForProduct * BETTERZON-58: Fixing SQL insert * BETTERZON-58: Adding access key verification * BETTERZON-58: Fixing API endpoint of the crawler - The list of products in the API request was treated like a string and henceforth, only the first product has been crawled * Added another selector for price on amazon (does not work for books) Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain> Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com>	2021-05-19 00:46:14 +02:00
henningxtro	9e9b442f35	BETTERZON-58: Basic Functionality with scrapy (#33)	2021-04-28 22:24:13 +02:00
Patrick	610808ad03	BETTERZON-59: Adding crawler basic framework (#29)	2021-04-14 21:51:36 +02:00
Patrick	f5fd1825d7	BETTERZON-56: Adding crawler load-balancing script (#28)	2021-04-14 18:52:22 +02:00
Patrick Müller	fafacdd942	BETTERZON-57: Adding utility sql functions	2021-04-13 21:10:02 +02:00
henningxtro	55a019d217	BETTERZON-49 (#24) ... * BETTERZON-49: Creating Dockerfile * BETTERZON-49: Added minimal Flask API as Docker container Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech>	2021-04-07 23:34:08 +02:00
Patrick Müller	8055f811d7	BETTERZON-49: Adding module .iml	2021-04-01 11:38:03 +02:00
Patrick Müller	05d4795f9d	BETTERZON-49: Creating Module for Crawler	2021-04-01 10:34:35 +02:00