Paddy/Betterzon
1
0
Fork 0
mirror of https://github.com/Mueller-Patrick/Betterzon.git synced 2024-12-23 12:15:11 +00:00
Code Issues Projects Releases Wiki Activity
213 Commits 2 Branches 0 Tags 16 MiB
13939a91ca
Commit Graph

9 Commits

Author SHA1 Message Date
Patrick Müller
5712c5c69e Merge remote-tracking branch 'origin/develop' into BETTERZON-109 
# Conflicts:
#	Frontend/src/app/app.component.html
#	Frontend/src/app/app.module.ts
#	Frontend/src/app/components/top-bar/top-bar.component.css
#	Frontend/src/app/components/top-bar/top-bar.component.html
#	Frontend/src/app/pages/landingpage/landingpage.component.html
#	Frontend/src/styles.css
 2021-05-20 10:35:41 +02:00
henningxtro
26ba21156a
BETTERZON-58 (#53) 
* BETTERZON-58: Basic Functionality with scrapy

* Added independent crawler function, yielding price

* moved logic to amazon.py

* .

* moved scrapy files to unused folder

* Added basic amazon crawler using beautifulsoup4

* Connected Api to Crawler

* Fixed string concatenation for sql statement in getProductLinksForProduct

* BETTERZON-58: Fixing SQL insert

* BETTERZON-58: Adding access key verification

* BETTERZON-58: Fixing API endpoint of the crawler
- The list of products in the API request was treated like a string and henceforth, only the first product has been crawled

* Added another selector for price on amazon (does not work for books)

Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain>
Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech>
Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com>
 2021-05-19 00:46:14 +02:00
henningxtro
9e9b442f35
BETTERZON-58: Basic Functionality with scrapy (#33) 2021-04-28 22:24:13 +02:00
Patrick
610808ad03
BETTERZON-59: Adding crawler basic framework (#29) 2021-04-14 21:51:36 +02:00
Patrick
f5fd1825d7
BETTERZON-56: Adding crawler load-balancing script (#28) 2021-04-14 18:52:22 +02:00
Patrick Müller
fafacdd942 BETTERZON-57: Adding utility sql functions 2021-04-13 21:10:02 +02:00
henningxtro
55a019d217
BETTERZON-49 (#24) 
* BETTERZON-49: Creating Dockerfile

* BETTERZON-49: Added minimal Flask API as Docker container

Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech>
 2021-04-07 23:34:08 +02:00
Patrick Müller
8055f811d7 BETTERZON-49: Adding module .iml 2021-04-01 11:38:03 +02:00
Patrick Müller
05d4795f9d BETTERZON-49: Creating Module for Crawler 2021-04-01 10:34:35 +02:00