Paddy/Betterzon
1
0
Fork 0
mirror of https://github.com/Mueller-Patrick/Betterzon.git synced 2024-11-15 02:53:57 +00:00
Code Issues Projects Releases Wiki Activity
5e450ea0c2
Betterzon/Crawler/requirements.txt

7 lines
80 B
Plaintext
Raw Normal View History

BETTERZON-49: Creating Module for Crawler
2021-04-01 08:34:35 +00:00
pymysql
Master deployment (#67) * BETTERZON-58: Basic Functionality with scrapy (#33) * BETTERZON-73: Adding API endpoint that returns the lowest non-amazon prices for a given list of product ids (#32) * BETTERZON-75: User registration API endpoint (#34) * BETTERZON-75: Adding backend functions to enable user registration * BETTERZON-75: Adding regex to check email and username * BETTERZON-83: FE unit testing (#35) * BETTERZON-83: Making pre-generated unit tests work * BETTERZON-83: Writing unit tests for angular to improve code coverage * BETTERZON-79: Adding API endpoint for logging in (#36) * BETTERZON-84: Adding service method to check if a session is valid (#37) * BETTERZON-77: Changing error behavior as the previous behavior cloud have opened up security vulnerabilities (#38) * BETTERZON-76: Adding method descriptions for backend service methods (#40) * Adding Codacy code quality badge to README * BETTERZON-89: Refactoring / Reformatting and adding unit tests (#41) * BETTERZON-90: Adding API endpoint for creating price alarms (#42) * BETTERZON-91: Adding API endpoint to GET all price alarms for the currently logged in user (#43) * BETTERZON-92: Adding API endpoint to edit (update) price alarms (#44) * BETTERZON-99: Adding some basic cucumber tests (#45) * BETTERZON-100: Switching to cookies for session management (#46) * BETTERZON-100: Switching session handling to cookies * BETTERZON-100: Some code reformatting * BETTERZON-100: Some more code reformatting * BETTERZON-93: Adding API endpoint to get managed shops (#47) * BETTERZON-94: Adding API endpoint to deactivate price listings as a vendor manager (#48) * BETTERZON-97: Adding API endpoint to get all products listed by a specific vendor (#50) * BETTERZON-98: Adding API endpoint for adding price entries as a registered vendor manager (#51) * BETTERZON-95: Adding API endpoint for getting, inserting and updating contact persons (#52) * BETTERZON-58 (#53) * BETTERZON-58: Basic Functionality with scrapy * Added independent crawler function, yielding price * moved logic to amazon.py * . * moved scrapy files to unused folder * Added basic amazon crawler using beautifulsoup4 * Connected Api to Crawler * Fixed string concatenation for sql statement in getProductLinksForProduct * BETTERZON-58: Fixing SQL insert * BETTERZON-58: Adding access key verification * BETTERZON-58: Fixing API endpoint of the crawler - The list of products in the API request was treated like a string and henceforth, only the first product has been crawled * Added another selector for price on amazon (does not work for books) Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain> Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-96: Adding API endpoint for delisting a whole vendor (#54) * BETTERZON-101: Adding service functions for pricealarms api (#55) - Not properly tested though as login functionality is required to test but not yet implemented * BETTERZON-110: Refactoring, reformatting and commenting api service (#56) * BETTERZON-107: Refactoring code with Proxy as design pattern (#49) * BETTERZON-78 (#39) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-109 (#57) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. * wip: new component created and added to the app.module.ts. Added a minimal grid layout. * wip: all components were wrapped now. Grid structure has been applied to the main wrapper-class "container". * wip: component created and added to the app.module.ts Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-108 (#58) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. * wip: new component created and added to the app.module.ts. Added a minimal grid layout. * wip: all components were wrapped now. Grid structure has been applied to the main wrapper-class "container". Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-106 (#59) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. * wip: new component created and added to the app.module.ts. Added a minimal grid layout. Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTEZON-102 (#60) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-113, BETTERZON-114, BETTERZON-115: Adding API endpoint for favorite shops (#61) * BETTERZON-116: Adding API endpoint for searching a new product (#62) * BETTERZON-117: Adding API endpoint for getting the latest crawling status (#63) * BETTERZON-111: Adding service functions for login and registration (#64) * BETTERZON-112: Adding service functions for managing vendor shops (#65) * BETTERZON-118: Adding service functions for managing favorite shops (#66) Co-authored-by: henningxtro <sextro.henning@student.dhbw-karlsruhe.de> Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain> Co-authored-by: Reboooooorn <61185041+Reboooooorn@users.noreply.github.com>
2021-05-29 08:58:27 +00:00
flask==1.1.2
BETTERZON-49 (#24) * BETTERZON-49: Creating Dockerfile * BETTERZON-49: Added minimal Flask API as Docker container Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech>
2021-04-07 21:34:08 +00:00
flask-sqlalchemy
BETTERZON-57: Adding utility sql functions
2021-04-13 19:10:02 +00:00
flask_restful
Master deployment (#67) * BETTERZON-58: Basic Functionality with scrapy (#33) * BETTERZON-73: Adding API endpoint that returns the lowest non-amazon prices for a given list of product ids (#32) * BETTERZON-75: User registration API endpoint (#34) * BETTERZON-75: Adding backend functions to enable user registration * BETTERZON-75: Adding regex to check email and username * BETTERZON-83: FE unit testing (#35) * BETTERZON-83: Making pre-generated unit tests work * BETTERZON-83: Writing unit tests for angular to improve code coverage * BETTERZON-79: Adding API endpoint for logging in (#36) * BETTERZON-84: Adding service method to check if a session is valid (#37) * BETTERZON-77: Changing error behavior as the previous behavior cloud have opened up security vulnerabilities (#38) * BETTERZON-76: Adding method descriptions for backend service methods (#40) * Adding Codacy code quality badge to README * BETTERZON-89: Refactoring / Reformatting and adding unit tests (#41) * BETTERZON-90: Adding API endpoint for creating price alarms (#42) * BETTERZON-91: Adding API endpoint to GET all price alarms for the currently logged in user (#43) * BETTERZON-92: Adding API endpoint to edit (update) price alarms (#44) * BETTERZON-99: Adding some basic cucumber tests (#45) * BETTERZON-100: Switching to cookies for session management (#46) * BETTERZON-100: Switching session handling to cookies * BETTERZON-100: Some code reformatting * BETTERZON-100: Some more code reformatting * BETTERZON-93: Adding API endpoint to get managed shops (#47) * BETTERZON-94: Adding API endpoint to deactivate price listings as a vendor manager (#48) * BETTERZON-97: Adding API endpoint to get all products listed by a specific vendor (#50) * BETTERZON-98: Adding API endpoint for adding price entries as a registered vendor manager (#51) * BETTERZON-95: Adding API endpoint for getting, inserting and updating contact persons (#52) * BETTERZON-58 (#53) * BETTERZON-58: Basic Functionality with scrapy * Added independent crawler function, yielding price * moved logic to amazon.py * . * moved scrapy files to unused folder * Added basic amazon crawler using beautifulsoup4 * Connected Api to Crawler * Fixed string concatenation for sql statement in getProductLinksForProduct * BETTERZON-58: Fixing SQL insert * BETTERZON-58: Adding access key verification * BETTERZON-58: Fixing API endpoint of the crawler - The list of products in the API request was treated like a string and henceforth, only the first product has been crawled * Added another selector for price on amazon (does not work for books) Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain> Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-96: Adding API endpoint for delisting a whole vendor (#54) * BETTERZON-101: Adding service functions for pricealarms api (#55) - Not properly tested though as login functionality is required to test but not yet implemented * BETTERZON-110: Refactoring, reformatting and commenting api service (#56) * BETTERZON-107: Refactoring code with Proxy as design pattern (#49) * BETTERZON-78 (#39) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-109 (#57) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. * wip: new component created and added to the app.module.ts. Added a minimal grid layout. * wip: all components were wrapped now. Grid structure has been applied to the main wrapper-class "container". * wip: component created and added to the app.module.ts Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-108 (#58) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. * wip: new component created and added to the app.module.ts. Added a minimal grid layout. * wip: all components were wrapped now. Grid structure has been applied to the main wrapper-class "container". Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-106 (#59) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. * wip: new component created and added to the app.module.ts. Added a minimal grid layout. Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTEZON-102 (#60) * BETTERZON-31, dependencies. * BETTERZON-31: Fixing dependencies * BETTERZON-31, BETTERZON-50 info popover and footer had been changed. * BETTERZON-74 simple top-bar has been created. * WIP: creating footer using grid. * BETTERZON-78 adding bottom bar and top bar * Adding cookieconsent as dependency again since it was removed by a merge * Adding cookieconsent as dependency again since it was removed by a merge * Apply suggestions from code review Switching from single to double quotes * BETTERZON-78 - grid added, structured as in Adobe XD mockup * wip: component rewritten, simple grid applied. Co-authored-by: Patrick Müller <patrick@mueller-patrick.tech> Co-authored-by: Patrick <50352812+Mueller-Patrick@users.noreply.github.com> * BETTERZON-113, BETTERZON-114, BETTERZON-115: Adding API endpoint for favorite shops (#61) * BETTERZON-116: Adding API endpoint for searching a new product (#62) * BETTERZON-117: Adding API endpoint for getting the latest crawling status (#63) * BETTERZON-111: Adding service functions for login and registration (#64) * BETTERZON-112: Adding service functions for managing vendor shops (#65) * BETTERZON-118: Adding service functions for managing favorite shops (#66) Co-authored-by: henningxtro <sextro.henning@student.dhbw-karlsruhe.de> Co-authored-by: root <root@DESKTOP-ARBPL82.localdomain> Co-authored-by: Reboooooorn <61185041+Reboooooorn@users.noreply.github.com>
2021-05-29 08:58:27 +00:00
beautifulsoup4
requests
lxml
Reference in New Issue Copy Permalink