Commit Graph

Select branches

Hide Pull Requests

main

#1

#10

#2

#3

#8

#9

v0.1.0

v0.1.1

v0.1.2

v0.1.3

Mono Color

• 58815d3ae5 Add worker count and type database connections. main Joshua Potter 2023-12-09 16:53:17 -0700
• 3ab4f893b7 Add logging around failed network requests. Joshua Potter 2023-12-09 15:39:56 -0700
• 811cf997d1 Update app name and version. v0.1.3 Joshua Potter 2023-12-08 17:48:54 -0700
• 8f21cb64af Add random position. Joshua Potter 2023-12-07 08:08:53 -0700
• 44a18fc59c Add language detection for chesscom profiles. Joshua Potter 2023-12-06 20:53:54 -0700
• f2fd289225 Remove activity download. Joshua Potter 2023-12-06 20:11:54 -0700
• 47e8d245c3 Scrape titles. Joshua Potter 2023-12-06 19:52:40 -0700
• 0b9a721368 Maintain order on languages. Joshua Potter 2023-12-05 16:06:04 -0700
• f20fc76081 Load languages into the database. Joshua Potter 2023-12-05 15:15:42 -0700
• ef5d296097
  Scrape languages from lichess listing. (#10) Joshua Potter 2023-12-05 14:20:46 -0700
• 2de54f05df Scrape languages from lichess listing. Joshua Potter 2023-12-05 14:05:59 -0700
• 82dbef21b6
  Fix all mypy warnings. (#9) Joshua Potter 2023-12-05 12:54:12 -0700
• 6cf6591180 Fix all mypy warnings. Joshua Potter 2023-12-05 12:17:07 -0700
• 8d7f1e7c4a
  Scrape content into an asynchronous pipeline. (#8) Joshua Potter 2023-12-05 11:43:13 -0700
• 7bd1b83994 Better readme, fix key error. Joshua Potter 2023-12-05 11:36:57 -0700
• 19f900484d Have lichess work as well. Joshua Potter 2023-12-05 11:31:31 -0700
• 0b5dbf3da4 Have chesscom pipeline results. Joshua Potter 2023-12-05 11:21:54 -0700
• 97d539ddb8 Delete unused and thread better. Joshua Potter 2023-12-05 10:40:48 -0700
• 714ef788d4 Add asynchronous processing flow. Joshua Potter 2023-12-05 10:23:08 -0700
• de8d655f2f Add cleaner pipeline interface. Joshua Potter 2023-12-05 10:11:46 -0700
• 63764a22c4 Transition to a CSV; Postgres can handle that better. Joshua Potter 2023-12-04 15:08:17 -0700
• ec94a16140 Re-order consistently. Joshua Potter 2023-12-04 08:17:24 -0700
• 1c0dc05b42 Separate initialization from loading. Prefer upserts. Joshua Potter 2023-12-04 08:14:33 -0700
• 9a5274f0dd Use more robust pre-commit. Joshua Potter 2023-12-03 14:31:36 -0700
• b4e42e6de2 Simplify pre-commit. Joshua Potter 2023-12-03 11:30:14 -0700
• 9b9e561e49 Apply pyls-isort. Joshua Potter 2023-12-01 16:36:53 -0700
• a4b1647e53 Allow specifying multiple sites in command line. Joshua Potter 2023-12-01 10:47:20 -0700
• 0c4e008b45
  Rewrite export as NDJSON and include script to load result into postgres. (#3) Joshua Potter 2023-12-01 10:30:44 -0700
• 9c4d4f1ecd Explanation on E2E. Joshua Potter 2023-12-01 10:29:25 -0700
• fcb7b961f5 Allow loading exported data into database. Joshua Potter 2023-12-01 10:14:55 -0700
• 9b81105a5e Use lxml to speed up parsing. Joshua Potter 2023-12-01 07:12:40 -0700
• d549e5f5eb Export blitz and bullet ratings. Joshua Potter 2023-12-01 07:10:58 -0700
• 36d471e395 Export rapid ratings. Joshua Potter 2023-11-30 20:35:20 -0700
• e050d13aa7 Add class for wrapping around exports. Joshua Potter 2023-11-30 17:12:16 -0700
• bc2ffeae9d
  Add a scraper for lichess. (#2) v0.1.2 Joshua Potter 2023-11-30 15:36:44 -0700
• 43fdf7c87c Add lichess scraper. Joshua Potter 2023-11-30 15:35:49 -0700
• aeec322c7f Add lichess scraper. Joshua Potter 2023-11-30 15:28:07 -0700
• 67a9d84df5 Update README. Joshua Potter 2023-11-30 15:20:08 -0700
• 10801b560c
  Generalize in anticipation of merging the lichess scraper. (#1) Joshua Potter 2023-11-30 15:15:15 -0700
• a58f604a5e Ensure exporting works. Joshua Potter 2023-11-30 15:14:43 -0700
• 7454016ee8 Add better logging and cleaner bash commands. Joshua Potter 2023-11-30 15:10:03 -0700
• d86212a390 Abstract original scraper into scraper class. Joshua Potter 2023-11-30 13:54:36 -0700
• 54b0ce84b0 Setup main as primary entrypoint. Joshua Potter 2023-11-30 13:05:47 -0700
• d4d93c3e31 Add a general `Scraper` class. Joshua Potter 2023-11-30 12:57:01 -0700
• 3cc31f8f24 Add guard on failed page download. v0.1.1 Joshua Potter 2023-11-28 07:57:05 -0700
• fe2e504de9 Package into app for `nix build`. v0.1.0 Joshua Potter 2023-11-28 05:28:21 -0700
• 27577aa5b0 Add `mypy` and lock poetry dependencies. Joshua Potter 2023-11-28 04:54:57 -0700
• 99c89a3a6d Restructure and add documentation. Require specifying user-agent. Joshua Potter 2023-11-27 20:04:47 -0700
• 7308c47eb5 Have requests actually use the `chesscom_request` function. Joshua Potter 2023-11-27 14:28:10 -0700
• 97cdb3b9cd Add User-Agent to requests. Joshua Potter 2023-11-27 14:13:56 -0700
• 1a4199e95d Add README descriptions. Joshua Potter 2023-11-27 14:13:46 -0700
• 932d97bf3b Create intermediate directories and explicit f-strings. Joshua Potter 2023-11-27 13:53:51 -0700
• 659988355c Fix classname. Joshua Potter 2023-11-27 13:41:06 -0700
• 23aa622126 Break coach listing download per-page. Joshua Potter 2023-11-27 13:30:22 -0700
• 1710e1aefa Initial commit. Joshua Potter 2023-11-27 13:09:40 -0700