Commit graph

162 commits

Author SHA1 Message Date
pezcurrel
5312aea0cc Added writing server rules in the db 2022-12-26 05:08:17 +01:00
pezcurrel
337eb32f51 Made mail optional, inactive by default 2022-12-25 23:45:31 +01:00
pezcurrel
429ab42ff5 Added 2 debug messages stating how mani dead instances the script got from Instances and Peers tables 2022-12-25 18:55:54 +01:00
pezcurrel
119c9119c2 Made the logic for “deadline” much more terse 2022-12-25 18:41:13 +01:00
pezcurrel
acde202b2e Made it write summary from crawl run into a log file of its own 2022-12-25 18:40:21 +01:00
pezcurrel
ba171bd5f2 Use specifically bash since we use &> redirect 2022-12-25 11:43:30 +01:00
pezcurrel
ec9b65e42f Little script to run peerscrawl.php in loop; first commit 2022-12-25 11:32:49 +01:00
pezcurrel
10e2e1b58a Added “lecho” for “message levels”, removed “gecho”, removed “verbose” option; removed “loop” option (do loop from a shell script if needed) 2022-12-25 11:32:08 +01:00
pezcurrel
1d0c6b799a Small edit to “logminmsglev” and “tuiminmsglev” TUI option parsing errors 2022-12-25 11:29:34 +01:00
pezcurrel
d95bc70b8a Exposed “deadline” option; minor changes 2022-12-25 09:47:04 +01:00
pezcurrel
c0802de828 Removed “restore” option: could work, but it’s not very useful and would require a big hassle; added loops and new found instances counters; made sighandler use mexit 2022-12-25 09:24:23 +01:00
pezcurrel
d6b77b0e29 Removed option “-p peers” from crawler cmdline because now peerscrawl directly writes new instances into the db 2022-12-24 08:59:33 +01:00
pezcurrel
9fabb3853b Infatti 2022-12-23 19:13:37 +01:00
pezcurrel
96aa6f3aa9 Quella roba lì 2022-12-23 19:12:18 +01:00
pezcurrel
05fed0142c Lowered a bit default values for “timeout” and “curltimeout” 2022-12-23 11:23:32 +01:00
pezcurrel
89a2ea0b26 Fixed “trending tags” ordering and fetching 2022-12-23 11:22:25 +01:00
pezcurrel
edee66b834 Temporarily disabled “restore” option because it needs more work to actually work 2022-12-22 15:32:30 +01:00
pezcurrel
61d0fcb3d8 Added “loop” option allowing to run the crawl in an infinite loop or until sig(int|hup|term) received; other minor changes 2022-12-22 15:05:55 +01:00
pezcurrel
6477e8812f Exposed “curltimeout” option; changed “timeout” default from 5 to 10; changed “curltimeout” default from 10 to 20 2022-12-22 14:24:48 +01:00
pezcurrel
6d4ce26f98 Adapted “restore” code to the new workings; minor changes and fixes 2022-12-22 14:04:29 +01:00
pezcurrel
c27053314a Added code to store and consider “instance checks” made by the script to independently mark peers ad dead 2022-12-22 11:32:18 +01:00
pezcurrel
706c831e23 Little change in a message 2022-12-22 11:28:29 +01:00
pezcurrel
f8cdf2cf3b Changed check against “activity” values, which are strings, not integers 2022-12-22 07:40:41 +01:00
pezcurrel
c6c3feb500 Removed leftovers of “jsonwrite” option 2022-12-22 07:05:21 +01:00
pezcurrel
277296512c Explicitly set idn_to_ascii flags, otherwise with php 7.3 it complained 2022-12-21 22:15:40 +01:00
pezcurrel
9316e686b9 Bir rewrite, made it shorter and hopefully a bit more readable 2022-12-21 22:07:05 +01:00
pezcurrel
732ea79480 Moved $mastodons definition upper 2022-12-21 22:06:10 +01:00
pezcurrel
1c524ffd69 Moved mysqli_close after the optional loading of dead instances from the db; renamed $eta to $tet 2022-12-21 22:05:15 +01:00
pezcurrel
0d74dbf243 Got rid of akeavinn; other minor changes 2022-12-21 07:54:11 +01:00
pezcurrel
2c86580bfb The regexp to decide whether an instance is Mastodon or not is now based on the Platforms table; made ckratelimit more precise about possible missing headers; added code to set Version from /api/v1/instance when it was not already set from nodeinfo 2022-12-21 06:53:31 +01:00
pezcurrel
4fdf287686 Little change in delete prompt 2022-12-20 23:02:53 +01:00
pezcurrel
d8f15f4b3a Does delete from UserFields much faster 2022-12-20 23:01:13 +01:00
pezcurrel
f6dc080ed6 Major rewrite: it was a mess, now it is less :-) 2022-12-20 23:00:22 +01:00
pezcurrel
f30bcc7e5e Updated to new delinstbyuid function form 2022-12-18 18:43:09 +01:00
pezcurrel
e744ae9d20 Fixed validhostname; added and using myq function for mysql queries; changed some exit codes; made connection error management better 2022-12-18 18:42:11 +01:00
pezcurrel
f29f636a70 Added mysql error code to error message that is echoed if a query fails 2022-12-18 18:40:22 +01:00
pezcurrel
cd16eabce5 Using new form of delinstbyuid function; removed references to New column 2022-12-18 18:27:22 +01:00
pezcurrel
835e02c171 Removed code referencing New, Good and Chosen columns 2022-12-18 18:26:03 +01:00
pezcurrel
f4aa3cb804 A script to set InsertTS field to something suitable when it is null 2022-12-18 18:24:35 +01:00
pezcurrel
c5debcb463 A function to delete an Instance record by ID, and all references to it in other tables; first commit 2022-12-18 18:23:52 +01:00
pezcurrel
ccc9f517fd Moved code to delete an Instances record and all its references in other tables to function delinstbyid in lib/delinstbyid.php; minor changes 2022-12-18 11:44:11 +01:00
pezcurrel
2b0e2398ae Made validhostname accept only valid hostnames :-)) (no ports or path specs) 2022-12-18 11:42:32 +01:00
pezcurrel
32251d1ba8 Added “deleteinstswhere” action 2022-12-18 11:41:09 +01:00
pezcurrel
7d2875075b Deleted 2022-12-18 09:35:24 +01:00
pezcurrel
690b54521b Moved 2022-12-18 09:35:04 +01:00
pezcurrel
f269bb901d Little cosmetic change 2022-12-18 07:00:49 +01:00
pezcurrel
e9b88d6735 Made $jsonfp be written into run dir 2022-12-18 07:00:19 +01:00
pezcurrel
a3ada274e7 Removed stdout/err redirect in cmd; passing proper descriptor and pipe to proc_open; minor changes 2022-12-18 06:59:25 +01:00
pezcurrel
a32a25e095 Many many changes :-)) 2022-12-18 00:34:27 +01:00
pezcurrel
d6dd03694c Removed $context 2022-12-17 22:54:32 +01:00