Commit graph

154 commits

Author SHA1 Message Date
pezcurrel
1d0c6b799a Small edit to “logminmsglev” and “tuiminmsglev” TUI option parsing errors 2022-12-25 11:29:34 +01:00
pezcurrel
d95bc70b8a Exposed “deadline” option; minor changes 2022-12-25 09:47:04 +01:00
pezcurrel
c0802de828 Removed “restore” option: could work, but it’s not very useful and would require a big hassle; added loops and new found instances counters; made sighandler use mexit 2022-12-25 09:24:23 +01:00
pezcurrel
d6b77b0e29 Removed option “-p peers” from crawler cmdline because now peerscrawl directly writes new instances into the db 2022-12-24 08:59:33 +01:00
pezcurrel
9fabb3853b Infatti 2022-12-23 19:13:37 +01:00
pezcurrel
96aa6f3aa9 Quella roba lì 2022-12-23 19:12:18 +01:00
pezcurrel
05fed0142c Lowered a bit default values for “timeout” and “curltimeout” 2022-12-23 11:23:32 +01:00
pezcurrel
89a2ea0b26 Fixed “trending tags” ordering and fetching 2022-12-23 11:22:25 +01:00
pezcurrel
edee66b834 Temporarily disabled “restore” option because it needs more work to actually work 2022-12-22 15:32:30 +01:00
pezcurrel
61d0fcb3d8 Added “loop” option allowing to run the crawl in an infinite loop or until sig(int|hup|term) received; other minor changes 2022-12-22 15:05:55 +01:00
pezcurrel
6477e8812f Exposed “curltimeout” option; changed “timeout” default from 5 to 10; changed “curltimeout” default from 10 to 20 2022-12-22 14:24:48 +01:00
pezcurrel
6d4ce26f98 Adapted “restore” code to the new workings; minor changes and fixes 2022-12-22 14:04:29 +01:00
pezcurrel
c27053314a Added code to store and consider “instance checks” made by the script to independently mark peers ad dead 2022-12-22 11:32:18 +01:00
pezcurrel
706c831e23 Little change in a message 2022-12-22 11:28:29 +01:00
pezcurrel
f8cdf2cf3b Changed check against “activity” values, which are strings, not integers 2022-12-22 07:40:41 +01:00
pezcurrel
c6c3feb500 Removed leftovers of “jsonwrite” option 2022-12-22 07:05:21 +01:00
pezcurrel
277296512c Explicitly set idn_to_ascii flags, otherwise with php 7.3 it complained 2022-12-21 22:15:40 +01:00
pezcurrel
9316e686b9 Bir rewrite, made it shorter and hopefully a bit more readable 2022-12-21 22:07:05 +01:00
pezcurrel
732ea79480 Moved $mastodons definition upper 2022-12-21 22:06:10 +01:00
pezcurrel
1c524ffd69 Moved mysqli_close after the optional loading of dead instances from the db; renamed $eta to $tet 2022-12-21 22:05:15 +01:00
pezcurrel
0d74dbf243 Got rid of akeavinn; other minor changes 2022-12-21 07:54:11 +01:00
pezcurrel
2c86580bfb The regexp to decide whether an instance is Mastodon or not is now based on the Platforms table; made ckratelimit more precise about possible missing headers; added code to set Version from /api/v1/instance when it was not already set from nodeinfo 2022-12-21 06:53:31 +01:00
pezcurrel
4fdf287686 Little change in delete prompt 2022-12-20 23:02:53 +01:00
pezcurrel
d8f15f4b3a Does delete from UserFields much faster 2022-12-20 23:01:13 +01:00
pezcurrel
f6dc080ed6 Major rewrite: it was a mess, now it is less :-) 2022-12-20 23:00:22 +01:00
pezcurrel
f30bcc7e5e Updated to new delinstbyuid function form 2022-12-18 18:43:09 +01:00
pezcurrel
e744ae9d20 Fixed validhostname; added and using myq function for mysql queries; changed some exit codes; made connection error management better 2022-12-18 18:42:11 +01:00
pezcurrel
f29f636a70 Added mysql error code to error message that is echoed if a query fails 2022-12-18 18:40:22 +01:00
pezcurrel
cd16eabce5 Using new form of delinstbyuid function; removed references to New column 2022-12-18 18:27:22 +01:00
pezcurrel
835e02c171 Removed code referencing New, Good and Chosen columns 2022-12-18 18:26:03 +01:00
pezcurrel
f4aa3cb804 A script to set InsertTS field to something suitable when it is null 2022-12-18 18:24:35 +01:00
pezcurrel
c5debcb463 A function to delete an Instance record by ID, and all references to it in other tables; first commit 2022-12-18 18:23:52 +01:00
pezcurrel
ccc9f517fd Moved code to delete an Instances record and all its references in other tables to function delinstbyid in lib/delinstbyid.php; minor changes 2022-12-18 11:44:11 +01:00
pezcurrel
2b0e2398ae Made validhostname accept only valid hostnames :-)) (no ports or path specs) 2022-12-18 11:42:32 +01:00
pezcurrel
32251d1ba8 Added “deleteinstswhere” action 2022-12-18 11:41:09 +01:00
pezcurrel
7d2875075b Deleted 2022-12-18 09:35:24 +01:00
pezcurrel
690b54521b Moved 2022-12-18 09:35:04 +01:00
pezcurrel
f269bb901d Little cosmetic change 2022-12-18 07:00:49 +01:00
pezcurrel
e9b88d6735 Made $jsonfp be written into run dir 2022-12-18 07:00:19 +01:00
pezcurrel
a3ada274e7 Removed stdout/err redirect in cmd; passing proper descriptor and pipe to proc_open; minor changes 2022-12-18 06:59:25 +01:00
pezcurrel
a32a25e095 Many many changes :-)) 2022-12-18 00:34:27 +01:00
pezcurrel
d6dd03694c Removed $context 2022-12-17 22:54:32 +01:00
pezcurrel
441d16a42d ckratelimit goes to sleep only when x-ratelimit-remaining==0; can spit debug info; limit fetching chunks from users directories is now 40 2022-12-17 22:54:02 +01:00
pezcurrel
ca4367b719 Removed the unlinking attempt at lockfp before exit: it was already done before by shutdown; other little changes about open files closing and the like 2022-12-17 18:43:13 +01:00
pezcurrel
e5ad18e619 Fixed a typo 2022-12-17 18:40:55 +01:00
pezcurrel
5c605cbe5b Some little cosmetic (readability of log files) changes 2022-12-17 18:40:22 +01:00
pezcurrel
2571396253 Tuned to recent changes in crawler.php (and getinstinfo.php) 2022-12-17 17:36:46 +01:00
pezcurrel
ad8fa26306 Made mysql connection and charset setting errors more graceful; added “users” page to updstats; other minor changes 2022-12-17 17:35:35 +01:00
pezcurrel
2d1d28b002 Fixed regexp checking if max_charcters is an integer; made mexit use eecho again, moving the closing of logf after eecho(s); made logf be opened only if logminmsglev < 4 2022-12-17 17:33:46 +01:00
pezcurrel
d1f088a026 Command for subprocesses gets now built on the fly using cmd function; logfile doesn’t get opened if logminmsglev < 4; other minor changes 2022-12-17 17:31:24 +01:00