27 commits

Author	SHA1	Message	Date
pezcurrel	2649e7d137	Info from nodeinfo didn’t end up into $info, now they do	2022-12-12 00:47:06 +01:00
pezcurrel	b16515f4e8	Lots of changes :-))	2022-12-11 23:29:51 +01:00
pezcurrel	61ad655a62	Disabled fetching profile’s page when “noindex” is not set in account because it takes too long; disabled featured tags fetching fro the same reason; other minor changes	2022-12-10 23:32:58 +01:00
pezcurrel	f343cb702e	Changed some eecho messages importance	2022-12-10 13:57:30 +01:00
pezcurrel	4b7f6a199c	Added truncs where needed; added code to check for “noindex” on user’s profile page when “noindex” is not set in accounts info	2022-12-10 12:35:22 +01:00
pezcurrel	18ce06871b	Added ckratelimit() where useful; made it more flexible with lowercasing every header key; more work on fetching users from users directories	2022-12-09 22:53:18 +01:00
pezcurrel	8341f0e209	Fixed a cosmetic bug; some more work into users directories fetching	2022-12-09 19:25:44 +01:00
pezcurrel	09765e566e	Added “normalization” of language codes (dash to underscore) and “pt_BR” language	2022-12-08 13:53:43 +01:00
pezcurrel	c3a90ba6b8	Adding “fetchuser” option; changed “timeout” default from 5 to 10 seconds	2022-12-08 00:03:10 +01:00
pezcurrel	a07e59d52c	Added option “--moreclauses”	2022-12-05 21:18:58 +01:00
pezcurrel	a9b5e72373	Added fedibird, ecko and hometown to the regex deciding whether an instance is mastodon or not	2022-12-03 14:44:59 +01:00
pezcurrel	c4b1a53439	Added German language (de)	2022-12-02 16:29:24 +01:00
pezcurrel	632202b69c	Added “Done” eecho in the end of run	2022-12-02 16:07:05 +01:00
pezcurrel	e194bd597f	Changed notifications “levels”, removed “eecho”es when “notify” is used since it “eecho”es by itself	2022-12-01 17:42:48 +01:00
pezcurrel	92cee555c7	Removed support to Blacklist table	2022-12-01 05:41:54 +01:00
pezcurrel	1eadd2f3ce	Removed “StartNodes” loading and “Noxious” table loading	2022-11-30 07:19:14 +01:00
pezcurrel	1a6446e82b	Don’t use “black” as derogatory (“blacklist” -> “list of noxious instances”, etc.)	2022-11-29 17:36:49 +01:00
pezcurrel	9b87447da0	Changed “langs” function so that, if “api/v1/instance” returned a language different from the default en, it assumes it is right, because it has been explicitly set, and avoid doing autodection of languages based on last toots	2022-11-23 20:35:53 +01:00
pezcurrel	86c26009e7	Added “NameGL” to the query in “langs” function (forgot it in previous commit)	2022-11-23 19:32:50 +01:00
pezcurrel	5421041a7c	Added gl, uk support into “langs” function	2022-11-23 19:30:32 +01:00
pezcurrel	87ff532831	Use gmmktime instead of mktime in pgdatetomy	2022-11-01 07:44:55 +01:00
pezcurrel	dcbd52a04b	Fixed a little flaw in get_toot_languages function	2022-05-06 09:21:56 +02:00
pezcurrel	136f99ba24	Rimossa l’opzione «-r»	2022-05-06 06:29:19 +02:00
pezcurrel	371e4c9a2e	...	2020-10-22 17:54:05 +02:00
pezcurrel	049c1d839a	...	2020-10-21 15:26:31 +02:00
pezcurrel	06f702ce71	...	2020-10-20 16:31:16 +02:00
pezcurrel	5ec1354546	...	2020-10-18 06:53:27 +02:00

Renamed from web/mustard/clitools/crawler.php (Browse further)