1
0
Fork 0
forked from blallo/rss-bridge
Commit graph

53 commits

Author SHA1 Message Date
LogMANOriginal
0d80a19e84
[FacebookBridge] Add context for public Facebook groups (#739)
The previous context is now labeled 'User', while the new context is
labeled 'Group'. The existing code was not changed, instead new group*
functions were implemented to handle groups.

The general principle of capturing groups is the same as done for users
with adjustments to account for different HTML structures.

Captcha responses are currently not supported for groups! There doesn't
seem to be a way to trigger them consistently, which makes it hard to
handle them properly.

Features of the group context:

- The feed title is based on the group name
- The group URI used for capturing is returned for the feed URI
- Author names and timestamps are reproduced from the source
- Post titles are reproduced from the source if they exist, otherwise
the title is build manually from the author name and the content
- Original contents are included with the feed
- All images are attached as enclosures as well

Closes #
2018-07-08 17:16:00 +02:00
LogMANOriginal
193ca87afa [phpcs] enforce single quotes (#732)
* [phpcs] Add rule to enforce single quoted strings
2018-06-29 22:55:33 +01:00
logmanoriginal
5087f5f79e [FacebookBridge] Support facebook links as user name
Allows users to paste facebook links as user name. The link must contain
the correct host (www.facebook.com) and a valid path (/user-name/...).
The first part of the path is used for the user name. Errors are returned
in case something went wrong.

References #706
2018-06-24 11:14:08 +02:00
logmanoriginal
4a5f190e0e [FacebookBridge] Add option to skip reviews
Reviews are provided the same way as summary posts and therefore returned
as separate feed item for each review. This commit adds a new option
'&skip_reviews=on' to skip reviews entirely.

References #706
2018-06-24 10:52:22 +02:00
teromene
79ebdc4b39 Warn the user when trying to fetch a non-public facebook page. 2018-05-05 13:49:49 +01:00
logmanoriginal
6caca4946b bridges: Fix bridges with custom headers and options
This commit fixes bridges which called getContents, getSimpleHTMLDOM
or getSimpleHTMLDOMCached with custom settings.
2018-04-06 20:42:19 +02:00
logmanoriginal
df6da837dc [FacebookBridge] Return error if username starts with slash
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.

For example username "/test" would result in:
https://facebook.com//test

References #628
2018-03-23 21:23:30 +01:00
Tameroski
be03764029 Fixing double quote issue at the end of URL (#623) 2018-01-23 11:27:45 +00:00
logmanoriginal
99e7e7876e exception: Use built-in HTTP response codes
PHP >= 5.4 provides a built-in function to generate valid HTTP
error header including the error description: http_response_code()

See: http://php.net/manual/en/function.http-response-code.php
See also: https://stackoverflow.com/a/12018482

This commit removes the '\Http' utility class and replaces all
calls to 'Http::getMessageForCode()' by 'http_response_code()'
2017-08-06 12:55:11 +02:00
logmanoriginal
cbda060b86 [FacebookBridge] Fix & in URLs
All formats except HTML return & instead of & in URLs causing
all links with parameters (...&id=...) to break.

Facebook does not return valid HTML URIs but instead provides them
with all special characters encoded (like using htmlspecialchars).
This seems to be related to the page being build almost entirely of
script blocks.

This commit adds htmlspecialchars_decode() to URI and content to
reverse the encoding.

References #550
2017-08-04 21:12:48 +02:00
logmanoriginal
a4b9611e66 [phpcs] Add missing rules
- Do not add spaces after opening or before closing parenthesis

  // Wrong
  if( !is_null($var) ) {
    ...
  }

  // Right
  if(!is_null($var)) {
    ...
  }

- Add space after closing parenthesis

  // Wrong
  if(true){
    ...
  }

  // Right
  if(true) {
    ...
  }

- Add body into new line
- Close body in new line

  // Wrong
  if(true) { ... }

  // Right
  if(true) {
    ...
  }

Notice: Spaces after keywords are not detected:

  // Wrong (not detected)
  // -> space after 'if' and missing space after 'else'
  if (true) {
    ...
  } else{
    ...
  }

  // Right
  if(true) {
    ...
  } else {
    ...
  }
2017-07-29 19:55:12 +02:00
logmanoriginal
6e4bc341b7 [FacebookBridge] Replace 'novideo' with 'media_type'
This replaces the 'novideo' parameter with 'media_type' in order
to filter for specific content types. Currently supported:

- 'all': Returns all posts (default)
- 'video': Returns only posts including videos
- 'novideo': Returns only posts that don't include videos

References #553
2017-07-25 16:04:21 +02:00
logmanoriginal
fa2df09b1b [FacebookBridge] Add option to hide posts with facebook videos
This adds a new option 'novideo' that can be set to 'on' or 'off'
in order to skip posts that include facebook videos (does not work
for linked videos like YouTube). This option is 'off' by default.

References #533
2017-07-25 15:41:05 +02:00
logmanoriginal
8ed4812e00 [FacebookBridge] Add requester languages to HTTP header
If no accepted languages are specified Facebook will guess your
language. This guess can go horribly wrong if your server does not
provide origin information.

This adds a context header with language information when retrieving
page contents. The accepted languages are read from the list of
accepted languages specified by the web browser of the requester.

References #530
2017-05-07 13:27:37 +02:00
logmanoriginal
e221358ead [FacebookBridge] Handle summary posts
Previously summary posts were ignored which resulted in the last
two posts not showing up in the feed (the latest two are shown in
the summary post).

Now summary posts are treated like regular posts, returning them
as part of the regular feed.

References #502, #505
2017-04-10 13:04:41 +02:00
logmanoriginal
c4169f1579 bridges: Return parent::getName by default 2017-02-15 19:38:32 +01:00
logmanoriginal
6f24858124 bridges: Fix coding styles
This commit is a squash of all commits that fix coding styles
for the new coding style policy.

[ABCTabsBridge] Fix coding style
[AcrimedBridge] Fix coding style
[AllocineFRBridge] Fix coding style
[AnimeUltimeBridge] Fix coding style
[Arte7Bridge] Fix coding style
[AskfmBridge] Fix coding style
[BandcampBridge] Fix coding style
[BastaBridge] Fix coding style
[BlaguesDeMerdeBridge] Fix coding style
[BooruprojectBridge] Fix coding style
[CADBridge] Fix coding style
[CNETBridge] Fix coding style
[CastorusBridge] Fix coding style
[CollegeDeFranceBridge] Fix coding style
[CommonDreamsBridge] Fix coding style
[CopieDoubleBridge] Fix coding style
[CourrierInternationalBridge] Fix coding style
[CpasbienBridge] Fix coding style
[CryptomeBridge] Fix coding style
[DailymotionBridge] Fix coding style
[DanbooruBridge] Fix coding style
[DansTonChatBridge] Fix coding style
[DauphineLibereBridge] Fix coding style
[DeveloppezDotComBridge] Fix coding style
[DemoBridge] Fix coding style
[DilbertBridge] Fix coding style
[DuckDuckGoBridge] Fix coding style
[DollbooruBridge] Fix coding style
[EliteDangerousGalnetBridge] Fix coding style
[ElsevierBridge] Fix coding style
[EstCeQuonMetEnProdBridge] Fix coding style
[EZTVBridge] Fix coding style
[FacebookBridge] Fix coding style
[FeedExpanderExampleBridge] Fix coding style
[FB2Bridge] Fix coding style
[FierPandaBridge] Fix coding style
[FlickrBridge] Fix coding style
[FootitoBridge] Fix coding style
[FourchanBridge] Fix coding style
[FuturaSciencesBridge] Fix coding style
[GBAtempBridge] Fix coding style
[GelbooruBridge] Fix coding style
[GiphyBridge] Fix coding style
[GithubIssueBridge] Fix coding style
[GizmodoBridge] Fix coding style
[GoComicsBridge] Fix coding style
[GooglePlusPostBridge] Fix coding style
[GoogleSearchBridge] Fix coding style
[HDWallpapersBridge] Fix coding style
[HentaiHavenBridge] Fix coding style
[IdenticaBridge] Fix coding style
[InstagramBridge] Fix coding style
[IsoHuntBridge] Fix coding style
[JapanExpoBridge] Fix coding style
[KonachanBridge] Fix coding style
[KoreusBridge] Fix coding style
[KununuBridge] Fix coding style
[LeBonCoinBridge] Fix coding style
[LegifranceJOBBridge] Fix coding style
[LeMondeInformatiqueBridge] Fix coding style
[LesJoiesDuCodeBridge] Fix coding style
[LichessBridge] Fix coding style
[LinkedInCompanyBridge] Fix coding style
[LolibooruBridge] Fix coding style
[LWNprevBridge] Fix coding style
[MangareaderBridge] Fix coding style
[MilbooruBridge] Fix coding style
[MixCloudBridge] Fix coding style
[MoebooruBridge] Fix coding style
[MondeDiploBridge] Fix coding style
[MsnMondeBridge] Fix coding style
[MspabooruBridge] Fix coding style
[NasaApodBridge] Fix coding style
[NeuviemeArtBridge] Fix coding style
[NextgovBridge] Fix coding style
[NextInpactBridge] Fix coding style
[NiceMatinBridge] Fix coding style
[NovelUpdatesBridge] Fix coding style
[OpenClassroomsBridge] Fix coding style
[ParuVenduImmoBridge] Fix coding style
[PickyWallpapersBridge] Fix coding style
[PinterestBridge] Fix coding style
[PlanetLibreBridge] Fix coding style
[ReadComicsBridge] Fix coding style
[Releases3DSBridge] Fix coding style
[ReporterreBridge] Fix coding style
[RTBFBridge] Fix coding style
[Rue89Bridge] Fix coding style
[Rule34Bridge] Fix coding style
[Rule34pahealBridge] Fix coding style
[SafebooruBridge] Fix coding style
[SakugabooruBridge] Fix coding style
[ScmbBridge] Fix coding style
[ScoopItBridge] Fix coding style
[SensCritiqueBridge] Fix coding style
[SexactuBridge] Fix coding style
[ShanaprojectBridge] Fix coding style
[Shimmie2Bridge] Fix coding style
[SoundcloudBridge] Fix coding style
[StripeAPIChangeLogBridge] Fix coding style
[SuperbWallpapersBridge] Fix coding style
[T411Bridge] Fix coding style
[TagBoardBridge] Fix coding style
[TbibBridge] Fix coding style
[TheCodingLoveBridge] Fix coding style
[TheHackerNewsBridge] Fix coding style
[ThePirateBayBridge] Fix coding style
[TheTVDBBridge] Fix coding style
[Torrent9Bridge] Fix coding style
[TwitterBridge] Fix coding style
[UnsplashBridge] Fix coding style
[ViadeoCompanyBridge] Fix coding style
[VineBridge] Fix coding style
[VkBridge] Fix coding style
[WallpaperStopBridge] Fix coding style
[WebfailBridge] Fix coding style
[WeLiveSecurityBridge] Fix coding style
[WhydBridge] Fix coding style
[WikipediaBridge] Fix coding style
[WordPressBridge] Fix coding style
[WorldOfTanksBridge] Fix coding style
[XbooruBridge] Fix coding style
[YandereBridge] Fix coding style
[YoutubeBridge] Fix coding style
[ZDNetBridge] Fix coding style
2017-02-12 15:34:08 +01:00
Pellaeon Lin
1028e538ab Fix duplicate https://www.facebook.com/ on captcha_action POST URL (#466) 2017-02-02 16:18:23 +00:00
Pellaeon Lin
49cc0661ad Fix FacebookBridge feed name empty when data loaded from cache (#456) 2017-02-02 16:17:18 +00:00
Pellaeon Lin
3109694b1c Ignore summary posts generated by facebook (#467) 2017-02-02 16:13:26 +00:00
Pellaeon Lin
9c1bedb33f Preserve br and p from Facebook post content 2017-01-13 06:38:05 +08:00
logmanoriginal
970e216c1f Merge branch 'methods2functions' of https://framagit.org/peetah/rss-bridge 2016-10-02 16:05:58 +02:00
Pierre Mazière
f1fb95b257 [core] extract BridgeAbstract methods to make them functions
- returnError, returnServerError, returnClientError ,debugMessage are
  moved to lib/error.php

- getContents, getSimpleHTMLDOM, getSimpleHTMLDOMCached are moved to
  lib/contents.php

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-25 23:22:33 +02:00
Pierre Mazière
235c985901 [BridgeAbstract] replace getCacheDuration method by CACHE_TIMEOUT class constant
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-25 17:04:28 +02:00
Pierre Mazière
9a0da733ef [bridges] use constants instead of variable members
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-01 23:15:51 +02:00
Pierre Mazière
8c7b53b883 [FacebookBridge] fix + code simplification
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 14:24:17 +02:00
Pierre Mazière
1b3c8a8aeb [core + bridges] add BridgeAbstract::$inputs and BridgeAbstract::getInput()
Inputs are not stored in BridgeAbstract::$parameters anymore to separate
static data from dynamic data.
The getInput method allows for more readable code.

Also fix an "undefined index 'global'" notice

Probability of breaking bridges: high !

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
de1b39c8e5 [core + bridges] get rid of loadMetadata
if a bridge needs to modify some of the data that were initialized
there, ::__construct() should be used instead.

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
117031bf0f [core] store parameters values in BridgeAbstract::parameters
This way, any BridgeAbstract method can now have access to these values,
no only collectData

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-25 01:24:53 +02:00
Pierre Mazière
4a1f360d39 [bridges] use BridgeAbstract::getContents instead of file_get_contents
use of the proxy feature when enabled

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-22 23:39:40 +02:00
logmanoriginal
bf0a9d754e [bridges] Change all occurrences of the Item object to array 2016-08-22 18:55:59 +02:00
Pierre Mazière
f0e502ce37 [bridges] migrate all bridges to an array based definition of parameters
see github issue 356

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-22 01:25:56 +02:00
Pierre Mazière
bba216073c [bridges] remove useless 'update' property from all bridges
That's the source code manager job to keep this information
consistent

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-21 00:21:41 +02:00
Pierre Mazière
3c0d13c1bb bridges: use BridgeAbstract::getSimpleHTMLDOM
instead of BridgeAbstract::file_get_html

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-19 10:20:27 +02:00
logmanoriginal
74f0572d91 bridges: Replace returnError function with more specific
Replacements depend on original error code:
400: returnClientError
404: returnServerError
500: returnServerError
501: returnServerError
2016-08-17 14:45:08 +02:00
logmanoriginal
fa59a25229 bridges: Update bridges with 'required' attributes 2016-08-15 01:19:16 +02:00
logmanoriginal
42b6c82753 bridges: Cleanup occurrences of $item->thumbnailUri
This is a subsequent change to f3eefab
2016-08-09 15:50:55 +02:00
logmanoriginal
0ddf664d89 bridges: Remove obsolete getName() and getURI() 2016-08-06 18:45:02 +02:00
Pierre Mazière
955eecc299 use BridgeAbstract::file_get_html in all bridges
instead of simple_html_dom function file_get_html

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-06-25 23:17:42 +02:00
ORelio
96de64af63 [Facebook] Update field retrieval
Due to changes on their side.
Related issue: #253
2016-06-12 22:10:59 +02:00
logmanoriginal
3288328ff7 [Facebook] Update date in loadMetadatas 2016-03-31 11:45:35 +02:00
logmanoriginal
e62c6af62e [Facebook] Select page content one level further down
This fixes loading issues where feeds appeared empty.
2016-03-31 11:33:38 +02:00
ORelio
d16d514736 [Facebook] Capcha retrieval and response submit
This does not solve the captcha issue but allows the viewer to manually
solve the captcha by displaying a form and using the response from the
viewer. Maybe a first step to automated captcha solving?

This process relies on the use of a PHP session for storing captcha
details so that the user cannot submit anything else than the response
to the captcha. Response is filtered before being forwarded, also. Once
captcha is solved we get a page ready to be parsed, as usual.

Combined to some kind of OCR, this could automatically solve the
captcha, but currently if only automates the process of retrieving the
challenge and submitting the response.
2016-01-23 21:53:25 +01:00
Teromene
e582c887fb Ajout du paramètre pattern pour les entrées de type texte, ajout du required côté HTML.
Correction de la page d'accueuil pour qu'elle soit conforme aux standards du W3C.
Correction de la regex de listage des fichiers pour qu'elle ignore les sauvegardes.
Ajout d'un nettoyeur HTML, par défaut.
2015-12-06 15:56:39 +01:00
teromene
78c9dcc705 Last batch of bridges, all implemented !
Signed-off-by: teromene <teromene@teromene.fr>
2015-12-06 15:56:39 +01:00
teromene
5d2e7ac83d Ajout d'options multiples pour les méta-données.
Ajout d'un bridge de démonstration, DemoBridge.
Ajout d'un début de documentation pour créer un bridge dans CREATE_BRIDGE.md
2015-12-06 15:56:39 +01:00
ORelio
b592f8ebbf [Facebook] Convert textual emoticons to ASCII
Currently emoticons are retrived in textual form eg <i><u>smile
emoticon</u></i> which is not really visual... so let's convert them back
as ASCII emoticons eg ':)'. This works using a hardcoded table mapping
emoticon names to their visual representation, and the regular expression
match the two words because eg in french facebook will display
<i><u>émoticône smile</u></i> so we need to test both. Unknown emoticon
descriptions will be left as is.
2015-10-24 20:14:12 +02:00
ORelio
300b8bba9b [Facebook] More cleaning and bypass link redirect
Remove a bunch of useless html node properties
Bypass all l.php facebook link redirections
Restore <u> and <i> basic formatting
2015-10-23 12:18:55 +02:00
ORelio
d7436c2d0a Improve Facebook Bridge
Retrieve author display name from page title
Build short readable title using name and content
Convert relative links into absolute links
Remove attributes from </a> tags (may cause issues)
Remove onclick and onmouseover (javascript code)
Retrieve url of first image of post as thumnail
Use author avatar as thumnail if no first image
Use display name in feed name: name - Fb bridge
Minor code indent fixes and use simple quotes
2015-09-05 14:42:54 +02:00
teromene
ee4d4d9603 Ajout du support des journaux de profils 2015-09-02 13:49:36 +02:00