PhpBB3 MOD: phpBB spam hammer
From DP
This MOD for the phpBB3 forum stops almost all spam by disabling links and bad words in posts, messages, signatures, and profiles until new users reach a certain number of posts and/or age.
- Based on the existing work of Philthy and Skinny Vinny
- Thread in phpBB development forum
Contents |
Features
- Disables links and bad words in posts, messages, signatures, and profiles until a user reaches a specified age and/or number of posts
- Own-site links are excluded from the filter, other sites can be added to a whitelist
- Configurable list of forbidden filter words, optionally show the user the trigger term
- Configurable unicode filters to prevent spam in languages not used in the forum
- Prevents 'sleeper agents' by disabling posts, signatures, and profiles for old accounts with 0 posts (optional)
- Zombie registration cleanup deletes old users with no posts (optional)
- Protections are 'automatically' disabled after the criteria are met (no moderation required)
Configuration
This currently shows the variable in the class and documents what they do.
No link strings
// An array of no-nos. Add whatever you need... private $no_link_strings=array();
This list is used to search for links. Update it as needed.
No word strings
//a secondary spam words filter private $no_word_strings=array();
Words on this list are not allowed, an error is shown. We recommend only obvious words taken from past spams. Less (20 or so) is more.
private $show_trigger_word=true;
If true the user is shown the word that caused the error. Recommended because the filter is not smart and will match inside another word (think scum).
Whitelist URLs
//URLS to ALLOW always. In addition to own-site urls private $whitelist_urls=array();
Links at the same site as the forum are automatically allowed. Additional URLs to ignore can be added here.
$whitelist_urls=array('http://code.google.com', 'http://sourceforge.net',);
This allows links to Google Code and Sourceforge, common sites we use in this forum.
Unicode range filter
private $unicode_filter='/([\x{0400}-\x{04FF}])+/u';
Some spam is in a language not used on the site. This is a regular expression that searches for 'unicode' character ranges, in this case Cyrillic. If the post contains these characters an error is shown.
- Leave blank to disable this feature
- Unicode block reference
Example ranges
- 0400-04FF - Cyrillic (Russian)
- 0600-06FF - Arabic
- 3100-312F - Mandarin Bopomofo block
$unicode_filter='/([\x{0400}-\x{04FF}]|[\x{0600}-\x{06FF}]|[\x{3100}-\x{312F}])+/u';
Example of chaining together multiple filter blocks.
Unicode percent filter
private $minimum_nonunicode_text=0.95;
There is a secondary unicode filter that simply tests for all unicode character. This is probably best for English and similar language sites only. If your language uses unicode consider blocking unused ranges with the filter above.
The value represents the percent of post that must be non-unicode. For example 0.95 means that 95% of the post must be non-unicode or an error is displayed.
- Leave blank to disable this feature
- Based on this
Help URL
private $help_url = 'http://da....';
A user might want to know why and how long their account is restricted. It's a good idea to write a forum post explaining the limits. If a URL is entered on this line it will be displayed with 'click for help' in the error message.
- Leave blank to disable this feature
- Our example help post, feel free to use it
Load from database
private $load_values_from_db=false; //enable to use with DB, or use the values below
The settings can cpme from the database values for the disable links plugin, or frpm the settings in the class. Set to true to use database settings.
Minimum account age
private $minimum_days=1;
Minimum number of days the account must be registered to post links.
- Enter 0 to disable age check
- This AND the minimum number of posts must be met
Minimum account posts
private $minimum_posts=1; //minimum posts for member to post links
Minimum number of posts a user must make before they can use links.
- Enter 0 to disable check
- This AND the minimum account age must be met
Sleeper agent check
private $sleeper_check=true;
User with 0 posts after minimum account age will be prohibited from posting, signatures, and profiles.
What is this about? We ban links based on age and posts, but imagine a spammer script that:
- Registers a bunch of accounts on one day
- Comes back after the minimum account age
- Posts the minimum account posts until the filter is off
- Then blasts the forum full of spam
All that could happen before you have a chance to see and delete the initial posts.
The sleeper agent check stops this. If the new user has not posted an initial post within the minimum account age windows, then posting, signatures, and profiles are disabled and an error is displayed.
Most of our members join and post right away, so this is transparent to almost everyone.
Zombie registration purge
We get 100s of script registrations every day. Captchas, challenge questions, and other stuff only stops the stupidest scripts, and annoys users. We forbid them from posting links, but that doesn't stop them from registering.
The zombie registration purge deletes all users with 0 posts who are older than the minimum account age.
Weaknesses
There is still a corner case that should be dealt with:
- The spammer has one 'innocent' post before the account is deleted, right at the end of the minimum account age before a moderator notices
- The spammer posts until minimum posts are reached
- After the restrictions are lifted the spam flows
A future option is to made the minimum account age only apply after the first post is made. That gives admins a better chance to notice the suspicious 'innocent' posts and delete the users before the links start flowing. It will require a database query though, so it has not yet been implemented.
Install
SQL
Run this SQL from phpmyadmin, etc:
INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_after_num_posts', '0', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_after_num_days', '0', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_allow_always', '', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_link_strings', 'http://,www.,.com,.net,.org,.uk,.ly,.me,.ru,.biz,.info,dot com,dot net,dot org,dotcom,dotnet,dotorg,_com,_org,_co_uk,_ru,dot ru', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_word_strings', 'home mortage,forex,free games', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_unicode_filter', '', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_nonunicode_percent', '', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_help_url', '', '0'); INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_disable_sleepers', '1', '0');
posting.php
Find in /posting.php:
if ($submit || $preview || $refresh) { $post_data['topic_cur_post_id'] = request_var('topic_cur_post_id', 0); $post_data['post_subject'] = utf8_normalize_nfc(request_var('subject', '', true)); $message_parser->message = utf8_normalize_nfc(request_var('message', '', true)); {
Add under:
//---------------------------------------------------------------// //check for URLs and bad words in new user post require($phpbb_root_path . 'includes/functions_link_filter.' . $phpEx); $f= new link_filter; //new link filter if($f->link_filter_test_post($message_parser->message, $post_data['post_subject']))//run the check { $error = array_merge($error, $f->error);//add the errors to the error array, if any } unset($f); //---------------------------------------------------------------//
ucp_profile.php
Find In /includes/ucp/ucp_profile.php:
case 'profile_info':
Add under:
//---------------------------------------------------------------// //display warning that new users can't have a profile require($phpbb_root_path . 'includes/functions_link_filter.' . $phpEx); $f= new link_filter; //new link filter if($f->link_filter_test_profile())//run the check { $error = array_merge($error, $f->error);//add the errors to the error array, if any } //---------------------------------------------------------------//
Find:
if (!check_form_key('ucp_profile_info')) { $error[] = 'FORM_INVALID'; }
Add under:
//-------------------------------------------------------// // don;t allow new users to update profile if(sizeof($f->error))//run the check { $error = array_merge($error, $f->error);//add the errors to the error array, if any } //-------------------------------------------------------//
Find
$cp->generate_profile_fields('profile', $user->get_iso_lang_id());
Add under
//-------------------------------------------------------// //unset f clear down here incase we want to do something more interesting with it later unset($f); //-------------------------------------------------------//
Find
if (!check_form_key('ucp_sig')) { $error[] = 'FORM_INVALID'; }
Add under
//---------------------------------------------------------------// //check for URLs and bad words in new user signiture require($phpbb_root_path . 'includes/functions_link_filter.' . $phpEx); $f= new link_filter; //new link filter if($f->link_filter_test_signiture($signature))//run the check { $error = array_merge($error, $f->error);//add the errors to the error array, if any } unset($f); //---------------------------------------------------------------//
ucp_pm_compose.php
Find in /includes/ucp/ucp_pm_compose.php:
if ($submit || $preview || $refresh) { if (($submit || $preview) && !check_form_key('ucp_pm_compose')) { $error[] = $user->lang['FORM_INVALID']; } $subject = utf8_normalize_nfc(request_var('subject', '', true)); $message_parser->message = utf8_normalize_nfc(request_var('message', '', true));
Add under
//---------------------------------------------------------------// //check for URLs and bad words in new user post require($phpbb_root_path . 'includes/functions_link_filter.' . $phpEx); $f= new link_filter; //new link filter if($f->link_filter_test_post($message_parser->message, $subject))//run the check { $error = array_merge($error, $f->error);//add the errors to the error array, if any } unset($f); //---------------------------------------------------------------//
board.php
Find in /language/en/acp/board.php:
'SMILIES_PER_PAGE' => 'Smilies per page', 'TOPICS_PER_PAGE' => 'Topics per page',
Add after:
//-------------------------------------------------------- 'LINKS_FILTER_OPTIONS' => 'Disable links', 'LINKS_AFTER_NUM_POSTS' => 'Min post count before posting external links', 'LINKS_AFTER_NUM_POSTS_EXPLAIN' => 'Users will need this number of posts before they are able to use the [URL] BBCode tag or post external page urls and domain references. Applies to posts, signatures, PMs, and profiles.', 'LINKS_AFTER_NUM_DAYS' => 'Min days from registration before posting external links', 'LINKS_AFTER_NUM_DAYS_EXPLAIN' => 'Users will need to have registered this many days before they are able to use the [URL] BBCode tag or post external page urls and domain references. Applies to posts, signatures, PMs, and profiles.', 'LINKS_LINK_STRINGS' => 'Link search strings', 'LINKS_LINK_STRINGS_EXPLAIN' => 'Anything in this line is considered a link and cannot be posted by a new user.', 'LINKS_WORD_STRINGS' =>'Bad word search strings', 'LINKS_WORD_STRINGS_EXPLAIN' =>'Anything in this line cannot be used in a new user post.', 'LINKS_ALLOW_ALWAYS' =>'Link whitelist', 'LINKS_ALLOW_ALWAYS_EXPLAIN' =>'In addition to links at the same site as the forum, new users can also posts links to these sites.', 'LINKS_UNICODE_FILTER' =>'Unicode REGEX filter', 'LINKS_UNICODE_FILTER_EXPLAIN' =>'A REGEX filter that detects unicode characters not normally used in the language of the forum. Leave blank to disable.', 'LINKS_NONUNICODE_PERCENT' =>'Non-unicode threshhold', 'LINKS_NONUNICODE_PERCENT_EXPLAIN' =>'What percent of a post must be non-unicode characters. Enter as 0.95 for 95%, leave blank to disable.', 'LINKS_HELP_URL' =>'Filter help link', 'LINKS_HELP_URL_EXPLAIN' =>'A link to a post that explains the filter criteria. Leave blank to disable.', 'LINKS_DISABLE_SLEEPERS' =>'Disable sleeper agents', 'LINKS_DISABLE_SLEEPERS_EXPLAIN'=>'Disable posting, signiture, and profiles for new users with 0 posts after minimum days (above). Prevents old accounts from coming back as spammers.', //-----------------------------------------------------------
acp_board.php
Find in /includes/acp/acp_board.php:
'max_post_img_width' => array('lang' => 'MAX_POST_IMG_WIDTH', 'validate' => 'int:0', 'type' => 'text:5:4', 'explain' => true, 'append' => ' ' . $user->lang['PIXEL']), 'max_post_img_height' => array('lang' => 'MAX_POST_IMG_HEIGHT', 'validate' => 'int:0', 'type' => 'text:5:4', 'explain' => true, 'append' => ' ' . $user->lang['PIXEL']),
Add after:
//-------------------------------------------------------------// 'legend3' => 'LINKS_FILTER_OPTIONS', 'links_after_num_posts' => array('lang' => 'LINKS_AFTER_NUM_POSTS', 'validate' => 'int:0', 'type' => 'text:4:4', 'explain' => true), 'links_after_num_days' => array('lang' => 'LINKS_AFTER_NUM_DAYS', 'validate' => 'int:0', 'type' => 'text:4:4', 'explain' => true), 'links_link_strings' => array('lang' => 'LINKS_LINK_STRINGS','validate' => 'string', 'type' => 'text:40:255', 'explain' => true), 'links_word_strings' => array('lang' => 'LINKS_WORD_STRINGS','validate' => 'string', 'type' => 'text:40:255', 'explain' => true), 'links_allow_always' => array('lang' => 'LINKS_ALLOW_ALWAYS','validate' => 'string', 'type' => 'text:40:255', 'explain' => true), 'links_unicode_filter' => array('lang' => 'LINKS_UNICODE_FILTER','validate' => 'string', 'type' => 'text:40:255', 'explain' => true), 'links_nonunicode_percent' => array('lang' => 'LINKS_NONUNICODE_PERCENT','validate' => 'string', 'type' => 'text:40:255', 'explain' => true), 'links_help_url' => array('lang' => 'LINKS_HELP_URL','validate' => 'string', 'type' => 'text:40:255', 'explain' => true), 'links_disable_sleepers' => array('lang' => 'LINKS_DISABLE_SLEEPERS', 'validate' => 'bool', 'type' => 'radio:yes_no', 'explain' => true), //-------------------------------------------------------------//
function_link_filter.php
Put the class in includes/function_link_filter.php.
- [Latest version in SVN]
To do
- Put in SVN
- Sliding window based on first post time