Phpbb support (config) for manticore


#1

I’ve just migrated my search server from (quite buggy old version of) sphinx to manticore and it works great. But I wonder if it would be possible to add a native support for manticore in another very popular opensource project - phpbb. As you can see


it has a native support for sphinx, and I guess it shouldn’t be difficult to add similar config for manticore. I think this could help to advertise the manticore to php community.

By the way, the default sphinx config for phpbb is quite old, and it may not be optimal.


#2

could you provide your configs for Sphinx and for Manticore to check that is difference there. As it not quite clear what should have changed.

I not on pair to phpbb do you want phpbb to generate Manticore compatible config or Manticore to load config generated by phpbb?
Do phpbb have option to switch config generation? or pull-request to phpbb should change phpbb/search/sphinx/config.php to generate only Manticore config?


#3

There is a very nice phpbb demo site:


choose Admin Page and login with given credentials (link at the bottom of the page). Then the last position in the menu is ‘Search settings’. If you try to do this, you will see what I mean by ‘add phpbb support for manticore’ - I guess it should be possible to make a pull request for a new search option.

I am not phpbb developer, but I think, that sphinx config (which you can generate on the above admin page) was written in 2012 for Sphinx 2.0.1.
https://wiki.phpbb.com/Sphinx_Fulltext_Search
For this reason I suppose that this config may be outdated. For example, specifying custom location of the pid file seems to be not compatible with systemctl start/stop (at least this is the case in my installation - executing searchd directly still works, but it leads to other permissions problems). My idea was, that it would be very nice to have a modern config for sphinx/manticore, and the addition of the support for manticore can be a good opportunity for this.


#4

could you create ticket at our Github bugtracker that phpBB should generate Manticore compatible configs


#5

I also tried

choose Admin Page and login with given credentials (link at the bottom of the page). Then the last position in the menu is ‘Search settings’

however I see only 5 option and can not get searchd config that phpbb generates.

To create pull request to phpbb we need someone who will be able to install phpbb locally or via docker container and check that php code after pull requests works as indented


#6

As for admin panel of phpbb see the screenshot below

As for installing and testing phpbb with manticore config - I can do this.

As for the ticket on Github, before I do this, I will try to contact phpbb devs and ask them for their opinion.

Finally, I paste below the default searchd config generated by phpbb - it does work ok, provided you manage to start searchd and bypass pid/permissions problems, which is not completely straightforward.

source source_phpbb_0e2abc9ada1d1495_main 
{
	type = mysql # mysql or pgsql 
	sql_host = localhost # SQL server host sphinx connects to 
	sql_user = [dbuser] 
	sql_pass = [dbpassword] 
	sql_db = cms_phpbb 
	sql_port = 3306 # optional, default is 3306 for mysql and 5432 for pgsql 
	sql_query_pre = SET NAMES 'utf8' 
	sql_query_pre = UPDATE phpbb_sphinx SET max_doc_id = (SELECT MAX(post_id) FROM phpbb_posts) WHERE counter_id = 1 
	sql_query_range = SELECT MIN(post_id), MAX(post_id) FROM phpbb_posts 
	sql_range_step = 5000 
	sql_query = SELECT \
						p.post_id AS id, \
						p.forum_id, \
						p.topic_id, \
						p.poster_id, \
						p.post_visibility, \
						CASE WHEN p.post_id = t.topic_first_post_id THEN 1 ELSE 0 END as topic_first_post, \
						p.post_time, \
						p.post_subject, \
						p.post_subject as title, \
						p.post_text as data, \
						t.topic_last_post_time, \
						0 as deleted \
					FROM phpbb_posts p, phpbb_topics t \
					WHERE \
						p.topic_id = t.topic_id \
						AND p.post_id >= $start AND p.post_id <= $end 
	sql_query_post =  
	sql_query_post_index = UPDATE phpbb_sphinx SET max_doc_id = $maxid WHERE counter_id = 1 
	sql_query_info = SELECT * FROM phpbb_posts WHERE post_id = $id 
	sql_attr_uint = forum_id 
	sql_attr_uint = topic_id 
	sql_attr_uint = poster_id 
	sql_attr_uint = post_visibility 
	sql_attr_bool = topic_first_post 
	sql_attr_bool = deleted 
	sql_attr_timestamp = post_time 
	sql_attr_timestamp = topic_last_post_time 
	sql_attr_string = post_subject 
}
source source_phpbb_0e2abc9ada1d1495_delta : source_phpbb_0e2abc9ada1d1495_main 
{
	sql_query_pre =  
	sql_query_range =  
	sql_range_step =  
	sql_query = SELECT \
						p.post_id AS id, \
						p.forum_id, \
						p.topic_id, \
						p.poster_id, \
						p.post_visibility, \
						CASE WHEN p.post_id = t.topic_first_post_id THEN 1 ELSE 0 END as topic_first_post, \
						p.post_time, \
						p.post_subject, \
						p.post_subject as title, \
						p.post_text as data, \
						t.topic_last_post_time, \
						0 as deleted \
					FROM phpbb_posts p, phpbb_topics t \
					WHERE \
						p.topic_id = t.topic_id \
						AND p.post_id >=  ( SELECT max_doc_id FROM phpbb_sphinx WHERE counter_id=1 ) 
}
index index_phpbb_0e2abc9ada1d1495_main 
{
	path = data_pathindex_phpbb_0e2abc9ada1d1495_main 
	source = source_phpbb_0e2abc9ada1d1495_main 
	docinfo = extern 
	morphology = none 
	stopwords =  
	min_word_len = 2 
	charset_type = utf-8 
	charset_table = U+FF10..U+FF19->0..9, 0..9, U+FF41..U+FF5A->a..z, U+FF21..U+FF3A->a..z, A..Z->a..z, a..z, U+0149, U+017F, U+0138, U+00DF, U+00FF, U+00C0..U+00D6->U+00E0..U+00F6, U+00E0..U+00F6, U+00D8..U+00DE->U+00F8..U+00FE, U+00F8..U+00FE, U+0100->U+0101, U+0101, U+0102->U+0103, U+0103, U+0104->U+0105, U+0105, U+0106->U+0107, U+0107, U+0108->U+0109, U+0109, U+010A->U+010B, U+010B, U+010C->U+010D, U+010D, U+010E->U+010F, U+010F, U+0110->U+0111, U+0111, U+0112->U+0113, U+0113, U+0114->U+0115, U+0115, U+0116->U+0117, U+0117, U+0118->U+0119, U+0119, U+011A->U+011B, U+011B, U+011C->U+011D, U+011D, U+011E->U+011F, U+011F, U+0130->U+0131, U+0131, U+0132->U+0133, U+0133, U+0134->U+0135, U+0135, U+0136->U+0137, U+0137, U+0139->U+013A, U+013A, U+013B->U+013C, U+013C, U+013D->U+013E, U+013E, U+013F->U+0140, U+0140, U+0141->U+0142, U+0142, U+0143->U+0144, U+0144, U+0145->U+0146, U+0146, U+0147->U+0148, U+0148, U+014A->U+014B, U+014B, U+014C->U+014D, U+014D, U+014E->U+014F, U+014F, U+0150->U+0151, U+0151, U+0152->U+0153, U+0153, U+0154->U+0155, U+0155, U+0156->U+0157, U+0157, U+0158->U+0159, U+0159, U+015A->U+015B, U+015B, U+015C->U+015D, U+015D, U+015E->U+015F, U+015F, U+0160->U+0161, U+0161, U+0162->U+0163, U+0163, U+0164->U+0165, U+0165, U+0166->U+0167, U+0167, U+0168->U+0169, U+0169, U+016A->U+016B, U+016B, U+016C->U+016D, U+016D, U+016E->U+016F, U+016F, U+0170->U+0171, U+0171, U+0172->U+0173, U+0173, U+0174->U+0175, U+0175, U+0176->U+0177, U+0177, U+0178->U+00FF, U+00FF, U+0179->U+017A, U+017A, U+017B->U+017C, U+017C, U+017D->U+017E, U+017E, U+0410..U+042F->U+0430..U+044F, U+0430..U+044F, U+4E00..U+9FFF 
	min_prefix_len = 0 
	min_infix_len = 0 
}
index index_phpbb_0e2abc9ada1d1495_delta : index_phpbb_0e2abc9ada1d1495_main 
{
	path = data_pathindex_phpbb_0e2abc9ada1d1495_delta 
	source = source_phpbb_0e2abc9ada1d1495_delta 
}
indexer 
{
	mem_limit = 512M 
}
searchd 
{
	compat_sphinxql_magics = 0 
	listen = localhost:9312 
	log = data_pathlog/searchd.log 
	query_log = data_pathlog/sphinx-query.log 
	read_timeout = 5 
	max_children = 30 
	pid_file = data_pathsearchd.pid 
	max_matches = 20000 
	binlog_path = data_path 
}