Preserve old config files upon update/installation

Feature Requests
1

It would be best to have /etc/manticoresearch/manticore.conf not overwritten by new installation /update if such file exists already.

Upon installation if installer sees /etc/manticoresearch/manticore.conf file - it should create /etc/manticoresearch/manticore.conf.dist and not overwrite my old config file which makes it much more convenient. Otherwise I have to copy my config file somewhere else and copy it back after installation or face my config file lost for good.

It is just a few lines of code extra but saves a lot of headache.

2

If you have /etc/manticoresearch/manticore.conf overwritten upon installation/upgrade it’s a bug, but I can’t reproduce it in Ubuntu bionic +

apt-key adv --fetch-keys 'https://repo.manticoresearch.com/GPG-KEY-manticore'
wget https://repo.manticoresearch.com/manticore-repo.noarch.deb
dpkg -i manticore-repo.noarch.deb
apt update
apt install manticore

How do you reproduce it?

3

I have installed it on Centos 7 with such instructions (i had 3.53 in that location installed previously):

get from github in /opt folder

git clone --depth 1 GitHub - manticoresoftware/manticoresearch: Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon

cd /opt/manticoresearch

wget http://snowball.tartarus.org/dist/libstemmer_c.tgz
tar -xvf libstemmer_c.tgz

yum install libmariadb-dev mariadb-devel libmysqlclient-dev mysql-devel
yum install re2
yum install gcc gcc-c++ make expat-devel libicu-devel openssl-devel boost-devel rpm-build systemd-units
yum install cmake (if installs lower than 3.13 version use:

cd /opt
wget https://cmake.org/files/v3.17/cmake-3.17.3.tar.gz

tar -zxf cmake-3.17.3.tar.gz

cd /opt/cmake-3.17.3

(Note: the make install step is optional, cmake will run from the build directory just fine.)

./bootstrap
make
#Optional, skipped this step, cmake is built in bin folder and that is enough:
make install

The installation directory is usually left at its default, which is /usr/local and usually better not to install here. It is possible to specify a different installation directory by adding

-DCMAKE_INSTALL_PREFIX=/usr/local/manticore to the CMake command line.

As for precompiling the configuration, this was actually done in the /build directory in

cd /opt/manticoresearch

mkdir build && cd build

/opt/cmake-3.17.3/bin/cmake -S /opt/manticoresearch -DWITH_MYSQL=1 -DWITH_RE2=1 -DWITH_STEMMER=1 -DCMAKE_INSTALL_PREFIX=/usr/local/manticore

to set the source as was recommended in the documentation combined with the flags.

Create a folder /usr/local/manticore

mkdir manticore

After cmake completed the precompile, within the same build/ directory to actually install was done with:

make -j4

and

make -j4 install

again per the documentation.

This installs Manticore in /usr/local/manticore

Check the version:

cd /usr/local/manticore/usr/bin

./searchd -v
Manticore 3.5.3 0709fc5b@201119 dev
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2020, Manticore Software LTD (http://manticoresearch.com)

4

I see. It may be not possible to preserve the config if you install it with just make install, but since you’ve reached that far you can probably make one step farther - make package which should build you an RPM which you can install and that should preserve the config.

5

Instead on make -j4 install do

make -j4 package

?

6

I have seen this with CentOS 7 rpms as well when using yum repos. You install a new package and the old config is moved to manitcore.conf.orig and the existing config is replaced by the stock manticore.conf. You can easily copy it back and restart but it is a bit inconvenient.

7

Yes:

snikolaev@dev:~/manticore_gitlab/build$ cmake -S /opt/manticoresearch -DWITH_MYSQL=1 -DWITH_RE2=1 -DWITH_STEMMER=1 -DCMAKE_INSTALL_PREFIX=/usr/local/manticore -DDISTR=rhel7 ..
...
snikolaev@dev:~/manticore_gitlab/build$ make -j8
...
snikolaev@dev:~/manticore_gitlab/build$ make -j8 package
...
CPack: - package: /home/snikolaev/manticore_gitlab/build/manticore-3.5.5_201223.4d792b4-1.x86_64.rpm generated.
CPack: - package: /home/snikolaev/manticore_gitlab/build/manticore-converter-3.5.5_201223.4d792b4-1.x86_64.rpm generated.
CPack: - package: /home/snikolaev/manticore_gitlab/build/manticore-converter-debuginfo-3.5.5_201223.4d792b4-1.x86_64.rpm generated.
CPack: - package: /home/snikolaev/manticore_gitlab/build/manticore-debuginfo-3.5.5_201223.4d792b4-1.x86_64.rpm generated.
8

I have seen this with CentOS 7 rpms as well when using yum repos

I’ve reproduced the issue in Centos 7. Created an issue about it - config is not preserved on removing a package · Issue #471 · manticoresoftware/manticoresearch · GitHub

9

Thank you. What is the difference between j4 and j8 parameter?

And possibly all of the above could go into docs for other to read in future.

10

make(1) - Linux man page :

-j [jobs], --jobs[=jobs]
Specifies the number of jobs (commands) to run simultaneously. If there is more than one -j option, the last one is effective. If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously.

And possibly all of the above could go into docs for other to read in future.

Actually there’s a modern way of building that we use in our CI. It’s based on building dockers - manticoresearch/dist/build_dockers at master · manticoresoftware/manticoresearch · GitHub

There’s a doc for that, but we should review and integrate it into our main manual. However we see it’s very uncommon to build from source rather than just using a package, that’s why it’s not a priority. Hopefully one day someone will contribute into manticoresearch/Compiling_from_sources.md at master · manticoresoftware/manticoresearch · GitHub . It’s open source after all :slight_smile:

11

Yes, agree with you completely. And I do have notes for installing with docker but did not like it very much as I was getting lost where exactly am I - within docker or in a regular shell. But that is more a small flaw in docker and fussiness in me.

Either way - issue resolved, solution known, a fix is possible some time in future. Case closed.

12

The issue has been fixed and will be included in the next release. Thank you again for pointing that out.