Json API query always fails with unable to parse: (null)

General Discussions - English
1

Apologies if this is a newbie issue, but following the documentation (Manticore Search Manual: Connecting to the server > HTTP), I cannot get any of the JSON api to work and it always returns the error: {“error”:“Error parsing json query: unable to parse: (null)”}.

Version: manticore-2.7.0-180611-3d60c8e-release-stemmer-rhel7-bin

My RT index definition:
index testrt
{
type = rt
rt_mem_limit = 128M
path = /var/lib/manticore/testrt
rt_field = title
rt_field = content
rt_attr_uint = gid
}

I have “listen = localhost:8080:http” in my searchd setting and it works fine against the /search API, but if I invoke the following:

curl -X POST ‘http://localhost:8080/json/search’ -d ‘{“index”:“testrt”,“query”:{“match”:{“title”:“text”}}}’

I get: {“error”:“Error parsing json query: unable to parse: (null)”}

Any tips?

2

could you provide reproducible example? Whole your config along with query attached.
After I replaced " at your query and issued

curl -X POST \
  http://127.0.0.1:8380/json/search \
  -d '{"index":"testrt","query":{"match":{"title":"text"}}}'

I got correct reply

{
    "took": 0,
    "timed_out": false,
    "hits": {
        "total": 0,
        "hits": []
    }
}
3

I appreciate the help! Here is my /etc/sphinx/sphinx.conf:

#
# Minimal Sphinx configuration sample (clean, simple, functional)
#

#source src1
#{
#	type			= mysql
#
#	sql_host		= localhost
#	sql_user		= test
#	sql_pass		=
#	sql_db			= test
#	sql_port		= 3306	# optional, default is 3306
#
#	sql_query		= \
#		SELECT id, group_id, UNIX_TIMESTAMP(date_added) AS date_added, title, content \
#		FROM documents
#
#	sql_attr_uint		= group_id
#	sql_attr_timestamp	= date_added
#}


#index test1
#{
#	source			= src1
#	path			= /var/lib/manticore/test1
#}


index testrt
{
	type			= rt
	rt_mem_limit		= 128M

	path			= /var/lib/manticore/testrt

	rt_field		= title
	rt_field		= content
	rt_attr_uint		= gid
}


indexer
{
	mem_limit		= 128M
}


searchd
{
	listen			= 9312
	listen			= 9306:mysql41
	listen			= localhost:8080:http
	log			= /var/log/manticore/searchd.log
	query_log		= /var/log/manticore/query.log
	read_timeout		= 5
	max_children		= 30
	pid_file		= /var/run/manticore/searchd.pid
	seamless_rotate		= 1
	preopen_indexes		= 1
	unlink_old		= 1
#	workers			= threads # for RT to work
	workers			= thread_pool
	binlog_path		= /var/lib/manticore/
}

Here is are some failed API calls I tried using curl - they all fail with the same error (shown below)

curl -X POST 'http://localhost:8080/json/search' -d '{ "index" : "testrt" , "query" : { "match" : { "title" : "text" } } }'
# with spaces removed
curl -X POST 'http://localhost:8080/json/search' -d '{"index":"testrt","query":{"match":{"title":"text"}}}'

The error is:

{"error":"Error parsing json query: unable to parse: (null)"}

EDIT: it appears that the HTTP json end point is dropping/ignoring the post’ed body. For example, if I post garbage data:

curl -X POST 'http://localhost:8080/json/search' -d BLAH

I get the same error: unable to parse: (null).

4

To follow up on this - when I run ‘searchd’ through the gdb debugger, I notice something strange with the constructor logic for the HttpHandler_JsonSearch_c class:

class HttpHandler_JsonSearch_c : public HttpSearchHandler_c
{
public:	
	HttpHandler_JsonSearch_c ( const CSphString & sQuery, const OptionsHash_t & tOptions, int iCID, bool bNeedHttpResponse )
		: HttpSearchHandler_c ( sQuery, tOptions, iCID, bNeedHttpResponse )
	{}

When I place a breakpoint at the HttpSearchHandler_c() super constructor line, the member variable m_sQuery has the expected value: I can see my JSON query payload that was in the POST operation. However, once we get to the {} line, the ctor code for the HttpHandler_JsonSearch_c class itself, the contents of the m_sQuery string has been cleared to an empty string.

Subsequently, when the following code is executed with m_sQuery now as an empty string:

		if ( !sphParseJsonQuery ( m_sQuery.cstr(), m_tQuery, m_bProfile, m_bAttrHighlight, sError, m_sWarning ) )
		{
			sError.SetSprintf( "Error parsing json query: %s", sError.cstr() );
			ReportError ( sError.cstr(), SPH_HTTP_STATUS_400 );
			return NULL;
		}

Because m_sQuery is an empty string at this stage, the error is returned an no processing takes place?

Of course, I’m not completely familiar with this code, but can someone comment on what could explain this? Could this be a C++ issue with the distro I’m using (Centos 7)?

5

I believe I found the issue and a possible fix. I believe the root cause is the declaration of the m_sQuery member variable as a pointer rather than a storage variable:

File: searchdhttp.cpp

class HttpHandler_c
{
[..]
protected:
	const CSphString &	m_sQuery;

Instead, if I change this line to:

class HttpHandler_c
{
[..]
protected:
	CSphString	m_sQuery;

The JSON functions all start to work as expected.

6

I’m unable to reproduce case you describing. I always got correct reply with config you provided.
For code it correct and hold reference to data up at stack. It works fine.

Could you provide docker container or image that reproduces issue locally here?

7

@tomat: I’ll try to arrange a docker image. I can’t share this exact CentOS VM image I am working on since it has proprietary licensed software on it, unfortunately.

While I’m working on that, just FYI: when debugging, I definitely see a call to the destructor of CSphString during the constructor invocation of HttpHandler_JsonSearch_c, so by the time the HttpHandler_JsonSearch_c constructor has finished, the contents of the m_sQuery member variable has been cleared.

8

@tomat, I’ve pushed a Docker image on DockHub:

docker run -ti -p 8080:8080 -p 9306:9306 twelveeighty/centos7-manticore:v1

To test, run:

curl -X POST 'http://0.0.0.0:8080/json/search' --data-binary '{"index":"mxrt","query":{"match":{"content":"test"}}}'

Dockerfile:

FROM centos:7
ENV container docker
RUN yum -y install git cmake make gcc-c++ bison flex; yum clean all
RUN mkdir /build && cd /build \
&& git clone https://github.com/manticoresoftware/manticore.git --single-branch \
&& cd manticore && git checkout tags/2.7.0 \
&& mkdir -p build && cd build \
&& cmake \
    -D CMAKE_BUILD_TYPE=Release \
    -D DISTR_BUILD=rhel7 \
    -D WITH_MYSQL=0 \
    -D WITH_PGSQL=0 \
    -D WITH_RE2=1 \
    -D CMAKE_INSTALL_PREFIX=/usr \
    -D SPHINX_TAG=release .. \
&& make -j4 searchd indexer indextool
COPY sphinx.conf /etc/sphinx/sphinx.conf
RUN mkdir -p /var/lib/manticore/log \
&& mkdir -p /var/lib/manticore/data \
&& mkdir -p /var/run/manticore \
&& mkdir -p /var/log/manticore \
&& cp /build/manticore/build/src/indexer /usr/bin/ \
&& cp /build/manticore/build/src/indextool /usr/bin/ \
&& cp /build/manticore/build/src/searchd /usr/bin/
VOLUME /var/lib/manticore /etc/sphinx
EXPOSE 9306
EXPOSE 9312
EXPOSE 8080
CMD [ "/usr/bin/searchd", "--nodetach" ]

sphinx.conf:

index mxrt
{
	type			= rt
	rt_mem_limit		= 128M

	path			= /var/lib/manticore/mxrt

	rt_field		= content
	rt_field		= longcontent
	rt_attr_string		= object
	rt_attr_uint		= gid
}

indexer
{
	mem_limit		= 128M
}


searchd
{
	listen			= 9312
	listen			= 9306:mysql41
#   listen			= 127.0.0.1:8080:http
	listen			= 8080:http
	log			= /var/log/manticore/searchd.log
	query_log		= /var/log/manticore/query.log
	read_timeout		= 5
	max_children		= 30
	pid_file		= /var/run/manticore/searchd.pid
	seamless_rotate		= 1
	preopen_indexes		= 1
	unlink_old		= 1
#	workers			= threads # for RT to work
	workers			= thread_pool
	binlog_path		= /var/lib/manticore/
}
9

With docker image provided I reproduces issue. I’ve created ticket at github json/search call failed · Issue #88 · manticoresoftware/manticoresearch · GitHub
there you can track progress