I will post this topic with also the solution just in case someone hits the same problem.
When you run indexer with the --rotate option the rotation of the indices will succeed only if the configuration file has not changed (the one used when searchd was started must be exactly the same as when rotation occurs). If the conf file is different then the new files will be left in place and rotation will fail (this may cause a big data disk usage like in my case).
The solution is to restart searchd when you make a change to the configuration file as when the SIGHUP is sent by indexer to searchd, searchd will check if the current conf file and the one it was started with are the same and if they are not it will not rotate the index files.
I hope it is clear and it helps.
Roberto