fogbound.net




Tue, 12 Jun 2012

Spelling Suggestions from Solr using Yii-solr

— SjG @ 7:48 pm

So a Yii app wants to query Solr using the Yii-solr extension, and wants to provide spelling suggestions for the provided search term. We need the suggestions to be unique, i.e., no repeated suggesttions, and we don’t want negative suggestions, i.e., if the user’s search wanted to exclude “foo”, we don’t want to suggest “foo” as a possible correction. There’s probably a better way to do all this, but I couldn’t figure it out. Here’s what I did instead (as always, forgive the code style/formatting):

I created a subclass of ASolrConnection that switches to using the “spell” Solr servlet:


class SolrSpellingConnection extends ASolrConnection {
public $servlet_type;
public $servlet_path;
public function resetClient()
{
parent::resetClient();
if (isset($this->servlet_type) && isset($this->servlet_path))
{
$this->_client->setServlet($this->servlet_type,$this->servlet_path);
}
}
}

This is instantiated in the app’s config file:

'solr' => array(
'class'=>'app.components.SolrSpellingConnection',
'clientOptions'=>array(
'hostname'=>'localhost',
'port'=>8983,
),
'servlet_type'=>SolrClient::SEARCH_SERVLET_TYPE,
'servlet_path'=>'spell'
),

Then, my Solr query data-provider looks something like:


$criteria = new ASolrCriteria;
$criteria->query = $queryterm;
$criteria->set('spellcheck','true');
$pages=array('pageSize'=>15);
$dataprovider = new ASolrDataProvider(ASolrDocument::model(),array('criteria'=>$criteria,'pagination'=>$pages));
return $dataprovider;

In my controller, I add some code to process the suggestions:

public function buildSuggestionList($dataprovider, $originalQueryString)
{
$suggestions = array();
$terms = array();
if ($dataprovider != null)
{
$facets = $dataprovider->getQueryFacets();
$resp = $dataprovider->getSolrQueryResponse()->getSolrObject();
if (isset($resp['spellcheck']))
{
if (isset($resp['spellcheck']['suggestions']))
{
foreach ($resp['spellcheck']['suggestions'] as $thisSuggest)
{
if (is_object($thisSuggest) && get_class($thisSuggest) == 'SolrObject')
{
if (isset($thisSuggest['suggestion']) && is_array($thisSuggest['suggestion']))
{
foreach($thisSuggest['suggestion'] as $thisTerm)
{
$terms[$thisTerm]=1;
}
}
}
}
}
}
}
foreach($terms as $termKey=>$val)
{
// Solr adds negated or added terms to suggestions if they don't match case
if (!preg_match('/\b'.$termKey.'\b/i',$originalQueryString))
{
$suggestions[] = CHtml::link($termKey,'/index.php/mysearchcontroller?q='.urlencode($termKey));
}
}
return $suggestions;
}

I can then display the main collection of results using a CWidget. For displaying suggestions, however, a bit of specific code is required:


$suggestions = $this->buildSuggestionList($dataprovider, $originalQueryString);
if (count($suggestions) > 0)
{
echo '<p>Did you mean '.implode(' or ',$suggestions).'?</p>';
}
?>

When this is all in place, you should get decent suggestions from Solr. Of course, you will need to have Solr build a spelling index! The easiest way to do that is simply connect to Solr’s web interface and tell it to build the index:

http://localhost:8983/solr/spell?spellcheck.build=true


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.