Quantcast

keyword search behavior after VuFind 3 update

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

keyword search behavior after VuFind 3 update

John Jung
Hello everyone,

We are in the process of upgrading to  VuFind 3.1.1. As we're testing our system, we see that keyword searches for multiple words behave differently than they used to. Instead of an implicit AND between each word, now it seems like there might be an implicit OR. 

For an example, searching "for whom the bell tolls" (without quotes) returns 128 results on our production system. (https://catalog.lib.uchicago.edu/vufind/Search/Results?lookfor=for+whom+the+bell+tolls&type=AllFields). On our test system it returns several million results.

When we turn debug mode on we see that VuFind is sending the same query string back to solr as it was before, so we think this issue might involve solr configuration in some way. Has anyone else experienced this while updating? Has anything changed in the solr configuration lately that might cause something like this? 

Thanks so much for any help you can provide. 

John




------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: keyword search behavior after VuFind 3 update

Demian Katz

This is a complicated area. Solr’s handling of default operators has changed over the years, and getting the desired behavior can require different q.op and mm settings under different circumstances. It is particularly confusing because there are significant differences between regular and extended Dismax.

 

In theory, VuFind should compensate for all of this – see this code for the most important bit of logic (and some relevant comments):

 

https://github.com/vufind-org/vufind/blob/a07b0bee7e8d9d28b5bbf904631047237518e59a/module/VuFindSearch/src/VuFindSearch/Backend/Solr/SearchHandler.php#L253

As a first step for troubleshooting, I’d recommend checking a few things:

 

1.)    Have you customized your Solr config at all – particularly, have you added explicit mm or q.op settings anywhere?

2.)    Have you customized your searchspecs.yaml at all? The switch to edismax in release 3.1.1 may be significant. Also, be sure you’ve cleared out your local/cache directory just to be sure everything is up to date.

3.)    Have you changed the default_dismax_handler in config.ini? (This is unlikely to be significant, but it is another potential factor).

 

I hope this helps explain what is going on and points you in the right direction, but if you still find yourself stuck, just let me know!

 

thanks,

Demian

 

From: John Jung [mailto:[hidden email]]
Sent: Thursday, January 12, 2017 11:33 AM
To: [hidden email]
Subject: [VuFind-Tech] keyword search behavior after VuFind 3 update

 

Hello everyone,

 

We are in the process of upgrading to  VuFind 3.1.1. As we're testing our system, we see that keyword searches for multiple words behave differently than they used to. Instead of an implicit AND between each word, now it seems like there might be an implicit OR. 

 

For an example, searching "for whom the bell tolls" (without quotes) returns 128 results on our production system. (<a href="https://catalog.lib.uchicago.edu/vufind/Search/Results?lookfor=for&#43;whom&#43;the&#43;bell&#43;tolls&amp;type=AllFields">https://catalog.lib.uchicago.edu/vufind/Search/Results?lookfor=for+whom+the+bell+tolls&type=AllFields). On our test system it returns several million results.

 

When we turn debug mode on we see that VuFind is sending the same query string back to solr as it was before, so we think this issue might involve solr configuration in some way. Has anyone else experienced this while updating? Has anything changed in the solr configuration lately that might cause something like this? 

 

Thanks so much for any help you can provide. 

 

John

 

 

 


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: keyword search behavior after VuFind 3 update

John Jung
This is very helpful, Demian- thank you. 

From: Demian Katz <[hidden email]>
Date: Thursday, January 12, 2017 at 11:13 AM
To: John Jung <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: RE: keyword search behavior after VuFind 3 update

This is a complicated area. Solr’s handling of default operators has changed over the years, and getting the desired behavior can require different q.op and mm settings under different circumstances. It is particularly confusing because there are significant differences between regular and extended Dismax.

 

In theory, VuFind should compensate for all of this – see this code for the most important bit of logic (and some relevant comments):

 

https://github.com/vufind-org/vufind/blob/a07b0bee7e8d9d28b5bbf904631047237518e59a/module/VuFindSearch/src/VuFindSearch/Backend/Solr/SearchHandler.php#L253

As a first step for troubleshooting, I’d recommend checking a few things:

 

1.)    Have you customized your Solr config at all – particularly, have you added explicit mm or q.op settings anywhere?

2.)    Have you customized your searchspecs.yaml at all? The switch to edismax in release 3.1.1 may be significant. Also, be sure you’ve cleared out your local/cache directory just to be sure everything is up to date.

3.)    Have you changed the default_dismax_handler in config.ini? (This is unlikely to be significant, but it is another potential factor).

 

I hope this helps explain what is going on and points you in the right direction, but if you still find yourself stuck, just let me know!

 

thanks,

Demian

 

From: John Jung [[hidden email]]
Sent: Thursday, January 12, 2017 11:33 AM
To: [hidden email]
Subject: [VuFind-Tech] keyword search behavior after VuFind 3 update

 

Hello everyone,

 

We are in the process of upgrading to  VuFind 3.1.1. As we're testing our system, we see that keyword searches for multiple words behave differently than they used to. Instead of an implicit AND between each word, now it seems like there might be an implicit OR. 

 

For an example, searching "for whom the bell tolls" (without quotes) returns 128 results on our production system. (<a href="https://catalog.lib.uchicago.edu/vufind/Search/Results?lookfor=for&#43;whom&#43;the&#43;bell&#43;tolls&amp;type=AllFields">https://catalog.lib.uchicago.edu/vufind/Search/Results?lookfor=for+whom+the+bell+tolls&type=AllFields). On our test system it returns several million results.

 

When we turn debug mode on we see that VuFind is sending the same query string back to solr as it was before, so we think this issue might involve solr configuration in some way. Has anyone else experienced this while updating? Has anything changed in the solr configuration lately that might cause something like this? 

 

Thanks so much for any help you can provide. 

 

John

 

 

 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Loading...