Best way to switch between two Solr instances/cores?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Best way to switch between two Solr instances/cores?

Delis, Christopher
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://vufind.org/wiki/indexing:deduplication ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Demian Katz

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Delis, Christopher
Thanks for reminding me about sharding support; I forgot it existed. However, I don't think the will help us in our case. I'm guessing sharding would be another way to separate the two catalogs --local and universal-- during the Search stage. But I already have a solution for that part using multisite configurations (each catalog --local and universal-- has its own config.ini section where I can configure Solr to use different instances/cores). 

The problem I'm dealing with occurs *after* the initial search takes place. The search itself is fine as it searches the correct Solr instance. E.g., Let's assume a patron searches the universal catalog and then proceeds to the Detailed record page for record UC.123. Remember, the record UC.123 is a representation of a deduplicated set of source records (which reside in the local catalog). Let's also assume that UC.123 has two source records of A.111 and B.222 (for institutions A and B respectively, both of which again reside in the local catalog). In the Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings information for the two source records based on the different configuration settings for institutions A and B (because MultiBackend is specifically designed to allow each institution its own ILS configuration settings). However, when a patron decides to request one of these items, say A.111, VuFind will attempt to load this Solr record based on the Solr settings in config.ini for the universal catalog. But the source record A.111 exists in a different Solr instance (for the local catalog). This is not a problem if I store both universal records (of the type UC.123) and local records (of the type A.111) in the same Solr instance. But it becomes a big problem if I separate the local and universal catalog into two separate Solr instances (or cores).

Perhaps I'm still missing something and there's still a way to get this to work using sharding or something else. I just thought it might be worthwhile to explain my situation in a different way (above), in case it wasn't completely clear in my previous email. I have to admit; it can get confusing at times (for me, anyway). It's also possible that it makes sense to keep using a single Solr instance for both sets/types of records.

Chris

 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 12:56 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Demian Katz

I don't claim to be an expert on Solr sharding, but I still think it might be useful for your situation. Sharding can be used as a way of making multiple Solr indexes behave as though they are a single index, since it distributes a search across all of the shards. I believe VuFind even contains some smarts so that, if you disable one shard for searching purposes, it still turns on that shard for record retrieval purposes, to ensure that all record IDs can be retrieved. So I think you might be able to get away with creating "universal" and "local" shards, with "universal" on by default. That will cause users to search the UC records, but also offer the ability to retrieve the A and B records when necessary. I may be overly optimistic about this -- as I say, I haven't worked too deeply with this functionality -- but I have a feeling it might actually work as desired out of the box, and if not, you might be able to get there with a bit of custom listener work.


- Demian


From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 2:56 PM
To: Demian Katz; [hidden email]
Subject: RE: Best way to switch between two Solr instances/cores?
 
Thanks for reminding me about sharding support; I forgot it existed. However, I don't think the will help us in our case. I'm guessing sharding would be another way to separate the two catalogs --local and universal-- during the Search stage. But I already have a solution for that part using multisite configurations (each catalog --local and universal-- has its own config.ini section where I can configure Solr to use different instances/cores). 

The problem I'm dealing with occurs *after* the initial search takes place. The search itself is fine as it searches the correct Solr instance. E.g., Let's assume a patron searches the universal catalog and then proceeds to the Detailed record page for record UC.123. Remember, the record UC.123 is a representation of a deduplicated set of source records (which reside in the local catalog). Let's also assume that UC.123 has two source records of A.111 and B.222 (for institutions A and B respectively, both of which again reside in the local catalog). In the Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings information for the two source records based on the different configuration settings for institutions A and B (because MultiBackend is specifically designed to allow each institution its own ILS configuration settings). However, when a patron decides to request one of these items, say A.111, VuFind will attempt to load this Solr record based on the Solr settings in config.ini for the universal catalog. But the source record A.111 exists in a different Solr instance (for the local catalog). This is not a problem if I store both universal records (of the type UC.123) and local records (of the type A.111) in the same Solr instance. But it becomes a big problem if I separate the local and universal catalog into two separate Solr instances (or cores).

Perhaps I'm still missing something and there's still a way to get this to work using sharding or something else. I just thought it might be worthwhile to explain my situation in a different way (above), in case it wasn't completely clear in my previous email. I have to admit; it can get confusing at times (for me, anyway). It's also possible that it makes sense to keep using a single Solr instance for both sets/types of records.

Chris

 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 12:56 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Delis, Christopher
OK, I will give it a try. Thanks! 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 2:19 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

I don't claim to be an expert on Solr sharding, but I still think it might be useful for your situation. Sharding can be used as a way of making multiple Solr indexes behave as though they are a single index, since it distributes a search across all of the shards. I believe VuFind even contains some smarts so that, if you disable one shard for searching purposes, it still turns on that shard for record retrieval purposes, to ensure that all record IDs can be retrieved. So I think you might be able to get away with creating "universal" and "local" shards, with "universal" on by default. That will cause users to search the UC records, but also offer the ability to retrieve the A and B records when necessary. I may be overly optimistic about this -- as I say, I haven't worked too deeply with this functionality -- but I have a feeling it might actually work as desired out of the box, and if not, you might be able to get there with a bit of custom listener work.


- Demian


From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 2:56 PM
To: Demian Katz; [hidden email]
Subject: RE: Best way to switch between two Solr instances/cores?
 
Thanks for reminding me about sharding support; I forgot it existed. However, I don't think the will help us in our case. I'm guessing sharding would be another way to separate the two catalogs --local and universal-- during the Search stage. But I already have a solution for that part using multisite configurations (each catalog --local and universal-- has its own config.ini section where I can configure Solr to use different instances/cores). 

The problem I'm dealing with occurs *after* the initial search takes place. The search itself is fine as it searches the correct Solr instance. E.g., Let's assume a patron searches the universal catalog and then proceeds to the Detailed record page for record UC.123. Remember, the record UC.123 is a representation of a deduplicated set of source records (which reside in the local catalog). Let's also assume that UC.123 has two source records of A.111 and B.222 (for institutions A and B respectively, both of which again reside in the local catalog). In the Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings information for the two source records based on the different configuration settings for institutions A and B (because MultiBackend is specifically designed to allow each institution its own ILS configuration settings). However, when a patron decides to request one of these items, say A.111, VuFind will attempt to load this Solr record based on the Solr settings in config.ini for the universal catalog. But the source record A.111 exists in a different Solr instance (for the local catalog). This is not a problem if I store both universal records (of the type UC.123) and local records (of the type A.111) in the same Solr instance. But it becomes a big problem if I separate the local and universal catalog into two separate Solr instances (or cores).

Perhaps I'm still missing something and there's still a way to get this to work using sharding or something else. I just thought it might be worthwhile to explain my situation in a different way (above), in case it wasn't completely clear in my previous email. I have to admit; it can get confusing at times (for me, anyway). It's also possible that it makes sense to keep using a single Solr instance for both sets/types of records.

Chris

 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 12:56 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Delis, Christopher
For the record (pun intended), it worked :-) Next time, I will simply try your suggestion before replying! Thank you so much, Demian! 

Chris

From: Delis, Christopher [[hidden email]]
Sent: Friday, February 03, 2017 2:32 PM
To: Demian Katz; [hidden email]
Subject: Re: [VuFind-Tech] Best way to switch between two Solr instances/cores?

OK, I will give it a try. Thanks! 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 2:19 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

I don't claim to be an expert on Solr sharding, but I still think it might be useful for your situation. Sharding can be used as a way of making multiple Solr indexes behave as though they are a single index, since it distributes a search across all of the shards. I believe VuFind even contains some smarts so that, if you disable one shard for searching purposes, it still turns on that shard for record retrieval purposes, to ensure that all record IDs can be retrieved. So I think you might be able to get away with creating "universal" and "local" shards, with "universal" on by default. That will cause users to search the UC records, but also offer the ability to retrieve the A and B records when necessary. I may be overly optimistic about this -- as I say, I haven't worked too deeply with this functionality -- but I have a feeling it might actually work as desired out of the box, and if not, you might be able to get there with a bit of custom listener work.


- Demian


From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 2:56 PM
To: Demian Katz; [hidden email]
Subject: RE: Best way to switch between two Solr instances/cores?
 
Thanks for reminding me about sharding support; I forgot it existed. However, I don't think the will help us in our case. I'm guessing sharding would be another way to separate the two catalogs --local and universal-- during the Search stage. But I already have a solution for that part using multisite configurations (each catalog --local and universal-- has its own config.ini section where I can configure Solr to use different instances/cores). 

The problem I'm dealing with occurs *after* the initial search takes place. The search itself is fine as it searches the correct Solr instance. E.g., Let's assume a patron searches the universal catalog and then proceeds to the Detailed record page for record UC.123. Remember, the record UC.123 is a representation of a deduplicated set of source records (which reside in the local catalog). Let's also assume that UC.123 has two source records of A.111 and B.222 (for institutions A and B respectively, both of which again reside in the local catalog). In the Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings information for the two source records based on the different configuration settings for institutions A and B (because MultiBackend is specifically designed to allow each institution its own ILS configuration settings). However, when a patron decides to request one of these items, say A.111, VuFind will attempt to load this Solr record based on the Solr settings in config.ini for the universal catalog. But the source record A.111 exists in a different Solr instance (for the local catalog). This is not a problem if I store both universal records (of the type UC.123) and local records (of the type A.111) in the same Solr instance. But it becomes a big problem if I separate the local and universal catalog into two separate Solr instances (or cores).

Perhaps I'm still missing something and there's still a way to get this to work using sharding or something else. I just thought it might be worthwhile to explain my situation in a different way (above), in case it wasn't completely clear in my previous email. I have to admit; it can get confusing at times (for me, anyway). It's also possible that it makes sense to keep using a single Solr instance for both sets/types of records.

Chris

 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 12:56 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Demian Katz

That's great news! Like I said, though, you'll definitely want to poke around into some corners of the system to be sure that everything is still working right, since sharding can have impact on relevance ranking, the related record engine, etc. Hopefully nothing too terrible will turn up, though, and if you do run into trouble, I think there's a fairly good chance that upgrading to newer versions of Solr may help, as the system has been getting better and better at dealing with distributed data.


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 3:49 PM
To: Demian Katz; [hidden email]
Subject: RE: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
For the record (pun intended), it worked :-) Next time, I will simply try your suggestion before replying! Thank you so much, Demian! 

Chris

From: Delis, Christopher [[hidden email]]
Sent: Friday, February 03, 2017 2:32 PM
To: Demian Katz; [hidden email]
Subject: Re: [VuFind-Tech] Best way to switch between two Solr instances/cores?

OK, I will give it a try. Thanks! 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 2:19 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

I don't claim to be an expert on Solr sharding, but I still think it might be useful for your situation. Sharding can be used as a way of making multiple Solr indexes behave as though they are a single index, since it distributes a search across all of the shards. I believe VuFind even contains some smarts so that, if you disable one shard for searching purposes, it still turns on that shard for record retrieval purposes, to ensure that all record IDs can be retrieved. So I think you might be able to get away with creating "universal" and "local" shards, with "universal" on by default. That will cause users to search the UC records, but also offer the ability to retrieve the A and B records when necessary. I may be overly optimistic about this -- as I say, I haven't worked too deeply with this functionality -- but I have a feeling it might actually work as desired out of the box, and if not, you might be able to get there with a bit of custom listener work.


- Demian


From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 2:56 PM
To: Demian Katz; [hidden email]
Subject: RE: Best way to switch between two Solr instances/cores?
 
Thanks for reminding me about sharding support; I forgot it existed. However, I don't think the will help us in our case. I'm guessing sharding would be another way to separate the two catalogs --local and universal-- during the Search stage. But I already have a solution for that part using multisite configurations (each catalog --local and universal-- has its own config.ini section where I can configure Solr to use different instances/cores). 

The problem I'm dealing with occurs *after* the initial search takes place. The search itself is fine as it searches the correct Solr instance. E.g., Let's assume a patron searches the universal catalog and then proceeds to the Detailed record page for record UC.123. Remember, the record UC.123 is a representation of a deduplicated set of source records (which reside in the local catalog). Let's also assume that UC.123 has two source records of A.111 and B.222 (for institutions A and B respectively, both of which again reside in the local catalog). In the Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings information for the two source records based on the different configuration settings for institutions A and B (because MultiBackend is specifically designed to allow each institution its own ILS configuration settings). However, when a patron decides to request one of these items, say A.111, VuFind will attempt to load this Solr record based on the Solr settings in config.ini for the universal catalog. But the source record A.111 exists in a different Solr instance (for the local catalog). This is not a problem if I store both universal records (of the type UC.123) and local records (of the type A.111) in the same Solr instance. But it becomes a big problem if I separate the local and universal catalog into two separate Solr instances (or cores).

Perhaps I'm still missing something and there's still a way to get this to work using sharding or something else. I just thought it might be worthwhile to explain my situation in a different way (above), in case it wasn't completely clear in my previous email. I have to admit; it can get confusing at times (for me, anyway). It's also possible that it makes sense to keep using a single Solr instance for both sets/types of records.

Chris

 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 12:56 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Ere Maijala
In reply to this post by Delis, Christopher
One option might be be to use search tabs with filters so that one tab
only displays local records and the other displays UC records. These are
configures in sections SearchTabs and SearchTabsFilters in config.ini.

--Ere

3.2.2017, 21.56, Delis, Christopher kirjoitti:

> Thanks for reminding me about sharding support; I forgot it existed.
> However, I don't think the will help us in our case. I'm guessing
> sharding would be another way to separate the two catalogs --local and
> universal-- during the Search stage. But I already have a solution for
> that part using multisite configurations (each catalog --local and
> universal-- has its own config.ini section where I can configure Solr to
> use different instances/cores).
>
> The problem I'm dealing with occurs *after* the initial search takes
> place. The search itself is fine as it searches the correct Solr
> instance. E.g., Let's assume a patron searches the universal catalog and
> then proceeds to the Detailed record page for record UC.123. Remember,
> the record UC.123 is a representation of a deduplicated set of source
> records (which reside in the local catalog). Let's also assume that
> UC.123 has two source records of A.111 and B.222 (for institutions A and
> B respectively, both of which again reside in the local catalog). In the
> Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings
> information for the two source records based on the different
> configuration settings for institutions A and B (because MultiBackend is
> specifically designed to allow each institution its own ILS
> configuration settings). However, when a patron decides to request one
> of these items, say A.111, VuFind will attempt to load this Solr record
> based on the Solr settings in config.ini for the universal catalog. But
> the source record A.111 exists in a different Solr instance (for the
> local catalog). This is not a problem if I store both universal records
> (of the type UC.123) and local records (of the type A.111) in the same
> Solr instance. But it becomes a big problem if I separate the local and
> universal catalog into two separate Solr instances (or cores).
>
> Perhaps I'm still missing something and there's still a way to get this
> to work using sharding or something else. I just thought it might be
> worthwhile to explain my situation in a different way (above), in case
> it wasn't completely clear in my previous email. I have to admit; it can
> get confusing at times (for me, anyway). It's also possible that it
> makes sense to keep using a single Solr instance for both sets/types of
> records.
>
> Chris
>
>
> ------------------------------------------------------------------------
> *From:* Demian Katz [[hidden email]]
> *Sent:* Friday, February 03, 2017 12:56 PM
> *To:* Delis, Christopher; [hidden email]
> *Subject:* Re: Best way to switch between two Solr instances/cores?
>
> Have you taken a look at Solr's built-in sharding functionality? There
> are some settings relating to this here:
>
>
> https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_vufind-2Dorg_vufind_blob_master_config_vufind_searches.ini-23L528&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=LyCuqT0xcuQQ20GqMLa3Wfzhzq9sWYVmAYRobfnVaSA&e=>
>
>
> This is a good way to spread records across multiple Solr indexes. I'm
> not sure if this is ideal for your use case -- there are certainly some
> downsides, including certain features that don't work as well in a
> sharded environment -- but if it works, I think it would be a very
> simple solution. It's certainly worth experimenting with if you haven't
> already.
>
>
> If that's no good, let us know and we can give it some more thought...
> but I figured I'd lead with the easy, available option first, in case
> this was news to you. :-)
>
>
> - Demian
>
>
> ------------------------------------------------------------------------
> *From:* Delis, Christopher <[hidden email]>
> *Sent:* Friday, February 3, 2017 1:09 PM
> *To:* [hidden email]
> *Subject:* [VuFind-Tech] Best way to switch between two Solr
> instances/cores?
>
> Background:
>
> I currently have a VuFind setup (for a consortium of libraries) that
> makes use of the MultiBackend ILS Driver to manage backend ILS services.
> This driver is working rather well, I might add! To complicate things
> further (or, to make things more "interesting," you might say), we not
> only have individual "local" catalogs for each library, we also have a
> single "universal" catalog for the consortium at large (which is a
> deduplicated set of the combined local catalogs). We also make use of
> the deduplication module (
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fvufind.org-252Fwiki-252Findexing-253Adeduplication-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3DNVCyRsAZsKbSRixPpYb3xRKLaYwlgi-252Bw9BVOyb61sc0-253D-26reserved-3D0&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=19KRJ_O0c7SgLsNIqgGhBGrZBz76qDfTvOYpnsPSvmQ&e=>
> ) to distinguish between the "local" and "universal" catalog during
> searches. This, too, is working rather well, I might add.
>
> However, our Solr index is quite large (> 200GB). But perhaps more
> importantly, the import process for both the "local" and the "universal"
> catalog are independent of each other. Because of this, I would like to
> separate this single Solr instance into two separate Solr instances (or
> cores). Doing so will allow me to index one without causing potential
> harm to the other, e.g., during a Solr "optimize," I wouldn't want to
> run it after importing records from one catalog while the other is still
> in the process of importing records (sure, I could come up with a way to
> coordinate this, but I'd rather not have to). Plus, it probably makes
> sense to have two smaller Solr instances (or cores) than the one very
> large one.
>
> In any case, here's my problem:
>
> While I am able to easily configure VuFind to search the appropriate
> Solr instance (or core) via config.ini (I have a multisite
> configuration), problems arise when I am in the "universal" catalog and
> attempt to request an item from a local catalog. This is also where the
> MultiBackend ILS driver shines: it correctly connects to the appropriate
> backend ILS driver based on the record format (e.g., InstitutionA.123
> goes to InstitutionA's driver and loads record 123). HOWEVER, since the
> local record resides in a different Solr instance (or core), VuFind will
> respond with a "record not found" error because it is assuming that the
> record belongs in the "universal" catalog's Solr instance (or core),
> which is different.
>
> I'm not sure what the best way to approach this might be, and wondered
> if anyone had a suggestion. If there were an analog to the MultiBackend
> ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind
> to the appropriate Solr instance (or core) based on record format, then
> that would solve my problem. Another potential solution might be to set
> up an nginx web server proxy to switch between the two Solr
> instances/cores based on the Solr query.
>
> Thanks in advance,
> Chris
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org!
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fsdm.link-252Fslashdot-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3Dq-252BvZLQqF2cWIfr9Gwy-252BoREHrkIvD3nUmC54kQz9xoQY-253D-26reserved-3D0&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=nNaDimPVaa0kqIEQ8G7JTqIWK3SJuTq53mwxKUyZ2cY&e=>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Flists.sourceforge.net-252Flists-252Flistinfo-252Fvufind-2Dtech-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3DrGN3OHb-252B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ-253D-26reserved-3D0&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=UDQdBGAI992FUgFQTTGnnoXbGJGZcO01sQX1sQqqUvc&e=>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Delis, Christopher
Ere,

I had already solved that particular separation issue. I have a multisite VuFind configuration: one for the local catalog, and the other for the UC. Since they each have their own set of configuration files (config.ini, searches.ini, etc.), I was able to easily configure each one to search the appropriate Solr index.

The problem occurs when a patron requests an item in the UC catalog. Here, VuFind must be able to load the *local* record (not the UC record) in order to fulfill the request. Using the shard configuration in the UC catalog (in searches.ini), VuFind is now able to search the local Solr index (in addition the already-configured UC Solr index in config.ini) in order to load the local record during the request phase. (BTW, this is the only scenario where the UC catalog needs access to the local record. Everywhere else the Solr setting in config.ini --which is configured to search the UC Solr index-- works just fine.)

Chris
________________________________________
From: Ere Maijala [[hidden email]]
Sent: Monday, February 06, 2017 2:27 AM
To: [hidden email]
Subject: Re: [VuFind-Tech] Best way to switch between two Solr instances/cores?

One option might be be to use search tabs with filters so that one tab
only displays local records and the other displays UC records. These are
configures in sections SearchTabs and SearchTabsFilters in config.ini.

--Ere

3.2.2017, 21.56, Delis, Christopher kirjoitti:

> Thanks for reminding me about sharding support; I forgot it existed.
> However, I don't think the will help us in our case. I'm guessing
> sharding would be another way to separate the two catalogs --local and
> universal-- during the Search stage. But I already have a solution for
> that part using multisite configurations (each catalog --local and
> universal-- has its own config.ini section where I can configure Solr to
> use different instances/cores).
>
> The problem I'm dealing with occurs *after* the initial search takes
> place. The search itself is fine as it searches the correct Solr
> instance. E.g., Let's assume a patron searches the universal catalog and
> then proceeds to the Detailed record page for record UC.123. Remember,
> the record UC.123 is a representation of a deduplicated set of source
> records (which reside in the local catalog). Let's also assume that
> UC.123 has two source records of A.111 and B.222 (for institutions A and
> B respectively, both of which again reside in the local catalog). In the
> Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings
> information for the two source records based on the different
> configuration settings for institutions A and B (because MultiBackend is
> specifically designed to allow each institution its own ILS
> configuration settings). However, when a patron decides to request one
> of these items, say A.111, VuFind will attempt to load this Solr record
> based on the Solr settings in config.ini for the universal catalog. But
> the source record A.111 exists in a different Solr instance (for the
> local catalog). This is not a problem if I store both universal records
> (of the type UC.123) and local records (of the type A.111) in the same
> Solr instance. But it becomes a big problem if I separate the local and
> universal catalog into two separate Solr instances (or cores).
>
> Perhaps I'm still missing something and there's still a way to get this
> to work using sharding or something else. I just thought it might be
> worthwhile to explain my situation in a different way (above), in case
> it wasn't completely clear in my previous email. I have to admit; it can
> get confusing at times (for me, anyway). It's also possible that it
> makes sense to keep using a single Solr instance for both sets/types of
> records.
>
> Chris
>
>
> ------------------------------------------------------------------------
> *From:* Demian Katz [[hidden email]]
> *Sent:* Friday, February 03, 2017 12:56 PM
> *To:* Delis, Christopher; [hidden email]
> *Subject:* Re: Best way to switch between two Solr instances/cores?
>
> Have you taken a look at Solr's built-in sharding functionality? There
> are some settings relating to this here:
>
>
> https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_vufind-2Dorg_vufind_blob_master_config_vufind_searches.ini-23L528&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=LyCuqT0xcuQQ20GqMLa3Wfzhzq9sWYVmAYRobfnVaSA&e=>
>
>
> This is a good way to spread records across multiple Solr indexes. I'm
> not sure if this is ideal for your use case -- there are certainly some
> downsides, including certain features that don't work as well in a
> sharded environment -- but if it works, I think it would be a very
> simple solution. It's certainly worth experimenting with if you haven't
> already.
>
>
> If that's no good, let us know and we can give it some more thought...
> but I figured I'd lead with the easy, available option first, in case
> this was news to you. :-)
>
>
> - Demian
>
>
> ------------------------------------------------------------------------
> *From:* Delis, Christopher <[hidden email]>
> *Sent:* Friday, February 3, 2017 1:09 PM
> *To:* [hidden email]
> *Subject:* [VuFind-Tech] Best way to switch between two Solr
> instances/cores?
>
> Background:
>
> I currently have a VuFind setup (for a consortium of libraries) that
> makes use of the MultiBackend ILS Driver to manage backend ILS services.
> This driver is working rather well, I might add! To complicate things
> further (or, to make things more "interesting," you might say), we not
> only have individual "local" catalogs for each library, we also have a
> single "universal" catalog for the consortium at large (which is a
> deduplicated set of the combined local catalogs). We also make use of
> the deduplication module (
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fvufind.org-252Fwiki-252Findexing-253Adeduplication-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3DNVCyRsAZsKbSRixPpYb3xRKLaYwlgi-252Bw9BVOyb61sc0-253D-26reserved-3D0&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=19KRJ_O0c7SgLsNIqgGhBGrZBz76qDfTvOYpnsPSvmQ&e=>
> ) to distinguish between the "local" and "universal" catalog during
> searches. This, too, is working rather well, I might add.
>
> However, our Solr index is quite large (> 200GB). But perhaps more
> importantly, the import process for both the "local" and the "universal"
> catalog are independent of each other. Because of this, I would like to
> separate this single Solr instance into two separate Solr instances (or
> cores). Doing so will allow me to index one without causing potential
> harm to the other, e.g., during a Solr "optimize," I wouldn't want to
> run it after importing records from one catalog while the other is still
> in the process of importing records (sure, I could come up with a way to
> coordinate this, but I'd rather not have to). Plus, it probably makes
> sense to have two smaller Solr instances (or cores) than the one very
> large one.
>
> In any case, here's my problem:
>
> While I am able to easily configure VuFind to search the appropriate
> Solr instance (or core) via config.ini (I have a multisite
> configuration), problems arise when I am in the "universal" catalog and
> attempt to request an item from a local catalog. This is also where the
> MultiBackend ILS driver shines: it correctly connects to the appropriate
> backend ILS driver based on the record format (e.g., InstitutionA.123
> goes to InstitutionA's driver and loads record 123). HOWEVER, since the
> local record resides in a different Solr instance (or core), VuFind will
> respond with a "record not found" error because it is assuming that the
> record belongs in the "universal" catalog's Solr instance (or core),
> which is different.
>
> I'm not sure what the best way to approach this might be, and wondered
> if anyone had a suggestion. If there were an analog to the MultiBackend
> ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind
> to the appropriate Solr instance (or core) based on record format, then
> that would solve my problem. Another potential solution might be to set
> up an nginx web server proxy to switch between the two Solr
> instances/cores based on the Solr query.
>
> Thanks in advance,
> Chris
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org!
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fsdm.link-252Fslashdot-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3Dq-252BvZLQqF2cWIfr9Gwy-252BoREHrkIvD3nUmC54kQz9xoQY-253D-26reserved-3D0&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=nNaDimPVaa0kqIEQ8G7JTqIWK3SJuTq53mwxKUyZ2cY&e=>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Flists.sourceforge.net-252Flists-252Flistinfo-252Fvufind-2Dtech-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3DrGN3OHb-252B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ-253D-26reserved-3D0&d=DwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=RTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4&m=LRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc&s=UDQdBGAI992FUgFQTTGnnoXbGJGZcO01sQX1sQqqUvc&e=>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Delis, Christopher
In reply to this post by Demian Katz
FYI, I notice that VuFind's sharding support does not include the "Similar Items" feature in the Detailed Record page (the Solr "MoreLikeThis" queries). While I see the other Solr searches (basic, advanced, facets) working across the different shards (via solr.log), the "MoreLikeThis" searches are only made to the Solr server defined in config.ini. I just thought I'd pass this along. I'm not entirely certain that we will enable "Similar Items" in our catalog. But if we do, then we'd need to come up with an alternative solution. Chris



From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 3:00 PM
To: Delis, Christopher; [hidden email]
Subject: Re: [VuFind-Tech] Best way to switch between two Solr instances/cores?

That's great news! Like I said, though, you'll definitely want to poke around into some corners of the system to be sure that everything is still working right, since sharding can have impact on relevance ranking, the related record engine, etc. Hopefully nothing too terrible will turn up, though, and if you do run into trouble, I think there's a fairly good chance that upgrading to newer versions of Solr may help, as the system has been getting better and better at dealing with distributed data.


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 3:49 PM
To: Demian Katz; [hidden email]
Subject: RE: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
For the record (pun intended), it worked :-) Next time, I will simply try your suggestion before replying! Thank you so much, Demian! 

Chris

From: Delis, Christopher [[hidden email]]
Sent: Friday, February 03, 2017 2:32 PM
To: Demian Katz; [hidden email]
Subject: Re: [VuFind-Tech] Best way to switch between two Solr instances/cores?

OK, I will give it a try. Thanks! 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 2:19 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

I don't claim to be an expert on Solr sharding, but I still think it might be useful for your situation. Sharding can be used as a way of making multiple Solr indexes behave as though they are a single index, since it distributes a search across all of the shards. I believe VuFind even contains some smarts so that, if you disable one shard for searching purposes, it still turns on that shard for record retrieval purposes, to ensure that all record IDs can be retrieved. So I think you might be able to get away with creating "universal" and "local" shards, with "universal" on by default. That will cause users to search the UC records, but also offer the ability to retrieve the A and B records when necessary. I may be overly optimistic about this -- as I say, I haven't worked too deeply with this functionality -- but I have a feeling it might actually work as desired out of the box, and if not, you might be able to get there with a bit of custom listener work.


- Demian


From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 2:56 PM
To: Demian Katz; [hidden email]
Subject: RE: Best way to switch between two Solr instances/cores?
 
Thanks for reminding me about sharding support; I forgot it existed. However, I don't think the will help us in our case. I'm guessing sharding would be another way to separate the two catalogs --local and universal-- during the Search stage. But I already have a solution for that part using multisite configurations (each catalog --local and universal-- has its own config.ini section where I can configure Solr to use different instances/cores). 

The problem I'm dealing with occurs *after* the initial search takes place. The search itself is fine as it searches the correct Solr instance. E.g., Let's assume a patron searches the universal catalog and then proceeds to the Detailed record page for record UC.123. Remember, the record UC.123 is a representation of a deduplicated set of source records (which reside in the local catalog). Let's also assume that UC.123 has two source records of A.111 and B.222 (for institutions A and B respectively, both of which again reside in the local catalog). In the Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings information for the two source records based on the different configuration settings for institutions A and B (because MultiBackend is specifically designed to allow each institution its own ILS configuration settings). However, when a patron decides to request one of these items, say A.111, VuFind will attempt to load this Solr record based on the Solr settings in config.ini for the universal catalog. But the source record A.111 exists in a different Solr instance (for the local catalog). This is not a problem if I store both universal records (of the type UC.123) and local records (of the type A.111) in the same Solr instance. But it becomes a big problem if I separate the local and universal catalog into two separate Solr instances (or cores).

Perhaps I'm still missing something and there's still a way to get this to work using sharding or something else. I just thought it might be worthwhile to explain my situation in a different way (above), in case it wasn't completely clear in my previous email. I have to admit; it can get confusing at times (for me, anyway). It's also possible that it makes sense to keep using a single Solr instance for both sets/types of records.

Chris

 

From: Demian Katz [[hidden email]]
Sent: Friday, February 03, 2017 12:56 PM
To: Delis, Christopher; [hidden email]
Subject: Re: Best way to switch between two Solr instances/cores?

Have you taken a look at Solr's built-in sharding functionality? There are some settings relating to this here:


https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528


This is a good way to spread records across multiple Solr indexes. I'm not sure if this is ideal for your use case -- there are certainly some downsides, including certain features that don't work as well in a sharded environment -- but if it works, I think it would be a very simple solution. It's certainly worth experimenting with if you haven't already.


If that's no good, let us know and we can give it some more thought... but I figured I'd lead with the easy, available option first, in case this was news to you. :-)


- Demian



From: Delis, Christopher <[hidden email]>
Sent: Friday, February 3, 2017 1:09 PM
To: [hidden email]
Subject: [VuFind-Tech] Best way to switch between two Solr instances/cores?
 
Background:

I currently have a VuFind setup (for a consortium of libraries) that makes use of the MultiBackend ILS Driver to manage backend ILS services. This driver is working rather well, I might add! To complicate things further (or, to make things more "interesting," you might say), we not only have individual "local" catalogs for each library, we also have a single "universal" catalog for the consortium at large (which is a deduplicated set of the combined local catalogs). We also make use of the deduplication module ( https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0 ) to distinguish between the "local" and "universal" catalog during searches. This, too, is working rather well, I might add.

However, our Solr index is quite large (> 200GB). But perhaps more importantly, the import process for both the "local" and the "universal" catalog are independent of each other. Because of this, I would like to separate this single Solr instance into two separate Solr instances (or cores). Doing so will allow me to index one without causing potential harm to the other, e.g., during a Solr "optimize," I wouldn't want to run it after importing records from one catalog while the other is still in the process of importing records (sure, I could come up with a way to coordinate this, but I'd rather not have to). Plus, it probably makes sense to have two smaller Solr instances (or cores) than the one very large one.

In any case, here's my problem:

While I am able to easily configure VuFind to search the appropriate Solr instance (or core) via config.ini (I have a multisite configuration), problems arise when I am in the "universal" catalog and attempt to request an item from a local catalog. This is also where the MultiBackend ILS driver shines: it correctly connects to the appropriate backend ILS driver based on the record format (e.g., InstitutionA.123 goes to InstitutionA's driver and loads record 123). HOWEVER, since the local record resides in a different Solr instance (or core), VuFind will respond with a "record not found" error because it is assuming that the record belongs in the "universal" catalog's Solr instance (or core), which is different.

I'm not sure what the best way to approach this might be, and wondered if anyone had a suggestion. If there were an analog to the MultiBackend ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind to the appropriate Solr instance (or core) based on record format, then that would solve my problem. Another potential solution might be to set up an nginx web server proxy to switch between the two Solr instances/cores based on the Solr query.

Thanks in advance,
Chris
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Best way to switch between two Solr instances/cores?

Ere Maijala
I'm not sure if it affects traditional sharding, but for some reason the
MLT handler that VuFind uses was never made compatible with sharding in
SolrCloud. Instead a new MLT query parser was created and you were
supposed to use it in a sharded index. There has, however, been a couple
of long-outstanding issues with the new version, and they practically
prevented using it. I've contributed a fix to Solr, and it was included
in Solr 6.4.0. Unfortunately 6.4.0 and 6.4.1 have a serious
performance-killing bug, so they can't really be used. 6.4.2 is supposed
to be the holy grail when it's released (that should pretty soon), but
there have been so many hurdles already that I won't hold my breath.

That said, I've already played a bit with doing MLT in two stages: First
use Solr's MLT but just to request interesting terms from the original
record, and the construct a normal search based on those terms. A quick
test shows that this should work quite well, so unless 6.4.2 is finally
usable, I'll look into making that an option. If you or anyone happens
to need it, the implementation is here:
<https://github.com/NatLibFi/NDL-VuFind2/commit/d909600979b734ecb4bad9e3649ce8223beffed0>

--Ere

20.2.2017, 22.53, Delis, Christopher kirjoitti:

> FYI, I notice that VuFind's sharding support does not include the
> "Similar Items" feature in the Detailed Record page (the Solr
> "MoreLikeThis" queries). While I see the other Solr searches (basic,
> advanced, facets) working across the different shards (via solr.log),
> the "MoreLikeThis" searches are only made to the Solr server defined in
> config.ini. I just thought I'd pass this along. I'm not entirely certain
> that we will enable "Similar Items" in our catalog. But if we do, then
> we'd need to come up with an alternative solution. Chris
>
>
> ------------------------------------------------------------------------
> *From:* Demian Katz [[hidden email]]
> *Sent:* Friday, February 03, 2017 3:00 PM
> *To:* Delis, Christopher; [hidden email]
> *Subject:* Re: [VuFind-Tech] Best way to switch between two Solr
> instances/cores?
>
> That's great news! Like I said, though, you'll definitely want to poke
> around into some corners of the system to be sure that everything is
> still working right, since sharding can have impact on relevance
> ranking, the related record engine, etc. Hopefully nothing too terrible
> will turn up, though, and if you do run into trouble, I think there's a
> fairly good chance that upgrading to newer versions of Solr may help, as
> the system has been getting better and better at dealing with
> distributed data.
>
>
> - Demian
>
>
> ------------------------------------------------------------------------
> *From:* Delis, Christopher <[hidden email]>
> *Sent:* Friday, February 3, 2017 3:49 PM
> *To:* Demian Katz; [hidden email]
> *Subject:* RE: [VuFind-Tech] Best way to switch between two Solr
> instances/cores?
>
> For the record (pun intended), it worked :-) Next time, I will simply
> try your suggestion before replying! Thank you so much, Demian!
>
> Chris
> ------------------------------------------------------------------------
> *From:* Delis, Christopher [[hidden email]]
> *Sent:* Friday, February 03, 2017 2:32 PM
> *To:* Demian Katz; [hidden email]
> *Subject:* Re: [VuFind-Tech] Best way to switch between two Solr
> instances/cores?
>
> OK, I will give it a try. Thanks!
> ------------------------------------------------------------------------
> *From:* Demian Katz [[hidden email]]
> *Sent:* Friday, February 03, 2017 2:19 PM
> *To:* Delis, Christopher; [hidden email]
> *Subject:* Re: Best way to switch between two Solr instances/cores?
>
> I don't claim to be an expert on Solr sharding, but I still think it
> might be useful for your situation. Sharding can be used as a way of
> making multiple Solr indexes behave as though they are a single index,
> since it distributes a search across all of the shards. I believe VuFind
> even contains some smarts so that, if you disable one shard for
> searching purposes, it still turns on that shard for record retrieval
> purposes, to ensure that all record IDs can be retrieved. So I think you
> might be able to get away with creating "universal" and "local" shards,
> with "universal" on by default. That will cause users to search the UC
> records, but also offer the ability to retrieve the A and B records when
> necessary. I may be overly optimistic about this -- as I say, I haven't
> worked too deeply with this functionality -- but I have a feeling it
> might actually work as desired out of the box, and if not, you might be
> able to get there with a bit of custom listener work.
>
>
> - Demian
>
> ------------------------------------------------------------------------
> *From:* Delis, Christopher <[hidden email]>
> *Sent:* Friday, February 3, 2017 2:56 PM
> *To:* Demian Katz; [hidden email]
> *Subject:* RE: Best way to switch between two Solr instances/cores?
>
> Thanks for reminding me about sharding support; I forgot it existed.
> However, I don't think the will help us in our case. I'm guessing
> sharding would be another way to separate the two catalogs --local and
> universal-- during the Search stage. But I already have a solution for
> that part using multisite configurations (each catalog --local and
> universal-- has its own config.ini section where I can configure Solr to
> use different instances/cores).
>
> The problem I'm dealing with occurs *after* the initial search takes
> place. The search itself is fine as it searches the correct Solr
> instance. E.g., Let's assume a patron searches the universal catalog and
> then proceeds to the Detailed record page for record UC.123. Remember,
> the record UC.123 is a representation of a deduplicated set of source
> records (which reside in the local catalog). Let's also assume that
> UC.123 has two source records of A.111 and B.222 (for institutions A and
> B respectively, both of which again reside in the local catalog). In the
> Holdings tab, the MultiBackend ILS Driver correctly looks up Holdings
> information for the two source records based on the different
> configuration settings for institutions A and B (because MultiBackend is
> specifically designed to allow each institution its own ILS
> configuration settings). However, when a patron decides to request one
> of these items, say A.111, VuFind will attempt to load this Solr record
> based on the Solr settings in config.ini for the universal catalog. But
> the source record A.111 exists in a different Solr instance (for the
> local catalog). This is not a problem if I store both universal records
> (of the type UC.123) and local records (of the type A.111) in the same
> Solr instance. But it becomes a big problem if I separate the local and
> universal catalog into two separate Solr instances (or cores).
>
> Perhaps I'm still missing something and there's still a way to get this
> to work using sharding or something else. I just thought it might be
> worthwhile to explain my situation in a different way (above), in case
> it wasn't completely clear in my previous email. I have to admit; it can
> get confusing at times (for me, anyway). It's also possible that it
> makes sense to keep using a single Solr instance for both sets/types of
> records.
>
> Chris
>
>
> ------------------------------------------------------------------------
> *From:* Demian Katz [[hidden email]]
> *Sent:* Friday, February 03, 2017 12:56 PM
> *To:* Delis, Christopher; [hidden email]
> *Subject:* Re: Best way to switch between two Solr instances/cores?
>
> Have you taken a look at Solr's built-in sharding functionality? There
> are some settings relating to this here:
>
>
> https://github.com/vufind-org/vufind/blob/master/config/vufind/searches.ini#L528
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fgithub.com-5Fvufind-2D2Dorg-5Fvufind-5Fblob-5Fmaster-5Fconfig-5Fvufind-5Fsearches.ini-2D23L528-2526d-253DDwMFAw-2526c-253D8hUWFZcy2Z-2DZa5rBPlktOQ-2526r-253DRTubeVA1lI4n-2DcIbV2u3csbF3ijSWHwbrh-5FUbzTjZK4-2526m-253DLRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk-5F08g3Ygc-2526s-253DLyCuqT0xcuQQ20GqMLa3Wfzhzq9sWYVmAYRobfnVaSA-2526e-253D-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257C81d62a0942754c725e0008d44c6ecf52-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217486201885573-26sdata-3D2lOKINR-252Bxm1KcgQWW-252FW80T3Cv4CpoUUU6Qg2069CE04-253D-26reserved-3D0%26d%3DDwMFAw%26c%3D8hUWFZcy2Z-Za5rBPlktOQ%26r%3DRTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4%26m%3DGrqWc19DU0q2mR1qs_pj9kGAsKoPWYNf-sWhlj1n4pk%26s%3DUKeVjjX-JdaT6OxkPlnDyRlhd_Uk8ugsTnUSwPLZRfs%26e%3D&data=02%7C01%7Cdemian.katz%40villanova.edu%7C1fff0f1d0ca04ad8fbca08d44c764d4a%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217518370318988&sdata=e52phjlX29vHIa%2BGkOKNxSYYZfPeeX8m3UvBKgWXpmU%3D&reserved=0>
>
>
> This is a good way to spread records across multiple Solr indexes. I'm
> not sure if this is ideal for your use case -- there are certainly some
> downsides, including certain features that don't work as well in a
> sharded environment -- but if it works, I think it would be a very
> simple solution. It's certainly worth experimenting with if you haven't
> already.
>
>
> If that's no good, let us know and we can give it some more thought...
> but I figured I'd lead with the easy, available option first, in case
> this was news to you. :-)
>
>
> - Demian
>
>
> ------------------------------------------------------------------------
> *From:* Delis, Christopher <[hidden email]>
> *Sent:* Friday, February 3, 2017 1:09 PM
> *To:* [hidden email]
> *Subject:* [VuFind-Tech] Best way to switch between two Solr
> instances/cores?
>
> Background:
>
> I currently have a VuFind setup (for a consortium of libraries) that
> makes use of the MultiBackend ILS Driver to manage backend ILS services.
> This driver is working rather well, I might add! To complicate things
> further (or, to make things more "interesting," you might say), we not
> only have individual "local" catalogs for each library, we also have a
> single "universal" catalog for the consortium at large (which is a
> deduplicated set of the combined local catalogs). We also make use of
> the deduplication module (
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fvufind.org%2Fwiki%2Findexing%3Adeduplication&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=NVCyRsAZsKbSRixPpYb3xRKLaYwlgi%2Bw9BVOyb61sc0%3D&reserved=0
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fvufind.org-252Fwiki-252Findexing-253Adeduplication-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3DNVCyRsAZsKbSRixPpYb3xRKLaYwlgi-252Bw9BVOyb61sc0-253D-26reserved-3D0%26d%3DDwMFAw%26c%3D8hUWFZcy2Z-Za5rBPlktOQ%26r%3DRTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4%26m%3DLRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc%26s%3D19KRJ_O0c7SgLsNIqgGhBGrZBz76qDfTvOYpnsPSvmQ%26e%3D&data=02%7C01%7Cdemian.katz%40villanova.edu%7C81d62a0942754c725e0008d44c6ecf52%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217486201895581&sdata=IPRzfimP1R0SYSIeUNzXiZS7M62GDxHAP8LAc%2BeHjrM%3D&reserved=0>
> ) to distinguish between the "local" and "universal" catalog during
> searches. This, too, is working rather well, I might add.
>
> However, our Solr index is quite large (> 200GB). But perhaps more
> importantly, the import process for both the "local" and the "universal"
> catalog are independent of each other. Because of this, I would like to
> separate this single Solr instance into two separate Solr instances (or
> cores). Doing so will allow me to index one without causing potential
> harm to the other, e.g., during a Solr "optimize," I wouldn't want to
> run it after importing records from one catalog while the other is still
> in the process of importing records (sure, I could come up with a way to
> coordinate this, but I'd rather not have to). Plus, it probably makes
> sense to have two smaller Solr instances (or cores) than the one very
> large one.
>
> In any case, here's my problem:
>
> While I am able to easily configure VuFind to search the appropriate
> Solr instance (or core) via config.ini (I have a multisite
> configuration), problems arise when I am in the "universal" catalog and
> attempt to request an item from a local catalog. This is also where the
> MultiBackend ILS driver shines: it correctly connects to the appropriate
> backend ILS driver based on the record format (e.g., InstitutionA.123
> goes to InstitutionA's driver and loads record 123). HOWEVER, since the
> local record resides in a different Solr instance (or core), VuFind will
> respond with a "record not found" error because it is assuming that the
> record belongs in the "universal" catalog's Solr instance (or core),
> which is different.
>
> I'm not sure what the best way to approach this might be, and wondered
> if anyone had a suggestion. If there were an analog to the MultiBackend
> ILS driver (e.g., a MultiBackend Solr driver) that could direct VuFind
> to the appropriate Solr instance (or core) based on record format, then
> that would solve my problem. Another potential solution might be to set
> up an nginx web server proxy to switch between the two Solr
> instances/cores based on the Solr query.
>
> Thanks in advance,
> Chris
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org!
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=q%2BvZLQqF2cWIfr9Gwy%2BoREHrkIvD3nUmC54kQz9xoQY%3D&reserved=0
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fsdm.link-252Fslashdot-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3Dq-252BvZLQqF2cWIfr9Gwy-252BoREHrkIvD3nUmC54kQz9xoQY-253D-26reserved-3D0%26d%3DDwMFAw%26c%3D8hUWFZcy2Z-Za5rBPlktOQ%26r%3DRTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4%26m%3DLRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc%26s%3DnNaDimPVaa0kqIEQ8G7JTqIWK3SJuTq53mwxKUyZ2cY%26e%3D&data=02%7C01%7Cdemian.katz%40villanova.edu%7C81d62a0942754c725e0008d44c6ecf52%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217486201895581&sdata=e1AvRt8dIkFYy0%2B%2B8LW4%2BhSTcMAlQREQvrA03PW%2FZdc%3D&reserved=0>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=02%7C01%7Cdemian.katz%40villanova.edu%7Ced340ab41f984dc7c5d108d44c620a57%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217431344535412&sdata=rGN3OHb%2B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ%3D&reserved=0
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Flists.sourceforge.net-252Flists-252Flistinfo-252Fvufind-2Dtech-26data-3D02-257C01-257Cdemian.katz-2540villanova.edu-257Ced340ab41f984dc7c5d108d44c620a57-257C765a8de5cf9444f09cafae5bf8cfa366-257C1-257C0-257C636217431344535412-26sdata-3DrGN3OHb-252B9hgjPVC9jyOgDVqOuEzMe8Lw3Jv29ia1YyQ-253D-26reserved-3D0%26d%3DDwMFAw%26c%3D8hUWFZcy2Z-Za5rBPlktOQ%26r%3DRTubeVA1lI4n-cIbV2u3csbF3ijSWHwbrh_UbzTjZK4%26m%3DLRN64fh4cjCEe2SYVV2TICiRGXR3FaNfJTk_08g3Ygc%26s%3DUDQdBGAI992FUgFQTTGnnoXbGJGZcO01sQX1sQqqUvc%26e%3D&data=02%7C01%7Cdemian.katz%40villanova.edu%7C81d62a0942754c725e0008d44c6ecf52%7C765a8de5cf9444f09cafae5bf8cfa366%7C1%7C0%7C636217486201895581&sdata=fNvXojc2bYIIbVKAV2LBo%2BDmFdGN8IHq3ZHDZDGLYr4%3D&reserved=0>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech