Quantcast

Adding marc tags for indexing

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Adding marc tags for indexing

Wong, Philip

Hi,

 

We tried to add some local mac tags for indexing, e.g.

 

in marc_local.properties, we defined a new "localfields"

    localfields = 964a:965a

 

in schema.xml, we added:  

    <field name="localfields" type="text" indexed="true" stored="true" multiValued="true"/>

 

in search.ini, we added a new search option for localfields.

 

After reindexing, the local tags become searchable. However, the retrieval is not limited to the local tags, but also keywords appearing in titles or subjects. For example, searching "parent" in localfields will also retrieve "parent" in title of a record which does not contain local tag 964

 

What is possibly missing in the new index definitions? We run this in development server and have rebuilt the index to make sure there is no cache. Thanks!

 

Philip

Systems Librarian
Legislative Library of BC


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Adding marc tags for indexing

Demian Katz

The one piece that is missing from your description is defining a search handler in searchspecs.yaml for your new field. The value in search.ini doesn’t refer directly to a Solr field but rather to a set of configurations in searchspecs.yaml – this is how VuFind is able to search more than one field at a time.

 

If that doesn’t help, please let me know and I’ll be happy to elaborate!

 

Also note that if you use dynamic field suffixes, you can avoid the need to modify schema.xml. I think calling the field localfields_txt_mv would take advantage of the default multi-valued text suffix and let you skip the schema modifications. (Obviously it’s fine to modify the schema if you prefer – but having one less customized file can save time in upgrades).

 

- Demian

 

From: Wong, Philip [mailto:[hidden email]]
Sent: Tuesday, April 04, 2017 6:40 PM
To: [hidden email]
Cc: Wong, Philip
Subject: [VuFind-General] Adding marc tags for indexing

 

Hi,

 

We tried to add some local mac tags for indexing, e.g.

 

in marc_local.properties, we defined a new "localfields"

    localfields = 964a:965a

 

in schema.xml, we added:  

    <field name="localfields" type="text" indexed="true" stored="true" multiValued="true"/>

 

in search.ini, we added a new search option for localfields.

 

After reindexing, the local tags become searchable. However, the retrieval is not limited to the local tags, but also keywords appearing in titles or subjects. For example, searching "parent" in localfields will also retrieve "parent" in title of a record which does not contain local tag 964

 

What is possibly missing in the new index definitions? We run this in development server and have rebuilt the index to make sure there is no cache. Thanks!

 

Philip

Systems Librarian
Legislative Library of BC


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Adding marc tags for indexing

Wong, Philip

Hi Demian,

 

Thanks for the reply. We are with vufind 3.1.1 under Windows. There are more than one searchspecs.yaml under vufind/module, which one should be modified? Thanks.

 

./VuFind/tests/fixtures/configs/1.1/searchspecs.yaml

./VuFind/tests/fixtures/configs/1.2/searchspecs.yaml

./VuFind/tests/fixtures/configs/1.3/searchspecs.yaml

./VuFind/tests/fixtures/configs/1.4/searchspecs.yaml

./VuFindSearch/tests/unit-tests/fixtures/searchspecs.yaml

 

From: Demian Katz [mailto:[hidden email]]
Sent: Wednesday, April 05, 2017 5:04 AM
To: Wong, Philip <[hidden email]>; [hidden email]
Subject: RE: Adding marc tags for indexing

 

The one piece that is missing from your description is defining a search handler in searchspecs.yaml for your new field. The value in search.ini doesn’t refer directly to a Solr field but rather to a set of configurations in searchspecs.yaml – this is how VuFind is able to search more than one field at a time.

 

If that doesn’t help, please let me know and I’ll be happy to elaborate!

 

Also note that if you use dynamic field suffixes, you can avoid the need to modify schema.xml. I think calling the field localfields_txt_mv would take advantage of the default multi-valued text suffix and let you skip the schema modifications. (Obviously it’s fine to modify the schema if you prefer – but having one less customized file can save time in upgrades).

 

- Demian

 

From: Wong, Philip [[hidden email]]
Sent: Tuesday, April 04, 2017 6:40 PM
To: [hidden email]
Cc: Wong, Philip
Subject: [VuFind-General] Adding marc tags for indexing

 

Hi,

 

We tried to add some local mac tags for indexing, e.g.

 

in marc_local.properties, we defined a new "localfields"

    localfields = 964a:965a

 

in schema.xml, we added:  

    <field name="localfields" type="text" indexed="true" stored="true" multiValued="true"/>

 

in search.ini, we added a new search option for localfields.

 

After reindexing, the local tags become searchable. However, the retrieval is not limited to the local tags, but also keywords appearing in titles or subjects. For example, searching "parent" in localfields will also retrieve "parent" in title of a record which does not contain local tag 964

 

What is possibly missing in the new index definitions? We run this in development server and have rebuilt the index to make sure there is no cache. Thanks!

 

Philip

Systems Librarian
Legislative Library of BC


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Adding marc tags for indexing

Wong, Philip

Sorry, I just found that it should be the one under config, please ignore my question. Thanks.

 

From: Wong, Philip
Sent: Wednesday, April 05, 2017 10:15 AM
To: Demian Katz <[hidden email]>; [hidden email]
Cc: Wong, Philip <[hidden email]>
Subject: RE: Adding marc tags for indexing

 

Hi Demian,

 

Thanks for the reply. We are with vufind 3.1.1 under Windows. There are more than one searchspecs.yaml under vufind/module, which one should be modified? Thanks.

 

./VuFind/tests/fixtures/configs/1.1/searchspecs.yaml

./VuFind/tests/fixtures/configs/1.2/searchspecs.yaml

./VuFind/tests/fixtures/configs/1.3/searchspecs.yaml

./VuFind/tests/fixtures/configs/1.4/searchspecs.yaml

./VuFindSearch/tests/unit-tests/fixtures/searchspecs.yaml

 

From: Demian Katz [[hidden email]]
Sent: Wednesday, April 05, 2017 5:04 AM
To: Wong, Philip <[hidden email]>; [hidden email]
Subject: RE: Adding marc tags for indexing

 

The one piece that is missing from your description is defining a search handler in searchspecs.yaml for your new field. The value in search.ini doesn’t refer directly to a Solr field but rather to a set of configurations in searchspecs.yaml – this is how VuFind is able to search more than one field at a time.

 

If that doesn’t help, please let me know and I’ll be happy to elaborate!

 

Also note that if you use dynamic field suffixes, you can avoid the need to modify schema.xml. I think calling the field localfields_txt_mv would take advantage of the default multi-valued text suffix and let you skip the schema modifications. (Obviously it’s fine to modify the schema if you prefer – but having one less customized file can save time in upgrades).

 

- Demian

 

From: Wong, Philip [[hidden email]]
Sent: Tuesday, April 04, 2017 6:40 PM
To: [hidden email]
Cc: Wong, Philip
Subject: [VuFind-General] Adding marc tags for indexing

 

Hi,

 

We tried to add some local mac tags for indexing, e.g.

 

in marc_local.properties, we defined a new "localfields"

    localfields = 964a:965a

 

in schema.xml, we added:  

    <field name="localfields" type="text" indexed="true" stored="true" multiValued="true"/>

 

in search.ini, we added a new search option for localfields.

 

After reindexing, the local tags become searchable. However, the retrieval is not limited to the local tags, but also keywords appearing in titles or subjects. For example, searching "parent" in localfields will also retrieve "parent" in title of a record which does not contain local tag 964

 

What is possibly missing in the new index definitions? We run this in development server and have rebuilt the index to make sure there is no cache. Thanks!

 

Philip

Systems Librarian
Legislative Library of BC


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Loading...