Has anyone else noticed SolrMarc stall completely during indexing? It appears to happen randomly (i.e., it doesn't fail on a particular record or anything like that). I'm suspecting that it might be a deadlock issue in the mulithreaded processing, but I haven't dug deeper into it. I was just wondering if I was the only one experiencing this.
I do index very large sets of records (tens of millions) and the stalling tends to occur every couple of million records or so (e.g., indexing 35 million records recently, it occurred 9 or 10 times). The marc record files are at most 1GB each. I've written a monitor script that looks for stalled index jobs and restarts them automatically, but that's just a workaround. Today, I am attempting to index authority records for the first time and I'm having even worse luck; I can't seem to load a single marc record file (1GB).
I'm about to try upgrading to the latest SolrMarc in case it helps, but I would love to know if others were experiencing this, too.
From: Demian Katz [[hidden email]]
Sent: Monday, December 19, 2016 10:22 AM
To: Zhang, Fan
Cc: [hidden email]
Subject: Re: [VuFind-General] vufind 3.1 indexing error
The SolrMarc indexer found in release 3.1 had some bugs. We released an improved version in 3.1.1, and even more fixes will be incorporated in 3.1.2 (due early next year). If you want the latest SolrMarc, you can get it here:
You should be able to unpack the SolrMarc release into VuFind’s import folder, and rename the solrmarc-core-x.x.x.jar to solrmarc-core.jar to upgrade manually.
If you’re still having trouble, please let me know… and I apologize for the inconvenience! SolrMarc 3.x is going to be a big improvement over 2.x both in features and performance, but these types of big transitions usually have some bumps along the way.
I am working on upgrading our vufind from 2.3 to 3.1. I downloaded vufind-3.1.tar.gz and tried to digest a couple of marc files. First I got an error showing that there is no lib-solrj in the release tar. I went to solrmarc site and downloaded the master branch zip file, and copied the lib-solrj drectory to vufind/import, now I am getting following error:
ERROR [main] (IndexDriver.java:119) - Error processing index configurations: marc.properties, marc_local.properties
ERROR [main] (IndexDriver.java:333) - collection = 9975
ERROR [main] (IndexDriver.java:335) - collection : Error: Illegal character <5> found in scanner state subfield
ERROR [main] (IndexDriver.java:333) - institution = 9974
ERROR [main] (IndexDriver.java:335) - institution : Error: Illegal character <4> found in scanner state subfield
Please help, thanks!
PALS, Minnesota State University, Mankato
Mankato, MN 56001
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
VuFind-General mailing list
|Free forum by Nabble||Edit this page|