Re: MARC / unicode / utf-8 conversions

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MARC / unicode / utf-8 conversions

Bess Sadler

On May 21, 2008, at 2:01 PM, Wayne Graham wrote:

> Not sure if this will answer you question, but here it goes.
>
> The Java that does the indexing has several converters for different
> formats . These include Ansel, ISO5426 (Latin), and ISO 6937 (ASCII).
> The Ansel converter will convert to- and from- the MARC-8 format.  
> Right
> now the code to do the indexing doesn't do any conversion... is this
> something you need? If so, we can do an enhancement request.
> Wayne

> James Farrugia wrote:
>
>> Andrew,
>>
>> Does VuFind offer a MARC to UTF-8 converter?
>>
>> Jim
>>

This unicode conversion is one of the things that is already built  
into solrmarc, so if people start moving to the solrmarc codebase for  
indexing they can skip the separate conversion step. It also does a  
good job of detecting incorrect encoding declarations, which is  
another common problem. To clarify, you would use solrmarc to index  
your records, and you would still use VuFind for everything else. You  
just need to choose the "vufind.properties" file when you index.

I really do believe that the faster we start moving to solrmarc  
(which at this point is a question of writing documentation and  
getting some feedback... it's only being tested by a few people right  
now) the faster we'll get these kinds of problems sorted out.

The solrmarc code base is here: http://code.google.com/p/solrmarc/

Bess

Elizabeth (Bess) Sadler
Research and Development Librarian
Digital Scholarship Services
Box 400129
Alderman Library
University of Virginia
Charlottesville, VA 22904

[hidden email]
(434) 243-2305


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MARC / unicode / utf-8 conversions

Naomi Dushay
Does this mean solrmarc  is the code vufind is switching to?  Is  
solrmarc more like the blacklight importer?  If so, I've been eagerly  
awaiting it and will take it out for a big test run "soon."   I'm  
going to be away for two weeks, or it would be "sooner."

- Naomi


On Jun 13, 2008, at 6:28 AM, Bess Sadler wrote:

>
> On May 21, 2008, at 2:01 PM, Wayne Graham wrote:
>
>> Not sure if this will answer you question, but here it goes.
>>
>> The Java that does the indexing has several converters for different
>> formats . These include Ansel, ISO5426 (Latin), and ISO 6937 (ASCII).
>> The Ansel converter will convert to- and from- the MARC-8 format.
>> Right
>> now the code to do the indexing doesn't do any conversion... is this
>> something you need? If so, we can do an enhancement request.
>> Wayne
>
>> James Farrugia wrote:
>>
>>> Andrew,
>>>
>>> Does VuFind offer a MARC to UTF-8 converter?
>>>
>>> Jim
>>>
>
> This unicode conversion is one of the things that is already built
> into solrmarc, so if people start moving to the solrmarc codebase for
> indexing they can skip the separate conversion step. It also does a
> good job of detecting incorrect encoding declarations, which is
> another common problem. To clarify, you would use solrmarc to index
> your records, and you would still use VuFind for everything else. You
> just need to choose the "vufind.properties" file when you index.
>
> I really do believe that the faster we start moving to solrmarc
> (which at this point is a question of writing documentation and
> getting some feedback... it's only being tested by a few people right
> now) the faster we'll get these kinds of problems sorted out.
>
> The solrmarc code base is here: http://code.google.com/p/solrmarc/
>
> Bess
>
> Elizabeth (Bess) Sadler
> Research and Development Librarian
> Digital Scholarship Services
> Box 400129
> Alderman Library
> University of Virginia
> Charlottesville, VA 22904
>
> [hidden email]
> (434) 243-2305
>
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> VuFind-General mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-general

Naomi Dushay
[hidden email]




-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MARC / unicode / utf-8 conversions

Bess Sadler
Hi, Naomi.

The idea is that Blacklight and Vufind would both use the same java  
importer, which will now be called solrmarc and will live in its own  
repository. So anyone can use the indexer to get their marc records  
into solr, and then slap whichever front end you want onto solr.

Bess

On Jun 13, 2008, at 12:49 PM, Naomi Dushay wrote:

> Does this mean solrmarc  is the code vufind is switching to?  Is
> solrmarc more like the blacklight importer?  If so, I've been eagerly
> awaiting it and will take it out for a big test run "soon."   I'm
> going to be away for two weeks, or it would be "sooner."
>
> - Naomi
>
>
> On Jun 13, 2008, at 6:28 AM, Bess Sadler wrote:
>
>>
>> On May 21, 2008, at 2:01 PM, Wayne Graham wrote:
>>
>>> Not sure if this will answer you question, but here it goes.
>>>
>>> The Java that does the indexing has several converters for different
>>> formats . These include Ansel, ISO5426 (Latin), and ISO 6937  
>>> (ASCII).
>>> The Ansel converter will convert to- and from- the MARC-8 format.
>>> Right
>>> now the code to do the indexing doesn't do any conversion... is this
>>> something you need? If so, we can do an enhancement request.
>>> Wayne
>>
>>> James Farrugia wrote:
>>>
>>>> Andrew,
>>>>
>>>> Does VuFind offer a MARC to UTF-8 converter?
>>>>
>>>> Jim
>>>>
>>
>> This unicode conversion is one of the things that is already built
>> into solrmarc, so if people start moving to the solrmarc codebase for
>> indexing they can skip the separate conversion step. It also does a
>> good job of detecting incorrect encoding declarations, which is
>> another common problem. To clarify, you would use solrmarc to index
>> your records, and you would still use VuFind for everything else. You
>> just need to choose the "vufind.properties" file when you index.
>>
>> I really do believe that the faster we start moving to solrmarc
>> (which at this point is a question of writing documentation and
>> getting some feedback... it's only being tested by a few people right
>> now) the faster we'll get these kinds of problems sorted out.
>>
>> The solrmarc code base is here: http://code.google.com/p/solrmarc/
>>
>> Bess
>>
>> Elizabeth (Bess) Sadler
>> Research and Development Librarian
>> Digital Scholarship Services
>> Box 400129
>> Alderman Library
>> University of Virginia
>> Charlottesville, VA 22904
>>
>> [hidden email]
>> (434) 243-2305
>>
>>
>> ---------------------------------------------------------------------
>> ----
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>> _______________________________________________
>> VuFind-General mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/vufind-general
>
> Naomi Dushay
> [hidden email]
>
>
>
>
> ----------------------------------------------------------------------
> ---
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> VuFind-General mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-general


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Loading...