Re: VuFind-General Digest, Vol 115, Issue 3

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: VuFind-General Digest, Vol 115, Issue 3

Sher Afzal
I am new to Vufind and needs your help in the following issues.
1. I installed Koha and DSpace in one server and both software are running smoothly. Now I want to install Vufind on the same server which is Linux Ubuntu flaver 16.4. could any of the group member guide me step by step installations and configuration of Vufind with DSpace and Koha?.
2. After installation how I can integrate Koha and DSpace with Vufind.

Regards
Sher


-------- Original Message --------
Subject: VuFind-General Digest, Vol 115, Issue 3
From: [hidden email]
To: [hidden email]
CC:


Send VuFind-General mailing list submissions to
[hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/vufind-general
or, via email, send a message with subject or body 'help' to
[hidden email]

You can reach the person managing the list at
[hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of VuFind-General digest..."


Today's Topics:

1. Re: VuFind add, delete and update records (Tod Olson)
2. Demian away this week (Demian Katz)
3. Re: vufind error (Demian Katz)
4. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)
5. vufind tutoring / set-up help for hire? (Mike McDermott)
6. Re: VuFind imported items hierarchy... (Demian Katz)
7. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)
8. Re: VuFind imported items hierarchy... (Demian Katz)
9. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)
10. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)


----------------------------------------------------------------------

Message: 1
Date: Mon, 10 Apr 2017 13:57:37 +0000
From: Tod Olson
Subject: Re: [VuFind-General] VuFind add, delete and update records
To: Gustavo Courault
Cc: vufind-general
Message-ID:
Content-Type: text/plain; charset="us-ascii"

Hello Gustavo,

What you describe is a common situation, even for single-library installations, and there are ways to deal with it. At our site it takes many hours to do a full export, and several hours to do a full import, so we rely on daily updates.

If you can export only the records that have been added or updated, you can load those with the same scripts that you already use. The new records are added, the updated records overwrite the ole records based on the Solr document ID.

For deletes, there is a script, util/deletes.php, which will take a file of records to delete from the Solr index. That can be just a file of record IDs, one per line, or MARC records, and there may be other options. You just need to use the right switches on the command line.

At our site, we arrange for a daily export from our library management system. It exports all records that have been added or updated since the previous daily export jobs, and it produced a file of record IDs that have been deleted or suppressed from public view.

That's how we deal with the need for daily updates, and I think many sites do something similar.

Best,

-Tod


Tod Olson >
Systems Librarian
Interim Director for Integrated Library Systems
University of Chicago Library

On Apr 10, 2017, at 8:09 AM, Gustavo Courault > wrote:

We can succefully import the marc records from our ILS to VuFind.
We have some large libraries and they add, delete and update items continuosly.
We found an only way to import records or items to VuFind: the shell
script provided by you.
Is there any way that add, update or delete records without import all them?

Thanks in advance
--
Gustavo Courault
http://www.courault.com.ar/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general

-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 2
Date: Mon, 10 Apr 2017 16:12:53 +0000
From: Demian Katz
Subject: [VuFind-General] Demian away this week
To: "[hidden email]"
,
"[hidden email]"

Message-ID:


Content-Type: text/plain; charset="us-ascii"

Hello, everyone -

Just wanted to let you know that I am once again attending a conference this week, so my email response times may be slower than usual beginning tomorrow. I'll be back in the office a week from today and will try to check messages as frequently as possible in the meantime.

- Demian
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 3
Date: Mon, 10 Apr 2017 16:35:40 +0000
From: Demian Katz
Subject: Re: [VuFind-General] vufind error
To: Kaustav Saha ,
"[hidden email]"

Message-ID:


Content-Type: text/plain; charset="utf-8"

Are you getting the 500 error everywhere now, or only when you specifically visit the Install/Home screen? That is, does the root URL continue to give you that 404 error while other pages time out? Are you seeing any errors in your Apache error log? What platform are you using? This problem could be a variety of different things, including a missing mod_rewrite plugin for Apache, a permission issue on disk, etc. If you can answer some of the above questions, that might help narrow down the issue.

- Demian

From: Kaustav Saha [mailto:[hidden email]]
Sent: Monday, April 10, 2017 4:04 AM
To: [hidden email]
Subject: [VuFind-General] vufind error


Dear Team,

I got the below mentioned error after installation. It came fine. But, when I try to change on http://localhost/vufind/Install/Home. Suddenly, "HTTP Error 500" arrived. Please help me.

[Inline image 4] [cid:image002.png@01D2B1F6.F5619550]


--
With regards,
Kaustav Saha




-------------- next part --------------
An HTML attachment was scrubbed...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 113295 bytes
Desc: image001.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 37430 bytes
Desc: image002.png

------------------------------

Message: 4
Date: Wed, 12 Apr 2017 14:35:12 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 5
Date: Fri, 14 Apr 2017 20:15:54 +0000
From: Mike McDermott
Subject: [VuFind-General] vufind tutoring / set-up help for hire?
To: "'[hidden email]'"

Message-ID:

Content-Type: text/plain; charset="us-ascii"

Hello -
My library is looking at setting up a 'Bento search' type service for a bunch of our Special Collections materials. After looking around I'd like to go with vufind as the application.
I've got a test server set up, and have successfully imported a collection of MARC records from our catalog, harvested from our Digital Commons via OAI-PMH and indexed a section of our website.
But, I feel like I am over my head when it comes to modifying the public interface and configuring the combined search results and related metadata. I've had quite a bit of PHP experience, but mostly smaller procedural scripts, not big object oriented projects, so going from the documentation to that mass of code is tough. I've come away from the modifications I have been able to make feeling like I'm doing it wrong.
It seems like if I could sit down with someone familiar with the system and walk through some of these activities I could pick it up pretty quickly.

Has anyone ever hired a contractor or gone with one of the companies on the list on the vufind website? (It seems like they are mostly out of the US).

Does anyone know of a vufind gunslinger-for-hire they would recommend?

Or are there other options?
Thanks for any suggestions...

Mike McDermott
Bowdoin College Library

-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 6
Date: Tue, 18 Apr 2017 15:23:46 +0000
From: Demian Katz
Subject: Re: [VuFind-General] VuFind imported items hierarchy...
To: Layale Bassil ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="us-ascii"

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 7
Date: Wed, 19 Apr 2017 09:29:51 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1490373193_oai_192_168_100_252_123456789_16.xml
Type: application/xml
Size: 4506 bytes
Desc: 1490373193_oai_192_168_100_252_123456789_16.xml

------------------------------

Message: 8
Date: Wed, 19 Apr 2017 12:08:20 +0000
From: Demian Katz
Subject: Re: [VuFind-General] VuFind imported items hierarchy...
To: Layale Bassil ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="us-ascii"

Hello,

Take a look at this pull request - we're in the process of developing some automated tests for the collection functionality, and as a first step, we have created a sample MARC file and custom mappings to create an example collection structure. The tests aren't written yet, but the sample data is visible here:

https://github.com/vufind-org/vufind/pull/934

That might help clarify some things.

The challenge, of course, is that each record needs to contain data about its context - like the IDs and names of its parents. In the case of our digital library, we write a custom exporter that includes all of this information with the metadata. My assumption is that DSpace is not going to give you this information by default - however, it must have access to the information internally, so there should be some way to expose it. It's just my lack of knowledge of DSpace that prevents me from being able to tell you specifically how to achieve that.

I hope this helps, but please let me know if I can help with further clarification!

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 19, 2017 5:30 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 9
Date: Wed, 19 Apr 2017 13:51:22 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hi Demian,

How can I make use the collections.mrc file found under the below link? I have downloaded it, should I import it to VuFind? Or what? Since you said the tests are not written yet, this means I can't make use of the sample data?

Yeah DSpace can never have these information. But let us assume I write a script that will read the xml files exported from DSpace and edit them and add to them the needed fields, will this work? Can I harvest them using the tool of VuFind?

I just don't want DSpace to limit the ability to use VuFind.

Thank you again, I just hope I can progress with this hoping to know what I can do next...

Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Wednesday, April 19, 2017 3:08 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

Hello,

Take a look at this pull request - we're in the process of developing some automated tests for the collection functionality, and as a first step, we have created a sample MARC file and custom mappings to create an example collection structure. The tests aren't written yet, but the sample data is visible here:

https://github.com/vufind-org/vufind/pull/934

That might help clarify some things.

The challenge, of course, is that each record needs to contain data about its context - like the IDs and names of its parents. In the case of our digital library, we write a custom exporter that includes all of this information with the metadata. My assumption is that DSpace is not going to give you this information by default - however, it must have access to the information internally, so there should be some way to expose it. It's just my lack of knowledge of DSpace that prevents me from being able to tell you specifically how to achieve that.

I hope this helps, but please let me know if I can help with further clarification!

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 19, 2017 5:30 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 10
Date: Wed, 19 Apr 2017 14:02:01 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hello again,

If it is possible to share with me any sample data (dummy) that contains a collection and couple of records that I can import into VuFind and see how it will look and how it works and how I can search and filter by collection, etc...

Having this, I will imitate and try to make the data out of DSPace look the same... can I bother you with this?

Many thanks!
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Wednesday, April 19, 2017 3:08 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

Hello,

Take a look at this pull request - we're in the process of developing some automated tests for the collection functionality, and as a first step, we have created a sample MARC file and custom mappings to create an example collection structure. The tests aren't written yet, but the sample data is visible here:

https://github.com/vufind-org/vufind/pull/934

That might help clarify some things.

The challenge, of course, is that each record needs to contain data about its context - like the IDs and names of its parents. In the case of our digital library, we write a custom exporter that includes all of this information with the metadata. My assumption is that DSpace is not going to give you this information by default - however, it must have access to the information internally, so there should be some way to expose it. It's just my lack of knowledge of DSpace that prevents me from being able to tell you specifically how to achieve that.

I hope this helps, but please let me know if I can help with further clarification!

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 19, 2017 5:30 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [mailto:[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil >; '[hidden email]' >
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [mailto:[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

------------------------------

_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general


End of VuFind-General Digest, Vol 115, Issue 3
**********************************************

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general
Reply | Threaded
Open this post in threaded view
|

Re: VuFind-General Digest, Vol 115, Issue 3

Demian Katz

You should find step-by-step installation instructions for VuFind on Ubuntu 16.04 here:

 

https://vufind.org/wiki/installation:ubuntu

 

Once VuFind is installed, you’ll find some notes on setting up Koha here:

 

https://vufind.org/wiki/indexing:koha

 

There are also some notes on DSpace here, but this page was written for VuFind 1.x and contains some steps that are no longer necessary in later versions because the import files are now included with VuFind by default. It may still be a useful reference in some regards, though:

 

https://vufind.org/wiki/indexing:dspace

 

If you have questions or run into problems, please let us know!

 

- Demian

 

From: Sher Afzal [mailto:[hidden email]]
Sent: Wednesday, April 19, 2017 10:56 AM
To: [hidden email]; [hidden email]
Subject: Re: [VuFind-General] VuFind-General Digest, Vol 115, Issue 3

 

I am new to Vufind and needs your help in the following issues.
1. I installed Koha and DSpace in one server and both software are running smoothly. Now I want to install Vufind on the same server which is Linux Ubuntu flaver 16.4. could any of the group member guide me step by step installations and configuration of Vufind with DSpace and Koha?.
2. After installation how I can integrate Koha and DSpace with Vufind.

Regards
Sher



-------- Original Message --------
Subject: VuFind-General Digest, Vol 115, Issue 3
From: [hidden email]
To: [hidden email]
CC:


Send VuFind-General mailing list submissions to
[hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/vufind-general
or, via email, send a message with subject or body 'help' to
[hidden email]

You can reach the person managing the list at
[hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of VuFind-General digest..."


Today's Topics:

1. Re: VuFind add, delete and update records (Tod Olson)
2. Demian away this week (Demian Katz)
3. Re: vufind error (Demian Katz)
4. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)
5. vufind tutoring / set-up help for hire? (Mike McDermott)
6. Re: VuFind imported items hierarchy... (Demian Katz)
7. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)
8. Re: VuFind imported items hierarchy... (Demian Katz)
9. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)
10. [SPAM] RE: VuFind imported items hierarchy... (Layale Bassil)


----------------------------------------------------------------------

Message: 1
Date: Mon, 10 Apr 2017 13:57:37 +0000
From: Tod Olson
Subject: Re: [VuFind-General] VuFind add, delete and update records
To: Gustavo Courault
Cc: vufind-general
Message-ID:
Content-Type: text/plain; charset="us-ascii"

Hello Gustavo,

What you describe is a common situation, even for single-library installations, and there are ways to deal with it. At our site it takes many hours to do a full export, and several hours to do a full import, so we rely on daily updates.

If you can export only the records that have been added or updated, you can load those with the same scripts that you already use. The new records are added, the updated records overwrite the ole records based on the Solr document ID.

For deletes, there is a script, util/deletes.php, which will take a file of records to delete from the Solr index. That can be just a file of record IDs, one per line, or MARC records, and there may be other options. You just need to use the right switches on the command line.

At our site, we arrange for a daily export from our library management system. It exports all records that have been added or updated since the previous daily export jobs, and it produced a file of record IDs that have been deleted or suppressed from public view.

That's how we deal with the need for daily updates, and I think many sites do something similar.

Best,

-Tod


Tod Olson <[hidden email]>
Systems Librarian
Interim Director for Integrated Library Systems
University of Chicago Library

On Apr 10, 2017, at 8:09 AM, Gustavo Courault <[hidden email]> wrote:

We can succefully import the marc records from our ILS to VuFind.
We have some large libraries and they add, delete and update items continuosly.
We found an only way to import records or items to VuFind: the shell
script provided by you.
Is there any way that add, update or delete records without import all them?

Thanks in advance
--
Gustavo Courault
http://www.courault.com.ar/

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general

-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 2
Date: Mon, 10 Apr 2017 16:12:53 +0000
From: Demian Katz
Subject: [VuFind-General] Demian away this week
To: "[hidden email]"
,
"[hidden email]"

Message-ID:


Content-Type: text/plain; charset="us-ascii"

Hello, everyone -

Just wanted to let you know that I am once again attending a conference this week, so my email response times may be slower than usual beginning tomorrow. I'll be back in the office a week from today and will try to check messages as frequently as possible in the meantime.

- Demian
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 3
Date: Mon, 10 Apr 2017 16:35:40 +0000
From: Demian Katz
Subject: Re: [VuFind-General] vufind error
To: Kaustav Saha ,
"[hidden email]"

Message-ID:


Content-Type: text/plain; charset="utf-8"

Are you getting the 500 error everywhere now, or only when you specifically visit the Install/Home screen? That is, does the root URL continue to give you that 404 error while other pages time out? Are you seeing any errors in your Apache error log? What platform are you using? This problem could be a variety of different things, including a missing mod_rewrite plugin for Apache, a permission issue on disk, etc. If you can answer some of the above questions, that might help narrow down the issue.

- Demian

From: Kaustav Saha [[hidden email]]
Sent: Monday, April 10, 2017 4:04 AM
To: [hidden email]
Subject: [VuFind-General] vufind error


Dear Team,

I got the below mentioned error after installation. It came fine. But, when I try to change on http://localhost/vufind/Install/Home. Suddenly, "HTTP Error 500" arrived. Please help me.

[Inline image 4] [<a href="cid:image002.png@01D2B1F6.F5619550">cid:image002.png@01D2B1F6.F5619550]


--
With regards,
Kaustav Saha




-------------- next part --------------
An HTML attachment was scrubbed...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 113295 bytes
Desc: image001.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 37430 bytes
Desc: image002.png

------------------------------

Message: 4
Date: Wed, 12 Apr 2017 14:35:12 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 5
Date: Fri, 14 Apr 2017 20:15:54 +0000
From: Mike McDermott
Subject: [VuFind-General] vufind tutoring / set-up help for hire?
To: "'[hidden email]'"

Message-ID:

Content-Type: text/plain; charset="us-ascii"

Hello -
My library is looking at setting up a 'Bento search' type service for a bunch of our Special Collections materials. After looking around I'd like to go with vufind as the application.
I've got a test server set up, and have successfully imported a collection of MARC records from our catalog, harvested from our Digital Commons via OAI-PMH and indexed a section of our website.
But, I feel like I am over my head when it comes to modifying the public interface and configuring the combined search results and related metadata. I've had quite a bit of PHP experience, but mostly smaller procedural scripts, not big object oriented projects, so going from the documentation to that mass of code is tough. I've come away from the modifications I have been able to make feeling like I'm doing it wrong.
It seems like if I could sit down with someone familiar with the system and walk through some of these activities I could pick it up pretty quickly.

Has anyone ever hired a contractor or gone with one of the companies on the list on the vufind website? (It seems like they are mostly out of the US).

Does anyone know of a vufind gunslinger-for-hire they would recommend?

Or are there other options?
Thanks for any suggestions...

Mike McDermott
Bowdoin College Library

-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 6
Date: Tue, 18 Apr 2017 15:23:46 +0000
From: Demian Katz
Subject: Re: [VuFind-General] VuFind imported items hierarchy...
To: Layale Bassil ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="us-ascii"

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 7
Date: Wed, 19 Apr 2017 09:29:51 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1490373193_oai_192_168_100_252_123456789_16.xml
Type: application/xml
Size: 4506 bytes
Desc: 1490373193_oai_192_168_100_252_123456789_16.xml

------------------------------

Message: 8
Date: Wed, 19 Apr 2017 12:08:20 +0000
From: Demian Katz
Subject: Re: [VuFind-General] VuFind imported items hierarchy...
To: Layale Bassil ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="us-ascii"

Hello,

Take a look at this pull request - we're in the process of developing some automated tests for the collection functionality, and as a first step, we have created a sample MARC file and custom mappings to create an example collection structure. The tests aren't written yet, but the sample data is visible here:

https://github.com/vufind-org/vufind/pull/934

That might help clarify some things.

The challenge, of course, is that each record needs to contain data about its context - like the IDs and names of its parents. In the case of our digital library, we write a custom exporter that includes all of this information with the metadata. My assumption is that DSpace is not going to give you this information by default - however, it must have access to the information internally, so there should be some way to expose it. It's just my lack of knowledge of DSpace that prevents me from being able to tell you specifically how to achieve that.

I hope this helps, but please let me know if I can help with further clarification!

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 19, 2017 5:30 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 9
Date: Wed, 19 Apr 2017 13:51:22 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hi Demian,

How can I make use the collections.mrc file found under the below link? I have downloaded it, should I import it to VuFind? Or what? Since you said the tests are not written yet, this means I can't make use of the sample data?

Yeah DSpace can never have these information. But let us assume I write a script that will read the xml files exported from DSpace and edit them and add to them the needed fields, will this work? Can I harvest them using the tool of VuFind?

I just don't want DSpace to limit the ability to use VuFind.

Thank you again, I just hope I can progress with this hoping to know what I can do next...

Regards.

From: Demian Katz [[hidden email]]
Sent: Wednesday, April 19, 2017 3:08 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

Hello,

Take a look at this pull request - we're in the process of developing some automated tests for the collection functionality, and as a first step, we have created a sample MARC file and custom mappings to create an example collection structure. The tests aren't written yet, but the sample data is visible here:

https://github.com/vufind-org/vufind/pull/934

That might help clarify some things.

The challenge, of course, is that each record needs to contain data about its context - like the IDs and names of its parents. In the case of our digital library, we write a custom exporter that includes all of this information with the metadata. My assumption is that DSpace is not going to give you this information by default - however, it must have access to the information internally, so there should be some way to expose it. It's just my lack of knowledge of DSpace that prevents me from being able to tell you specifically how to achieve that.

I hope this helps, but please let me know if I can help with further clarification!

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 19, 2017 5:30 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 10
Date: Wed, 19 Apr 2017 14:02:01 +0000
From: Layale Bassil
Subject: [VuFind-General] [SPAM] RE: VuFind imported items
hierarchy...
To: Demian Katz ,
"'[hidden email]'"

Message-ID:


Content-Type: text/plain; charset="iso-8859-1"

Hello again,

If it is possible to share with me any sample data (dummy) that contains a collection and couple of records that I can import into VuFind and see how it will look and how it works and how I can search and filter by collection, etc...

Having this, I will imitate and try to make the data out of DSPace look the same... can I bother you with this?

Many thanks!
Regards.

From: Demian Katz [[hidden email]]
Sent: Wednesday, April 19, 2017 3:08 PM
To: Layale Bassil ; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...

Hello,

Take a look at this pull request - we're in the process of developing some automated tests for the collection functionality, and as a first step, we have created a sample MARC file and custom mappings to create an example collection structure. The tests aren't written yet, but the sample data is visible here:

https://github.com/vufind-org/vufind/pull/934

That might help clarify some things.

The challenge, of course, is that each record needs to contain data about its context - like the IDs and names of its parents. In the case of our digital library, we write a custom exporter that includes all of this information with the metadata. My assumption is that DSpace is not going to give you this information by default - however, it must have access to the information internally, so there should be some way to expose it. It's just my lack of knowledge of DSpace that prevents me from being able to tell you specifically how to achieve that.

I hope this helps, but please let me know if I can help with further clarification!

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 19, 2017 5:30 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you as always for the continuous support. I am attaching a sample output of the oai export from DSpace. This xml file is being imported into VuFind. I will contact DSpace to see if I have other options.

But I was reading about the hierarchies and collections under https://vufind.org/wiki/indexing:hierarchies_and_collections and feel confused. How can I fill all the fields needed for the Solr Setup? When do I fill them? (i.e. hierarchytype, hierarchy_browse, hierarchy_top_id, etc...) for example how can I know what the Solr ID of the record is?

There is an example when using the top level collection identifier:

hierarchy_top_id = 1234
hierarchy_top_title = My Collection
hierarchy_browse = My Collection{{{_ID_}}}1234

where do I fill these? Sorry but I am somehow confused about it... I check the https://digital.library.villanova.edu/Collection/ you provided and this is exactly what I want, any idea how did they populate the data? Is it in marc format?

If you can send me a sample (couple of records) marc file which I can import into VuFind having a hierarchy of collections and items maybe the idea gets clearer. Or having the attached file what should I edit to create a hierarchy like structure?

I am just afraid that I will be stuck unable to preserve the hierarchy when exporting from DSpace.

Thank you again for the support, highly appreciated.
Best Regards,
Layale.

From: Demian Katz [[hidden email]]
Sent: Tuesday, April 18, 2017 6:24 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

Sorry for the late reply - I was at a conference last week and this got buried in my inbox.

Unfortunately, I don't know enough about DSpace to answer this question especially well. I suspect it might be worth talking to the DSpace community to proceed with this - from VuFind's perspective, there are a few things that might help:


a.) The ability to customize the OAI Dublin Core output to include additional information on the DSpace side

b.) The ability to use richer metadata than Dublin Core in the OAI-PMH server

c.) The ability to export data in some other fashion

I have never worked with DSpace, so I simply don't know what is possible. I can only tell you what VuFind needs as input to create a useful structure. Since DSpace is open source, I'm sure there has to be some way to produce that input - but since I am not familiar with the project, I'm not sure what the path of least resistance might be.

If you don't have any DSpace contacts, let me know and perhaps I can find someone; I'm pretty sure I spoke to a DSpace developer at a recent conference, and if I can track down his name, he might be able to point you in the right direction... but you might do just as well to go directly to the DSpace community (see http://www.dspace.org/community for some starting points) and see if they can help you with the idea of exporting DSpace hierarchy as part of the OAI-PMH output.

Please let me know how else I can be of assistance - I'm definitely happy to join a conversation with a DSpace developer and see how we can meet in the middle... I just think you're going to need a little more guidance from the DSpace side before I'll be in a position to be particularly helpful.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Wednesday, April 12, 2017 10:35 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you always for your continuous support. My problem is that I need to get my data from DSpace, so I am using the OAI-Harvester which has a flat representation of data; it is only showing a one level field not even the correct Dublin core field (i.e: dc.contributor.author is harvested as dc.contributor ==> it is only taking the first part of the dc field). How can I preserve hierarchy in this case?

How can I get my data from DSpace and be able to preserver the collection and items hierarchy?

Any help or suggestion is highly appreciated.

From: Demian Katz [[hidden email]]
Sent: Tuesday, March 21, 2017 2:43 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

The Solr fields related to hierarchy are part of the schema by default. You simply need to populate them for any records that form part of a hierarchy. Depending on how you fill them in (as described on the wiki page), they can either represent a "collection" - a node in the hierarchy - or an item that is part of a collection. Some additional configuration settings (also described in the wiki) need to be turned on to make VuFind recognize and use these fields. If you would find it helpful to look at an example of how the hierarchy fields are used in practice, you might find the MARC records and mappings in this pull request useful:

https://github.com/vufind-org/vufind/pull/934

This is all totally fake data, but it's work in progress on writing some automated tests of the collection functionality so that we can ensure that they continue to work as the code is changed over time.

The VuFind online demo does not currently use a hierarchy, but you can find an example of it in Villanova's digital library:

https://digital.library.villanova.edu/

Here is the top level showing all of the collections:

https://digital.library.villanova.edu/Collection/

As you navigate inward, you will see that some items go directly to digital assets, while others go to collections of additional items. Note the "view context" link, which pops up a visual tree of the hierarchy. This VuFind instance has been customized fairly heavily to support our digital library, but all of the hierarchical behavior on display is standard "out of the box" VuFind.

Please let me know if you still have questions about any of this - I realize that it is a bit confusing, so I'm happy to help clarify things as needed.

Regarding comments, if you simply want to turn them off, see the [Social] section of config.ini - this allows you to disable several features that currently rely on user logins.

https://github.com/vufind-org/vufind/blob/master/config/vufind/config.ini#L1535

- Demian

From: Layale Bassil [[hidden email]]
Sent: Tuesday, March 21, 2017 6:36 AM
To: Demian Katz; '[hidden email]'
Subject: RE: VuFind imported items hierarchy...
Importance: High

Hello Demian,

Thank you so much for your support. I will try to apply the hierarchy as suggested in the link you provided and get back to you for further questions. But in order to be able to start, if I am exporting my items from DSpace and then harvesting them into VuFind, where can I find the Solr fields relating to hierarchy like (hierarchytype, hierarchy_browse, hierarchy_top_id, etc...)? how and when do I populate these fields?

Is there any search or view in the current VuFind online demo version where I can see a tree like hierarchy of items? Or in any other working VuFind installation?

Regarding commenting, it would be great if someone develops it. I will vote for the below two Jira tasks ask you suggested. However, can I stop this feature for now since the system won't allow users to register and then be able to login.

Thank you again,
Regards.

From: Demian Katz [[hidden email]]
Sent: Monday, March 20, 2017 9:44 PM
To: Layale Bassil <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: VuFind imported items hierarchy...

Hello,

The example DSpace mappings do not account for the DSpace hierarchy - I do not believe that any of the necessary contextual information is included with the default DSpace Dublin Core output. I'm not sure if it is possible to add it, or to export different formats, since I am not personally familiar with DSpace. However, if there is a way to extract the necessary information from DSpace, VuFind is able to index it, as described at https://vufind.org/wiki/indexing:hierarchies_and_collections. Let me know if I can be of more assistance with this.

Regarding comments, right now, the comment system is linked to user accounts, and there is no moderation system. It would certainly be possible to add both anonymous comments and comment moderation, but both of these things would be new features that would require some adjustments to the database schema as well as some new coding. We actually have JIRA tickets that request these features:

https://vufind.org/jira/browse/VUFIND-690
https://vufind.org/jira/browse/VUFIND-704

However, as far as I know, no one has worked on implementing either. If you have time to invest in these developments, I would be happy to provide support and advice - I just lack the time to fully implement these on my own right now. If not, feel free to vote on the tickets, and that will help encourage others to work on these things in future.

- Demian

From: Layale Bassil [[hidden email]]
Sent: Monday, March 20, 2017 10:39 AM
To: '[hidden email]'
Subject: [VuFind-General] VuFind imported items hierarchy...
Importance: High

Hello,

I would like to know if VuFind preserves the hierarchy if the imported items from DSpace. Let us assume I have a community A which has a collection B and 2 items inside collection B. If I import these items into VuFind, will it preserve the hierarchy of the items (Community A --> Collection B --> Item 1 and Item 2)?

Another thing regarding the commenting section, it requires login. If I want to keep it public but have a moderator approves the added comments, is this doable? Can anyone add the comment that he likes without being moderated?

Thank you in advance,
Regards.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

------------------------------

_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general


End of VuFind-General Digest, Vol 115, Issue 3
**********************************************


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
VuFind-General mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-general