Page loading times

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Page loading times

Luke O'Sullivan
Hi Folks,

Some of our record pages at cronfa.swan.ac.uk are taking 10 seconds
plus to load. This is rather frustrating for users and we suspect it is
also hampering our attempts to get content into Google Scholar.

The cause is probably code that we have introduced - does anyone have
any experience analysing potential bottlenecks? If so, can you
recommend any tools? I've had a look at Webgrind but I haven't been
able to identify a likely culprit.

Any help would be gratefully received,
Kind regards,

Luke O'Sullivan

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Demian Katz
Are the slow pages consistently slow, or does the whole site get slow for periods of time?

Have you monitored your garbage collection logs, etc.?

Do you have any sort of server-level monitoring to identify odd performance trends overall?

(For example, here at Villanova we have a Nagios server that checks some of our key web resources every ten minutes and throws an alert if it takes more than 5 seconds for a page to load. When we get one of those alerts, we can look at other Nagios graphs to see if there was heavy traffic, high CPU load, etc. -- this has been very valuable for tracking down some problems related to, for example, misconfigurations causing sub-optimal search engine crawling patterns that imposed unnecessary extra load).

- Demian

-----Original Message-----
From: Luke O'Sullivan [mailto:[hidden email]]
Sent: Monday, January 23, 2017 10:01 AM
To: vufind-tech
Subject: [VuFind-Tech] Page loading times

Hi Folks,

Some of our record pages at cronfa.swan.ac.uk are taking 10 seconds plus to load. This is rather frustrating for users and we suspect it is also hampering our attempts to get content into Google Scholar.

The cause is probably code that we have introduced - does anyone have any experience analysing potential bottlenecks? If so, can you recommend any tools? I've had a look at Webgrind but I haven't been able to identify a likely culprit.

Any help would be gratefully received,
Kind regards,

Luke O'Sullivan

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=01%7C01%7Cdemian.katz%40villanova.edu%7C49b0a9b9312647e0053108d443a41a88%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=hWbDPok1aAxl401Ls0GPZ2wcKm25tdNodqmXgUyd4Yg%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=01%7C01%7Cdemian.katz%40villanova.edu%7C49b0a9b9312647e0053108d443a41a88%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=G5k3%2FXxQXTSEaqjdeqkr8qpUVAicxO1hNq6Jc%2B5aY8E%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Luke O'Sullivan
Hi Demian,

Thanks for your response.

The page loads are consistently slow, both on my development machine
and on the main server.

We use Zabbix to monitor site availability and that gives us a similar
set of results to Nagios. There doesn't appear to be anything remiss. 

Can you recommend any tools for analysing the Solr GC logs? I've
attached one to this email just in case you can spot anything out of
the ordinary at a quick glance. I would be surprised if that was an
issue though as the service is not heavily used (it just happens to
have some users who have quite a lot of clout!).

Thanks,

Luke

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech

solr_gc.log (297K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Demian Katz
I've always used gcviewer for analyzing GC logs, though since it is now unmaintained for close to a decade, I'm not sure how useful it remains: http://www.tagtraum.com/gcviewer.html

Another question is whether you've tried XHProf. Since it is fully web-based, it's a bit less cumbersome to deal with than XDebug, and VuFind supports it through a setting in httpd-vufind.conf. See this:

https://github.com/vufind-org/vufind/blob/master/config/vufind/httpd-vufind.conf#L66

You can turn that on on your development server, access one of the slow pages, and then follow the resulting profiling link to drill down to the bottlenecks.

Let me know if you have questions/problems!

- Demian

-----Original Message-----
From: Luke O'Sullivan [mailto:[hidden email]]
Sent: Monday, January 23, 2017 1:45 PM
To: Demian Katz; vufind-tech
Subject: Re: [VuFind-Tech] Page loading times

Hi Demian,

Thanks for your response.

The page loads are consistently slow, both on my development machine and on the main server.

We use Zabbix to monitor site availability and that gives us a similar set of results to Nagios. There doesn't appear to be anything remiss. 

Can you recommend any tools for analysing the Solr GC logs? I've attached one to this email just in case you can spot anything out of the ordinary at a quick glance. I would be surprised if that was an issue though as the service is not heavily used (it just happens to have some users who have quite a lot of clout!).

Thanks,

Luke
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Luke O'Sullivan
Thanks Damian :) Much appreciated!

Sent from my HTC

----- Reply message -----
From: "Demian Katz" <[hidden email]>
To: "Osullivan L." <[hidden email]>, "vufind-tech" <[hidden email]>
Subject: [VuFind-Tech] Page loading times
Date: Mon, Jan 23, 2017 19:16

I've always used gcviewer for analyzing GC logs, though since it is now unmaintained for close to a decade, I'm not sure how useful it remains: http://www.tagtraum.com/gcviewer.html

Another question is whether you've tried XHProf. Since it is fully web-based, it's a bit less cumbersome to deal with than XDebug, and VuFind supports it through a setting in httpd-vufind.conf. See this:

https://github.com/vufind-org/vufind/blob/master/config/vufind/httpd-vufind.conf#L66

You can turn that on on your development server, access one of the slow pages, and then follow the resulting profiling link to drill down to the bottlenecks.

Let me know if you have questions/problems!

- Demian

-----Original Message-----
From: Luke O'Sullivan [[hidden email]]
Sent: Monday, January 23, 2017 1:45 PM
To: Demian Katz; vufind-tech
Subject: Re: [VuFind-Tech] Page loading times

Hi Demian,

Thanks for your response.

The page loads are consistently slow, both on my development machine and on the main server.

We use Zabbix to monitor site availability and that gives us a similar set of results to Nagios. There doesn't appear to be anything remiss. 

Can you recommend any tools for analysing the Solr GC logs? I've attached one to this email just in case you can spot anything out of the ordinary at a quick glance. I would be surprised if that was an issue though as the service is not heavily used (it just happens to have some users who have quite a lot of clout!).

Thanks,

Luke

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Ere Maijala
In reply to this post by Demian Katz
Fortunately someone has continued development of GC Viewer here:
<https://github.com/chewiebug/GCViewer>. That version supports G1GC etc.

--Ere

23.1.2017, 21.16, Demian Katz kirjoitti:

> I've always used gcviewer for analyzing GC logs, though since it is now unmaintained for close to a decade, I'm not sure how useful it remains: http://www.tagtraum.com/gcviewer.html
>
> Another question is whether you've tried XHProf. Since it is fully web-based, it's a bit less cumbersome to deal with than XDebug, and VuFind supports it through a setting in httpd-vufind.conf. See this:
>
> https://github.com/vufind-org/vufind/blob/master/config/vufind/httpd-vufind.conf#L66
>
> You can turn that on on your development server, access one of the slow pages, and then follow the resulting profiling link to drill down to the bottlenecks.
>
> Let me know if you have questions/problems!
>
> - Demian
>
> -----Original Message-----
> From: Luke O'Sullivan [mailto:[hidden email]]
> Sent: Monday, January 23, 2017 1:45 PM
> To: Demian Katz; vufind-tech
> Subject: Re: [VuFind-Tech] Page loading times
>
> Hi Demian,
>
> Thanks for your response.
>
> The page loads are consistently slow, both on my development machine and on the main server.
>
> We use Zabbix to monitor site availability and that gives us a similar set of results to Nagios. There doesn't appear to be anything remiss.
>
> Can you recommend any tools for analysing the Solr GC logs? I've attached one to this email just in case you can spot anything out of the ordinary at a quick glance. I would be surprised if that was an issue though as the service is not heavily used (it just happens to have some users who have quite a lot of clout!).
>
> Thanks,
>
> Luke
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Ere Maijala
In reply to this post by Luke O'Sullivan
Luke,

it's not usual for GC to cause consistent slowliness. Typical GC issues
cause intermittent long pauses. A quick glance through the log doesn't
show anything especially scary either.

Just to make sure you're looking at the right way: If pages that don't
search Solr load slowly, it's not a Solr issue. Make sure that e.g. your
database connection to MySQL or Postgresql works quickly.

--Ere

23.1.2017, 20.44, Luke O'Sullivan kirjoitti:

> Hi Demian,
>
> Thanks for your response.
>
> The page loads are consistently slow, both on my development machine
> and on the main server.
>
> We use Zabbix to monitor site availability and that gives us a similar
> set of results to Nagios. There doesn't appear to be anything remiss.
>
> Can you recommend any tools for analysing the Solr GC logs? I've
> attached one to this email just in case you can spot anything out of
> the ordinary at a quick glance. I would be surprised if that was an
> issue though as the service is not heavily used (it just happens to
> have some users who have quite a lot of clout!).
>
> Thanks,
>
> Luke
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/vufind-tech
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Demian Katz
In reply to this post by Ere Maijala
That's great news -- I've updated the relevant links to the tool in the wiki and my old blog post. Thanks for tracking this down!

- Demian

-----Original Message-----
From: Ere Maijala [mailto:[hidden email]]
Sent: Tuesday, January 24, 2017 2:30 AM
To: [hidden email]
Subject: Re: [VuFind-Tech] Page loading times

Fortunately someone has continued development of GC Viewer here:
<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchewiebug%2FGCViewer&data=01%7C01%7Cdemian.katz%40villanova.edu%7C8662b00c4fdc4071c60808d4442b0e82%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=%2Biw65W%2FfKfRGGiUlzfRTiU9nZblRV0cUuz5MXsY0AUA%3D&reserved=0>. That version supports G1GC etc.

--Ere

23.1.2017, 21.16, Demian Katz kirjoitti:

> I've always used gcviewer for analyzing GC logs, though since it is
> now unmaintained for close to a decade, I'm not sure how useful it
> remains:
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ta
> gtraum.com%2Fgcviewer.html&data=01%7C01%7Cdemian.katz%40villanova.edu%
> 7C8662b00c4fdc4071c60808d4442b0e82%7C765a8de5cf9444f09cafae5bf8cfa366%
> 7C1&sdata=WJrF7LJI8JDSHXh%2FPKF8RvHikvoH41OmnH0xTsO9Xaw%3D&reserved=0
>
> Another question is whether you've tried XHProf. Since it is fully web-based, it's a bit less cumbersome to deal with than XDebug, and VuFind supports it through a setting in httpd-vufind.conf. See this:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu
> b.com%2Fvufind-org%2Fvufind%2Fblob%2Fmaster%2Fconfig%2Fvufind%2Fhttpd-
> vufind.conf%23L66&data=01%7C01%7Cdemian.katz%40villanova.edu%7C8662b00
> c4fdc4071c60808d4442b0e82%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata
> =6jTQlOTZxZbUDB7NocwbOVWDE%2Fg%2F0gWj86O4PnM8GnI%3D&reserved=0
>
> You can turn that on on your development server, access one of the slow pages, and then follow the resulting profiling link to drill down to the bottlenecks.
>
> Let me know if you have questions/problems!
>
> - Demian
>
> -----Original Message-----
> From: Luke O'Sullivan [mailto:[hidden email]]
> Sent: Monday, January 23, 2017 1:45 PM
> To: Demian Katz; vufind-tech
> Subject: Re: [VuFind-Tech] Page loading times
>
> Hi Demian,
>
> Thanks for your response.
>
> The page loads are consistently slow, both on my development machine and on the main server.
>
> We use Zabbix to monitor site availability and that gives us a similar set of results to Nagios. There doesn't appear to be anything remiss.
>
> Can you recommend any tools for analysing the Solr GC logs? I've attached one to this email just in case you can spot anything out of the ordinary at a quick glance. I would be surprised if that was an issue though as the service is not heavily used (it just happens to have some users who have quite a lot of clout!).
>
> Thanks,
>
> Luke
> ----------------------------------------------------------------------
> -------- Check out the vibrant tech community on one of the world's
> most engaging tech sites, SlashDot.org!
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.li
> nk%2Fslashdot&data=01%7C01%7Cdemian.katz%40villanova.edu%7C8662b00c4fd
> c4071c60808d4442b0e82%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=FmN
> nM%2Fajlx4wVXNTnnOcUm%2BVla%2Fzm2MhBCW98hzDr5I%3D&reserved=0
> _______________________________________________
> Vufind-tech mailing list
> [hidden email]
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists
> .sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=01%7C01%7Cdemia
> n.katz%40villanova.edu%7C8662b00c4fdc4071c60808d4442b0e82%7C765a8de5cf
> 9444f09cafae5bf8cfa366%7C1&sdata=xNJ%2FvmdQ79GBg%2BaQJGKq2e7HmiL4EE9Re
> 9qb0AOI%2F7E%3D&reserved=0
>

--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=01%7C01%7Cdemian.katz%40villanova.edu%7C8662b00c4fdc4071c60808d4442b0e82%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=FmNnM%2Fajlx4wVXNTnnOcUm%2BVla%2Fzm2MhBCW98hzDr5I%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=01%7C01%7Cdemian.katz%40villanova.edu%7C8662b00c4fdc4071c60808d4442b0e82%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=xNJ%2FvmdQ79GBg%2BaQJGKq2e7HmiL4EE9Re9qb0AOI%2F7E%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Luke O'Sullivan
In reply to this post by Ere Maijala
Hi Demian,

Thanks to your response, I've got XHProf running and confirmed that the
problem is with the SQL stuff I've introduced to track statistics.

Now I guess I need to find out if it's the methods themselves or the
interaction with MySQL.

Thanks,

Luke

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|

Re: Page loading times

Demian Katz
Excellent, glad to hear you're making progress! If it is indeed on the MySQL side, there are some useful troubleshooting tips on that end -- like EXPLAIN -- which might help reveal bottlenecks. I've found that adding indexes and foreign keys can sometimes make a huge difference. If you're lucky, maybe it's something as simple as that.

- Demian

-----Original Message-----
From: Luke O'Sullivan [mailto:[hidden email]]
Sent: Tuesday, January 24, 2017 1:09 PM
To: Ere Maijala; [hidden email]
Subject: Re: [VuFind-Tech] Page loading times

Hi Demian,

Thanks to your response, I've got XHProf running and confirmed that the problem is with the SQL stuff I've introduced to track statistics.

Now I guess I need to find out if it's the methods themselves or the interaction with MySQL.

Thanks,

Luke

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=01%7C01%7Cdemian.katz%40villanova.edu%7C530e9cc2bd8d4ef67c7c08d44484b6e1%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=XrBTejn93usiVfo3vsFQr8EdexWGrkWdCYe0ug8ydz4%3D&reserved=0
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fvufind-tech&data=01%7C01%7Cdemian.katz%40villanova.edu%7C530e9cc2bd8d4ef67c7c08d44484b6e1%7C765a8de5cf9444f09cafae5bf8cfa366%7C1&sdata=8DDld8hpsAR2jHv5FYPx1nTtHI7wa3ppl6qR1aNs%2B7g%3D&reserved=0

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech