Article Annotation Notification Feed (aka feeds are slow)

[russ]

howdy!

the feeds are really slow, so slow that building a feed often sends the stack into an unrecoverable hang.

i'm not sure why the feeds should be so slow.

most of the feeds we run have only category and/or date parameters, so it should be possible to server this list from the browse cache, using the browse services.

that would remove the necessity of running each article through XACML, which may be one of the bottlenecks.

i'm also pretty sure that all the info necessary to pull for each article exists in the browse cache for the non-extended feed.

the only other parameter (not used at all in the production system) is author. in this case, we'd need to go to mulgara (or lucene!!!) instead of building from browse.

[amit]

Wrong version number.

[rcave]

Confirm with 0.9

[russ]

like fetchArticle, things seem to have gotten worse in 0.9

the uncached main feed went from avg. 33293ms to avg. 73083 ms.

the uncached subject area feeds got slightly faster (from avg. 3358 ms to avg. 1793 ms), however that may be because, with the passage of time since creating the mirror of production, there are fewer articles to return in the default time frame.

i also noticed that we seem to be adding keys to FeedCache? and ObjectCache? when building a feed in 0.9. in 0.8 we also added keys to article-state. i'm not sure if this means that feeds are ignoring article-state (i will test separately) or if we're missing some caching that would be helpful, or if this is just the new way of doing things.

[amit]

Based on the email exchange on the mailing list we are not sure the numbers are. Handing it back to Russ (although we might still run some tests)

[amit]

Based on latest emails from Russ.

[russ]

really? this is absolutely not fixed. it takes about 14 seconds to build the main plosone feed.

building an extended feed for plosone with a date range takes around 8 minutes.

[amit]

:) Well, this was closed based on your emails. I have no problems with reopening it if those are the new numbers you are seeing.

[russ]

which emails? yes, things are faster (about twice as fast) 0.9 vs. 0.8.

but they are still too slow to be usable.

[russ]

taking this back to test performance post r6368

[russ]

things seem to be about 50% faster in 0.9.1 - which is still 5s for the main feed, which is barely okay.

however, both 0.9 and 0.9.1, it seems that the feed query itself is not the real limiting factor.

rather, after getting the list of articles with the feed query, it's the time taken to build the article objects for display that really impacts performance.

if the article cache is empty, and we have to build articles, things take a long time. if the article cache is full, then the feed is fast enough.

the real goal of this ticket is to make the feeds fast enough, and cached enough, to move away from the static feeds and let them build dynamically. bumping this to the next milestone, so that we can test dynamic feeds with the production release of 0.9.2

[amit]

Please do not assign milestone without a discussion with Rich or me first.

[russ]

in 0.9.2, using the feedburner feed settings for plosone (maxResults = 100 for main feed, maxResults = 15 for subject feed)

totally uncached, main feed is 25s. cached, main feed is 7s

totally uncached, subject feed is 10s cached, subject feed is 2s

most of the time in the feed is spent building articles, and we publish articles a few at a time, so i think that's okay.

however, the cached time of 7s for the plosone feed is a too long. even the cached time of 2s for the subject area feed is longer than it should be.

so i don't think we're ready for prime time here. also, feed cache invalidation is still broken, but that's a separate block to going live with dynamic feeds.

[rcave]

Moving into 0.9.3

[dragisak]

(In [7611]) Use unmanaged blobs to store annotation bodies.

Unmanaged blobs are always read into byte array and are going to be cached in ObjectCache?.

Addresses #814

[dragisak]

(In [7613]) Redesign annotation feeds to include replies.

• Cache annotation feeds in feedCache
• Invalidate results of reply query separately
• Add anchors to annotation display and link to them from reply feed
• Rename Blob and UnmanagedBlob? classes
• Move feed action and service classes into their own package

Addresses #814

[josowski]

(In [7616]) This adds various limits to the lenth of a number of form fields.

References #814

[dragisak]

(In [7621]) Rename annotation feed types. References #814

[dragisak]

(In [7623]) Truncate annotation body to 512 characters and display author's first and last name instead of author's username.

Addresses #814

[rcave]

Assigning to Russ to verify that the Article annotation feed is working correctly.

[russ]

if cached feeds look zippy, and uncached feeds aren't a horror show, in performance testing then let's close this.