robbat2: (Default)

This is a copy+paste from my email to the gentoo-dev mailing list, simply because some developers and users follow the RSS feeds rather than read email. If you want the bot in your channel and you are a channel founder/lead op, please respond on the thread in the mailing list

Hi folks,

Sorry that it's taken this long to get completed, but the Jeeves
replacement, Willikins, is finally 99% done, and ready to join lots of
channels.

Getting the bot out there
-------------------------
If you would like to have the new bot in your #gentoo-* channel, would
each channel founder/leader please respond to this thread, stating the
channel name, and that they are the contact for any problems/troubles.

Bug reports
-----------
Please open a bug in the Gentoo Infrastructure product, using the
'Other' component, and assign it directly to me.

Custom bot functionality:
-------------------------
Here's all the functionality that we have assembled, beyond the standard
rbot stuff.
Bugzilla
========
!bug [ZILLA] ID
Looks up bug #ID in the per-channel default or specified bugzilla.

!bugstats [ZILLA]
Totals of bugs per the bugzilla 'status' field.

!archstats [ZILLA] [STATUS] [RESO]
Totals of bugs per architecture, optionally with some specific set of
status or resolution values, comma delimited.

status = OPEN, DONE, UNCONFIRMED,NEW,ASSIGNED,REOPENED, RESOLVED, VERIFIED, CLOSED
Reso = FIXED, INVALID, WONTFIX, LATER, REMIND, DUPLICATE, WORKSFORME,
       CANTFIX, NEEDINFO, TEST-REQUEST, UPSTREAM
zilla = gentoo xine sourcemage redhat mozilla kernel fdo abisource
        apache kde gnome
If you want another bugzilla, file a bug.

Gentoo-specific
===============
!meta [-v] [CAT/]PACKAGE
Print the metadata and optionally herd members for a given package.

!changelog [CAT/]PACKAGE
Changelog stats for a package

!devaway list
List all away developers.

!devaway DEVNAME
Display .away message for a single developer.

!herd HERD
Show herd members

!expn NAME
Show the expansion of any public Gentoo mail alias

!glsa GLSAID
Shows the title and external IDS for any given GLSA ID.

!earch [CAT/]PACKAGE
Earch output for a given package

!rdep [CAT/]PACKAGE
Reverse RDEPEND for a given package

!ddep
Reverse DEPEND for a given package

What isn't supported yet
------------------------
1. !glsa -s TEXT
This used to search for GLSAs that matched that string in their title or
external IDS.

2. New bug announcements
Jeeves used to announce brand new bugs to #gentoo-bugs as well as
targeted channels or users, depending on the product, component,
assignee, cc and a number of other factors (deeply nested if/else
trees). The old implementation had this in code entirely, and it would
be nice to avoid having to modify the code whatsoever, and instead have
some domain-specific language for doing this.

Source availability
-------------------
Gentoo specific:
http://git.overlays.gentoo.org/gitweb/?p=proj/rbot-gentoo.git
Bugzilla support:
http://git.overlays.gentoo.org/gitweb/?p=proj/rbot-bugzilla.git
(flameeyes has his own tree as well, but he's been sick lately, so it
was lagging behind my development)

Right now, if you want to run your own instance of the bot, you will
need the latest Git tree of the rBot itself, as upstream only fixed the
last remaining issue a couple of hours ago.

Thanks to
---------
solar:
Running the old Jeeves Eggdrop till now, and helping to document all of
the Eggdrop functionality we used.

flameeyes:
Bugzilla plugin development

halcy0n:
Gentoo-specific stuff

tango_, jsn-:
(rbot upstream developers) For fixing the bugs as I found them :-).
robbat2: (Default)

I meant to get back to doing more statistics on Bugzilla, but it fell by the wayside. The following is mainly for completeness, and the interest of those as to why Bugzilla has been so bog slow for Gentoo in the past.

First of all, I had some questions as to why I focused on specific actions in Bugzilla. The truth of this is, that we can break down Bugzilla's usage of the database into three specifics:

  1. Changes to bugs (INSERT, UPDATE)
  2. Loads of specific bugs and attachments (SELECT with a primary key)
  3. Searches for bugs (Complex SELECT)

Unfortunetly, the usage patterns are heavily against Bugzilla here. Searches for bugs using some string plus a variety of conditions are the most common action. Benchmarking slow queries? That's pretty much any of the complex SELECTS. "Add more indexes" I hear some people shouting. The indexes are already nearly the same size as the actual dataset they index (400mb of index for 500mb of data)! There is an index on every field that is used for searching! One of the problems is that mysql trashes it's caches on UPDATEs and INSERTs in many cases, so spends a lot of time reloading them.

Bugzilla could massively benefit from an external text indexing system like Apache's Lucene, that can handle live modifications to the index without wasting anything. Changes are fed realtime to the index, and searches for text are performed against the dedicated index (which can also be parallized easily).

More numbers

Stuart asked for some more actual numbers, so I've put them together.

Breakdown by Request Type
TypeMeanMaxMin
Total GET53809 60409 45094
—Static GET 35401 39774 28797
—Dynamic GET 18407 20635 16223
Total POST 1394 1569 1106

Graphs below the cut, hidden to avoid spamming the page )

robbat2: (Default)

As a few folks are aware, I've been working on the new Gentoo Bugzilla. While I have got it up now (bugstest.gentoo.org), I'm uncertain as to how the new system would cope under the load, having never done any management of a Bugzilla installation previously, nor handled anything about the existing Gentoo Bugzilla instance.

This of course nessicates a performance analysis of the existing Bugzilla 8-). Read on for statistics on the matter, including the top 3 most active users in the Gentoo Bugzilla.

Read more... )

May 2017

S M T W T F S
 123456
78910111213
141516171819 20
21222324252627
28293031   

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
OSZAR »