On Sun, 2010-11-21 at 23:34 +0000, Bruce Rodger wrote:
> Does anyone have any useful tools - or even just useful thoughts!
> - on monitoring an IdP using tools such as Nagios, Zabbix, BigBrother
> etc.
We've been experimenting with scripts that test the IDP and how it
interacts with various SPs, by scripting a browser to mimic simple user
actions such as logging in, searching, etc. After we've made changes to
the IDP we can re-run the tests to hopefully make sure everything still
works.
We're using two Ruby libraries called Cucumber and Watir to write tests
that look like this:
----------------------
@shibb @jrul @wip
Feature: Users should be able to log in to Zetoc and search for articles
Students and staff need to be able to access Zetoc to find articles
Scenario: Logging in and searching
Given that I have a valid username and password
And I am at the Zetoc homepage
When I select the search option from the Zetoc sidebar
And I then choose The University from the Zetoc WAYF
And login at our CAS login page with my credentials
Then I should be redirected to Zetoc and authenticated
And I should be able to search for journals
-----------------------
Cucumber will parse a list of files like that, using Watir to control a
browser and follow the expected behaviour. Any failing tests are
reported. The @ tags allow groups of specs to be selected or excluded
when they are run.
It's possible to run these tests automatically on a Continuous
Integration server or using a Nagios plugin. There are tools to run the
tests in parallel and headless without a real web browser. We've haven't
tried that yet but the CI option looks interesting. Cucumber will work
with JRuby so it can be integrated with Ant and Maven.
The main problems we've hit are that the many redirects involved in
Shibboleth authentication can sometimes confuse Watir so pauses need to
be added, and that it's tricky to get all the tests to work with all
browsers on all platforms all the time. Choosing one browser should fix
this, but we might also switch to using Capybara rather than Watir.
http://cukes.info/
http://watir.com/
https://github.com/jnicklas/capybara
http://auxesis.github.com/cucumber-nagios/
Pete
--
Pete Birkinshaw
Senior Directory and Registration Administrator
IT Services Division | +44 (0)161 306 3118
The University of Manchester | PGP: 0xB7B0B433
|