Slightly off topic but still kinda relevant...

We have our resilience in other ways. We have one IdP yes, but it's a
virtual one. And IT built a second server room on the same VM
infrastructure, so if the main one got blown up there is a full mirror
in a separate building with a road in the way. Plus there are regular

Our first Shib2 deployment (2.1.5) was a bit unstable but since we
moved to 2.3.2 it's been good. 2.3.8 is a happy IdP.


David Perry
eContent Developer, eLearning Team (L34 - Library)
Hull College
Wilberforce Drive, Queen's Gardens, Hull
Extension 2230 / Direct Dial: 01482 381930

* * * Think about the environment - Do you really need to print this
email?>>> Mark Cairney <[log in to unmask]> 04/10/2012 11:00 >>>

I've got my 2 Shib IdPs running behind a Citrix Netscaler with
Terracotta. I initially had some teething problems with getting this
working with Terracotta but it now seems to be running fairly happily
(touch wood).

There was also a bit of a black art involved in setting up sufficient
health checks on the Loadbalancer side of things to ensure that broken
IdPs are removed from service and that working IdP servers aren't
removed erroneously!

Terracotta can be a bit of a headache but when it's working it works
pretty well. Most (all?) of our Terracotta outages have occurred when
the communication between the servers is interrupted for a period of
time and they both go into Active mode and stay there when communication
resumes, causing a split-brain situation. We've worked around this to a
certain extent by some tweaking of the various timeout values in
Terracotta but as it's a quorum issue the best solution would be to
simply have an odd number of terracotta servers.

On 4 Oct 2012, at 10:44, Andy Swiffin wrote:

> Hi
> Unfortunately those are the kind of solutions that are really
"deprecated" in todays environment, at least here at Dundee.   The
thrust is to keep things as pure and vanilla as possible.   I'd had a
browse back in the Shib users list which is where I'd gleaned a lot of
anti Terracotta bias and also seen
 (although I don't seem to be able to unzip the actual login handler
from contributions).
> Shib 2 is SO reliable that the chance of an unscheduled failover is
very rare.   I'm not therefore worried about people having to
reauthenticate on failover.   I just want a hardware box (two actually!)
to do the hot standy redirection.
> Cheers
> Andy
> From: Discussion list for Shibboleth developments
[[log in to unmask]] on behalf of Rod Widdowson
[[log in to unmask]] 
> Sent: 04 October 2012 10:19
> To: [log in to unmask] 
> Subject: Re: Resilient Shib IdPs
> Andy,
> There’s a lot of experience over in the Shib-users list, and you
might to ask there and also consult
> Two to look out for are Scott Cantor’s “Ohio State extensions,
primarily a custom login module for SSO with stateless clustering, and
workflow-like login handler with Velocity-based UI and post-login
notification hooks” and Paul Hethmon’s “A replacement storage
service for Shibboleth IdP v2 that uses Infinispan to provide cluster
> Rod
> From: Discussion list for Shibboleth developments
[mailto:[log in to unmask]] On Behalf Of Andy Swiffin
> Sent: 04 October 2012 05:04
> To: [log in to unmask] 
> Subject: Resilient Shib IdPs
> Hi
> I'm currently putting together plans to update our IdP infrastructure
and want to add in automatic failover.   We currently have a manual
failover to a secondary IdP, should it ever be needed (which since the
move to Shib 2 many many moons ago it hasn't!).   However I would like
to push Shib authentication for some high profile services and without
having a demonstrable auto failover mechanism this won't be well
> Because Shibboleth is stateful, if you are going to loadbalance
cluster it you need a mechanism for sharing state information, the
shibboleth documentation says:  "By default the Shibboleth team
recommends the use of Terracotta as the mechanism for doing this"  which
is a shame because I have it on high authority that "I think you'd be
insane to consider it."....    I know a lot of people have found
Terracotta to have, itself, caused shib outages.
> Without state sharing you need to go for a hot standby rather than
loadbalanced approach, but unfortunately our existing Cisco content
switch (which is well overdue for replacement) cannot do this.    
> So,  I'd be interested to hear from anyone who is doing hot standby
with their Shibboleth IdP  (i.e. if IdP1 is responding always use it, if
it fails the test switch to IdP2)  and what type of hardware
loadbalancer you're using at the front to do this.
> Cheers
> Andy
> The University of Dundee is a registered Scottish Charity, No:
> The University of Dundee is a registered Scottish Charity, No:


Mark R Cairney
ITI UNIX Section
Information Services

Tel: 0131 650 6565
Email: [log in to unmask] 


The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

This message is sent in confidence for the addressee
only. It may  contain confidential or sensitive
information.  The contents are not to be disclosed
to anyone other than the addressee.  Unauthorised
recipients are requested to preserve this
confidentiality and to advise us of any errors in
transmission.  Any views expressed in this message
are solely the views of the individual and do not
represent the views of the College.  Nothing in this
message should be construed as creating a contract.

Hull College owns the email infrastructure, including the contents.

Hull College is committed to sustainability, please reflect before printing this email.