Christoph Wissing wrote:

> Dear *,
> 
> on one of our LCG-RBs we have the problem that the workload_manager
> process keeps producing segmentation faults. It might be related to the
> problems observed last week with crashing network server, but I do not
> know.

Most probably this bug:

     https://savannah.cern.ch/bugs/index.php?20973

It has a recipe to get you out of that mess.

> Running the orkload_manager in gdb I end with:
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 37665712 (LWP 1808)]
> 0x08116029 in std::_Construct<classad::ExprTree*, classad::ExprTree*> ()
> 
> The RB is on the most recent update level of gLite/LCG. Three other RBs
> at DESY with same software run fine. So it might be that a user found a
> JDL that crashes the box. I put the backtrace for experts at the very
> end of this mail.
> 
> Any help is welcome!
> 
> Best regards,
> Christoph
> 
> The backtrace reads as follows:
> (gdb) backtrace
> #0  0x08116029 in std::_Construct<classad::ExprTree*,
> #classad::ExprTree*> () 1  0x08115d80 in std::vector<classad::ExprTree*,
> #std::allocator<classad::ExprTree*> >::push_back () 2  0x081a43ec in
> #classad::ExprList::GetComponents () 3  0x00469333 in
> #edg::workload::planning::plugin::doMatch ()
>    from /opt/edg/lib/libedg_wl_classad_plugin.so
> #4  0x081aa9f3 in classad::FunctionCall::_Evaluate ()
> #5  0x081a54a6 in classad::ExprTree::Evaluate ()
> #6  0x081b61b0 in classad::Operation::_Evaluate ()
> #7  0x081a54a6 in classad::ExprTree::Evaluate ()
> #8  0x081b6140 in classad::Operation::_Evaluate ()
> #9  0x081a54a6 in classad::ExprTree::Evaluate ()
> #10 0x081b6140 in classad::Operation::_Evaluate ()
> #11 0x081a54a6 in classad::ExprTree::Evaluate ()
> #12 0x0819da1e in classad::AttributeReference::_Evaluate ()
> #13 0x081a54a6 in classad::ExprTree::Evaluate ()
> #14 0x081a14e4 in classad::ClassAd::EvaluateAttr ()
> #15 0x081a1b0d in classad::ClassAd::EvaluateAttrBool ()
> #16 0x00ec3667 in edg::workload::common::utilities::match ()
>    from /opt/edg/lib/libedg_wl_util.so.0
> #17 0x00ec3702 in edg::workload::common::utilities::left_matches_right
> #()
>    from /opt/edg/lib/libedg_wl_util.so.0
> #18 0x00d54cad in
> #edg::workload::planning::broker::matchmaking::matchmakerGlueIm---Type
> #<return> to continue, or q <return> to quit---
> pl::checkRequirement () from
> /opt/edg/lib/libedg_wl_helper_broker_prefetch.so.0
> #19 0x00d39fd8 in
> #edg::workload::planning::broker::matchmaking::MatchMaker<edg::workload
> #::planning::broker::matchmaking::matchmakerGlueImpl>::checkRequirement
>     () from /opt/edg/lib/libedg_wl_helper_broker_prefetch.so.0
> #20 0x00d3efd1 in
> #edg::workload::planning::broker::RBSimpleImpl::findSuitableCEs () from
> #/opt/edg/lib/libedg_wl_helper_broker_prefetch.so.0 21 0x00d34ec4 in
> #edg::workload::planning::broker::ResourceBroker::findSuitableCEs ()
> #from /opt/edg/lib/libedg_wl_helper_broker_prefetch.so.0 22 0x00d2a517
> #in edg::workload::planning::broker::(anonymous namespace)::f_resolve_mm
> #() from /opt/edg/lib/libedg_wl_helper_broker_prefetch.so.0 23
> #0x00d2cbd8 in edg::workload::planning::broker::Helper::resolve ()
>    from /opt/edg/lib/libedg_wl_helper_broker_prefetch.so.0
> #24 0x001d4d41 in edg::workload::planning::helper::Helper::resolve ()
>    from /opt/edg/lib/libedg_wl_helper.so.0
> #25 0x001dfef6 in
> #edg::workload::planning::helper::RequestStateMachine::next_step () from
> #/opt/edg/lib/libedg_wl_helper.so.0 26 0x001de5c0 in
> #edg::workload::planning::helper::Request::Impl::resolve ()
>    from /opt/edg/lib/libedg_wl_helper.so.0
> #27 0x001debee in edg::workload::planning::helper::Request::resolve ()
>    from /opt/edg/lib/libedg_wl_helper.so.0
> #28 0x00f978f1 in
> #edg::workload::planning::manager::RequestPlanningPolicy::Plan
>     () from /opt/edg/lib/libedg_wl_manager_server.so.0
> #29 0x00f80a61 in
> #edg::workload::planning::manager::WMReal<edg::workload::planni---Type
> #<return> to continue, or q <return> to quit---
> ng::manager::RequestPlanningPolicy,
> edg::workload::planning::manager::JCDeliveryPolicy,
> edg::workload::planning::manager::JCCancellingPolicy>::submit ()
>    from /opt/edg/lib/libedg_wl_manager_server.so.0
> #30 0x00c9592d in
> #edg::workload::planning::manager::WorkloadManager::submit ()
>    from /opt/edg/lib/libedg_wl_manager_common.so.0
> #31 0x00f7aae1 in
> #edg::workload::planning::manager::RequestHandler::Impl::run
>     () from /opt/edg/lib/libedg_wl_manager_server.so.0
> #32 0x00f7c938 in edg::workload::planning::manager::RequestHandler::run
> #()
>    from /opt/edg/lib/libedg_wl_manager_server.so.0
> #33 0x0810ca07 in
> #edg::workload::common::task::ReaderFunctor<std::pair<boost::function0<
> #void, boost::empty_function_policy, boost::empty_function_mixin,
> #std::allocator<boost::function_base> >,
> #boost::shared_ptr<classad::ClassAd> > >::operator() () 34 0x0810c687 in
> #boost::detail::function::void_function_obj_invoker0<edg::workload::com
> #mon::task::ReaderFunctor<std::pair<boost::function0<void,
> #boost::empty_function_policy, boost::empty_function_mixin,
> #std::allocator<boost::function_base> >,
> #boost::shared_ptr<classad::ClassAd> > >, void>::invoke () 35 0x08214631
> #in boost::thread_group::join_all () 36 0x0042fdd8 in start_thread ()
> #from /lib/tls/libpthread.so.0 37 0x007a8d2a in clone () from
> #/lib/tls/libc.so.6
>