Hi Andrew,
Thanks, I'll test it and let you know. I am doing the "pseudo-old" way, in the sense that I still use logstash-forwarder on clients to send logs to a central logstash server. I guess I will replace logstash-forwarder with its modern child filebeat.
Cheers
Federico
-----Original Message-----
From: Testbed Support for GridPP member institutes [mailto:[log in to unmask]] On Behalf Of Andrew Lahiff
Sent: 29 January 2016 11:14
To: [log in to unmask]
Subject: Re: hunger for logstash patterns
Hi Federico,
Below (*) is an attempt at a filter for the ARC CE gm-jobs log. It seems to work, but it hasn't been tested extensively yet.
Just curious - are you doing things 'the old way' and are running Logstash directly on the machines you want to monitor, or are you using Filebeat instead? i.e. you have Filebeat reading log files, sending the data to remote host(s) running Logstash.
Thanks,
Andrew.
(*)
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:status} - job id: %{WORD:jobid}, unix user: %{POSINT:user}:%{POSINT:group}, name: %{NOTSPACE:name}, owner: %{QUOTEDSTRING:owner}, lrms: %{NOTSPACE:lrms}, queue: %{WORD:queue}(, lrmsid: %{HOSTNAME:lrmsid})?(, failure: %{QUOTEDSTRING:failure})?" }
remove_field => "message"
}
date {
match => [ "timestamp", "YYYY-MM-dd HH:mm:ss" ]
}
}
________________________________
From: Testbed Support for GridPP member institutes [[log in to unmask]] on behalf of Federico Melaccio [[log in to unmask]]
Sent: Thursday, January 28, 2016 3:25 PM
To: [log in to unmask]
Subject: hunger for logstash patterns
Hi all,
Given the seemingly wide diffusion of the ELK stack, I was wondering if logstash patterns to parse some wlcg services exist or have been written by someone. More specifically, I am interested in argus and arc ce logs. Thanks.
Regards,
Federico
|