• 为什么写博客,是因为遇到的坑,不希望别人在踩一遍!
  • 选择了,剩下的就是坚持和努力-------致自己!
  • 当能力达不到梦想时,更需要学习,努力,拼搏

ELK5系列日志收集平台(十)elasticalert 日志告警

ELK 雪豹 7年前 (2018-02-11) 1017次浏览 0个评论

images

1.1 下载编译elastalert

找到一个存放tar包的目录,git clone 复制
[root@elk1 yunwei]# cd /home/yunwei/tools/
[root@elk1 tools]# git clone https://github.com/Yelp/elastalert.git
[root@elk1 tools]# cd elastalert/
[root@elk1 elastalert]# pip install cryptography

1.2 提示:这一步可能报错 bash: pip: command not found…

下载安装
 wget"https://pypi.python.org/packages/source/p/pip/pip-1.5.4.tar.gz#md5=834b2904f92d46aaa333267fb1c922bb" --no-check-certificate
 tar -axf pip-1.5.4.tar.gz 
 cd pip-1.5.4/
python setup.py install

python错误:ImportError: No module named setuptools
这句错误提示的表面意思是:没有setuptools的模块,说明python缺少这个模块,那我们只要安装这个模块即可解决此问题,下面我们来安装一下:
在命令行下:
下载setuptools包
shell# wgethttp://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz
解压setuptools包
shell# tar zxvf setuptools-0.6c11.tar.gz
shell# cd setuptools-0.6c11
编译setuptools
shell# python setup.py build
开始执行setuptools安装
shell# python setup.py install
在elastalert/ 目录下再次安装,提示每个人的目录不一样,需要自己改
[root@elk1 tools]# cd elastalert/
[root@elk1 elastalert]# pip install cryptography
[root@elk1 elastalert]# pip install -r requirements.txt 
[root@elk1 elastalert]# cp config.yaml.example config.yaml
[root@elk1 elastalert]# pip install "setuptools>=11.3"  ## 这里默认你已yum install python-pip并将源指向了aliyun.
[root@elk1 elastalert]# python setup.py install

1.3 配置邮箱验证

[root@elk1 elastalert]# vim /home/yunwei/tools/elastalert/example_rules/smtp_auth_file.yaml
user: 18501399u9  #自己的邮箱用户
password: zhangLLLLLLL  #邮箱密码,如果是163等是 客户端授权密码

1.4 修改配置alert配置文件

• name:配置,每个rule需要有自己独立的name,一旦重复,进程将无法启动。
• type:配置,选择某一种数据验证方式。
• index:配置,从某类索引里读取数据,目前已经支持Ymd格式,需要先设置 use_strftime_index:true,然后匹配索引,配置形如:index: logstash-es-test%Y.%m.%d,表示匹配logstash-es-test名称开头,以年月日作为索引后缀的index。
• filter:配置,设置向ES请求的过滤条件。
• timeframe:配置,累积触发报警的时长。
• alert:配置,设置触发报警时执行哪些报警手段。不同的type还有自己独特的配置选项。目前

[root@elk1 example_rules]# cat example_frequency.yaml 
# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
# es_host: elasticsearch.example.com

# (Optional)
# Elasticsearch port
# es_port: 14900

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True

# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# (Required)
# Rule name, must be unique
name: Example frequency rule

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: frequency

# (Required)
# Index to search, wildcard supported
index: logstash-*

# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
<span style="color: #ff0000;">num_events: 1   # 限制时间内,发生事件次数</span>

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
 <span style="color: #ff0000;"> minutes: 1   #限制时间刻度,hours 小时 minutes 分钟</span>

# (Required)
# A list of Elasticsearch filters used for find events
# These filters are joined with AND and nested in a filtered query
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
filter:
#- term:
#    some_field: "some_value"
<span style="color: #ff0000;"># ES 查询,用以过滤 发现有message: 中有  测试一下  文字就告警。</span>
<span style="color: #ff0000;">- query_string: 
    query: "message:测试一下"</span>

# (Required)
# The alert is use when a match is found
alert:
- "email"
<span style="color: #ff0000;">#接收报警邮件的邮箱
email:
    - "18501ghg909@163.com"</span>
# (required, email specific)
# a list of email addresses to send alerts to
<span style="color: #ff0000;">smtp_host: smtp.163.com
smtp_port: 25
smtp_ssl: false
# smtp_auth_file.yaml 地址路径具体根据自己而定
smtp_auth_file: /home/yunwei/tools/elastalert/example_rules/smtp_auth_file.yaml
# 从哪个邮箱发送
from_addr: 18501dddd3909@163.com
</span>

1.5 config.yaml 配置讲解

• Rules_folder:用来加载下一阶段rule的设置,默认是example_rules
• Run_every:用来设置定时向elasticsearch发送请求
• Buffer_time:用来设置请求里时间字段的范围,默认是45分钟
• Es_host:elasticsearch的host地址
• Es_port:elasticsearch 对应的端口号
• Use_ssl:可选的,选择是否用SSL连接es,true或者false
• Verify_certs:可选的,是否验证TLS证书,设置为true或者false,默认为true
• Es_username:es认证的username
• Es_password:es认证的password
• Es_url_prefix:可选的,es的url前缀(我的理解是https或者http)
• Es_send_get_body_as:可选的,查询es的方式,默认的是GET
• Writeback_index:elastalert产生的日志在elasticsearch中的创建的索引
• Alert_time_limit:失败重试的时间限制

[root@elk1 elastalert]# cat config.yaml
# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: example_rules

# How often ElastAlert will query Elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  minutes: 1

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 15

# The Elasticsearch hostname for metadata writeback
# Note that every rule can have its own Elasticsearch host
<span style="color: #ff0000;">es_host: 172.16.1.20    #修改为自己es 的地址</span>

# The Elasticsearch port
es_port: 9200

# The AWS region to use. Set this when using AWS-managed elasticsearch
#aws_region: us-east-1

# The AWS profile to use. Use this if you are using an aws-cli profile.
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
# for details
#profile: test

# Optional URL prefix for Elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to Elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# Use SSL authentication with client certificates client_cert must be
# a pem file containing both cert and key for client
#verify_certs: True
#ca_certs: /path/to/cacert.pem
#client_cert: /path/to/client_cert.pem
#client_key: /path/to/client_key.key

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

1.6 启动

[root@elk1 elastalert]#  python -m elastalert.elastalert --verbose --rule example_frequency.yaml

测试:
根据自己监控的日志,可以自己手动写条测试日志。


有需要可以联系微信xuebao19930721和加入微信群
喜欢 (3)
发表我的评论
取消评论

表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址