Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
D
docker-elk
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
JIRA
JIRA
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
段孔乐
docker-elk
Commits
7f8489f8
Commit
7f8489f8
authored
Jul 05, 2021
by
段孔乐
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
modify install script
parent
7cf3eb4a
Expand all
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
62 additions
and
91 deletions
+62
-91
ELK部署文档.md
ELK部署文档.md
+1
-47
README.md
README.md
+0
-0
logstash.conf
logstash/pipeline/logstash.conf
+61
-44
start-and-init.sh
start-and-init.sh
+0
-0
No files found.
ELK部署文档.md
deleted
100644 → 0
View file @
7cf3eb4a
# Elastic stack (ELK) 部署说明
++ /dev/null
# Elastic stack (ELK) 部署说明
本项目基于github开源项目:https://github.com/deviantony/docker-elk
项目地址:https://gitlab.33.cn/dkl/docker-elk
###测试环境
http://124.71.144.118:15601/app/discover
### 简介
Elasticsearch 用于数据存储
Logstash 用于数据处理
Kibana 用于数据的展示
Filebeat 负责数据采集
###安装
wget https://gitlab.33.cn/dkl/docker-elk/-/archive/1.0.0/docker-elk-1.0.0.tar.gz
tar -zxvf docker-elk-1.0.0.tar.gz
###启动
bash start.sh
###配置kibana
bash init-kibana.sh
### Filebeat 部署
项目地址: https://gitlab.33.cn/dkl/filebeat
### 1. quick install
Download quick install script
wget https://gitlab.33.cn/dkl/filebeat/raw/master/filebeat-quick-install.sh
Config logstash host and scan dir
vim filebeat-quick-install.sh
Run this script
bash filebeat-quick-install.sh
README.md
View file @
7f8489f8
This diff is collapsed.
Click to expand it.
logstash/pipeline/logstash.conf
View file @
7f8489f8
input
{
beats
{
port
=>
5044
}
beats
{
port
=>
5044
}
tcp
{
port
=>
5000
}
tcp
{
port
=>
5000
}
}
## Add your filters / logstash plugins configuration here
filter
{
if
(
"stat.log"
in
[
log
][
file
][
path
]) {
drop
{}
}
mutate
{
add_field
=> {
"input-type"
=>
"%{[agent][type]}-%{[input][type]}"
}
remove_field
=> [
"agent"
,
"ecs"
,
"tags"
,
"input"
,
"host"
]
}
if
(
"stat.log"
in
[
log
][
file
][
path
]) {
drop
{}
}
mutate
{
add_field
=> {
"input-type"
=>
"%{[agent][type]}-%{[input][type]}"
}
remove_field
=> [
"agent"
,
"ecs"
,
"tags"
,
"input"
,
"host"
]
}
json
{
source
=>
"message"
skip_on_invalid_json
=>
true
target
=>
"json"
}
if
"gateway"
in
[
log
][
file
][
path
]{
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"gateway-service-%{+YYYY.MM}"
} }
}
else
if
"swagger"
in
[
log
][
file
][
path
]{
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"swagger-service-%{+YYYY.MM}"
} }
}
else
if
"auth-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"auth-service-%{+YYYY.MM}"
} }
}
else
if
"file-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"file-service-%{+YYYY.MM}"
} }
}
else
if
"notify-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"notify-service-%{+YYYY.MM}"
} }
}
else
if
"pay-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"pay-service-%{+YYYY.MM}"
} }
}
else
if
"proof-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"proof-service-%{+YYYY.MM}"
} }
}
else
if
"tx-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"tx-service-%{+YYYY.MM}"
} }
if
"v1"
in
[
log
][
file
][
path
]{
mutate
{
add_field
=> {
"[service][version]"
=>
"v1"
} }
}
else
if
"v2"
in
[
log
][
file
][
path
]{
mutate
{
add_field
=> {
"[service][version]"
=>
"v2"
} }
}
else
{
mutate
{
add_field
=> {
"[service][version]"
=>
"v"
} }
}
else
if
"user-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"user-service-%{+YYYY.MM}"
} }
if
"gateway"
in
[
log
][
file
][
path
]{
mutate
{
add_field
=> {
"[service][name]"
=>
"gateway-service"
} }
}
else
if
"swagger"
in
[
log
][
file
][
path
]{
mutate
{
add_field
=> {
"[service][name]"
=>
"swagger-service"
} }
}
else
if
"auth-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"auth-service"
} }
}
else
if
"file-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"file-service"
} }
}
else
if
"notify-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"notify-service"
} }
}
else
if
"pay-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"pay-service"
} }
}
else
if
"proof-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"proof-service"
} }
}
else
if
"tx-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"tx-service"
} }
}
else
if
"role-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"role-service"
} }
}
else
if
"user-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"user-service"
} }
}
else
if
"department-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"department-service"
} }
}
else
if
"process-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"process-service"
} }
}
else
if
"template-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"template-service"
} }
}
else
if
"trace-service"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"trace-service"
} }
}
else
if
"sync"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"sync"
} }
}
else
if
"convert"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"convert"
} }
}
else
if
"exrpc"
in
[
log
][
file
][
path
] {
mutate
{
add_field
=> {
"[service][name]"
=>
"exrpc"
} }
}
else
{
mutate
{
add_field
=> {
"[@metadata][target_index]"
=>
"others-%{+YYYY.MM}
"
} }
}
mutate
{
add_field
=> {
"[service][name]"
=>
"others
"
} }
}
}
output
{
elasticsearch
{
index
=>
"%{[@metadata][target_index]
}"
hosts
=> [
"elasticsearch:9200"
]
elasticsearch
{
index
=>
"%{[service][name]}-%{[service][version]}-%{+YYYY.MM
}"
hosts
=> [
"elasticsearch:9200"
]
}
}
}
\ No newline at end of file
install
.sh
→
start-and-init
.sh
View file @
7f8489f8
File moved
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment