tdc2016poa | trilha devops - monitoramento real-time com elk
TRANSCRIPT
![Page 1: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/1.jpg)
ELKMétricas e monitoramento "real-time"
![Page 3: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/3.jpg)
Fluxo de requisição3
Loja Pagamento
![Page 4: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/4.jpg)
Fluxo de requisição no século 214
LB
API
API
API
S
S
S
S
S
S
S
S
S
S
S
S
S
S
S
S
S
S
S
S
![Page 5: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/5.jpg)
Elastic.co
https://www.elastic.co/
5
![Page 6: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/6.jpg)
Elastic.co
Produtos
● Elasticsearch● Logstash● Kibana● Beats● X-Pack
○ Shield○ Marvel○ Graph○ Watcher
6
Serviços● Suporte● Treinamento● Cloud● Consultoria
![Page 7: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/7.jpg)
ELK
● Elasticsearch● Logstash● Kibana
7
![Page 8: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/8.jpg)
Elasticsearch
https://www.elastic.co/products/elasticsearch
8
![Page 9: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/9.jpg)
Logstash
https://www.elastic.co/products/logstash
9
![Page 10: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/10.jpg)
Kibana
https://www.elastic.co/products/kibana
10
![Page 11: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/11.jpg)
PARSEARINDEXARVISUALIZAR
11 ELK WorkflowCLUSTER
![Page 12: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/12.jpg)
Etapa de log parsing com Logstash
Entrada● Beats● Couchdb_changes● Heroku● File● Kafka● TCP● sqs● Redis● Rabbitmq● Elasticsearch
Filtro● Aggregate● Mutate● Alter● CSV● environment● TCP● JSON● range● grok● geoip
Saída● CSV● Datadog● Email● Elasticsearch● Exec● File● Graphite● InfluxDB● MongoDB● s3
12
![Page 13: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/13.jpg)
Etapa de indexar no Elasticsearch
● Normalização de texto○ Acentos○ Sinônimos○ Linguística...
13
![Page 14: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/14.jpg)
Etapa de visualizar com Kibana
● Dashboard em tempo real○ Com várias visualizations
● Vários tipos de gráficos○ Pie chart○ Mapas○ Barras○ Gráfico de linhas
● Criação de visualizations dinâmica○ Suporte a queries do Elasticsearch○ Suporte a agregações
14
![Page 15: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/15.jpg)
Criando Visualization15
![Page 16: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/16.jpg)
Monitoramento de serviços16
![Page 17: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/17.jpg)
Monitoramento e métricas de APIs17
![Page 18: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/18.jpg)
Métricas de negócio, BigData e anomalias 18
![Page 19: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/19.jpg)
Alertas, proteção e monitoramento19CLUSTER
● Security● Alerting● Monitoring● Graph● Reporting
![Page 20: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/20.jpg)
Na produção
● ELK as a service○ Logz.io○ Qbox.io
● Cloud○ AWS○ DigitalOcean○ Elastic
20
![Page 21: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/21.jpg)
Por onde começar21
![Page 22: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/22.jpg)
Valeu!Perguntas?
![Page 23: TDC2016POA | Trilha DevOps - Monitoramento real-time com ELK](https://reader031.vdocuments.com.br/reader031/viewer/2022020204/58a059f21a28ab5c1c8b5a13/html5/thumbnails/23.jpg)
Referências23http://walde.co/tag/elasticsearch/
https://github.com/guardian/elk-stack
http://techblog.netflix.com/2016/02/evolution-of-netflix-data-pipeline.html
http://www.front2backdev.com/2016/05/03/geo-temporal-anomaly/
https://dzone.com/articles/which-elk-stack-solution-is-right-for-you?utm_content=buffer29ebc&utm_med
ium=social&utm_source=twitter.com&utm_campaign=buffer
http://blog.takipi.com/hosted-elasticsearch-the-future-of-your-elk-stack/
http://everythingshouldbevirtual.com/highly-available-elk-elasticsearch-logstash-kibana-setup