talk at qconsp
DESCRIPTION
Talk presented at qconsp 2014. This talk shows how we scale django @ globoesporte.com to stand 8 million uses per day.TRANSCRIPT
![Page 1: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/1.jpg)
Usando Django paraServir 20 milhões de Usuários
QConSP 2014
![Page 2: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/2.jpg)
Rômulo Jales
2
Victor Pantoja
![Page 3: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/3.jpg)
Motivação
3
![Page 4: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/4.jpg)
Motivação
4
![Page 5: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/5.jpg)
Motivação
5
![Page 6: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/6.jpg)
Motivação
6
![Page 7: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/7.jpg)
Motivação
7
![Page 8: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/8.jpg)
Alguns números
Visitantes únicos: 20,7 milhões por mês
8
Visitas: 215 milhões por mês
8 milhões de visitas por dia!
![Page 9: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/9.jpg)
Alguns números
9
- 5 dias de matérias republicadas: 2020- confrontos republicados: 1023- homes republicadas: 3367- galerias republicadas: 92- enquetes republicadas: 57- CSS regerados: 708
![Page 10: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/10.jpg)
Tecnologias
10
![Page 11: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/11.jpg)
11
![Page 12: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/12.jpg)
12
Framework WEB MVC
Propicia uso de melhores práticas de Eng SW
Mas Django não escala...
![Page 13: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/13.jpg)
13
Uso de Middlewares
Cache de View
Mas Django não escala...
@cache_control(max_age=300)def top_atletas(request, periodo, quantidade, formato): dados = MateriaSemanticaController().top_atletas(periodo, quantidade) return render_to_json(dados, formato, 'top_atletas')
![Page 14: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/14.jpg)
Bottlenecks
14
MySQL»
Virtuoso
»
I/O»
3 BANCOS-
+18 APIs
-5 mounts
-
![Page 15: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/15.jpg)
15
32 Bits com PAE (problemas ao servir cache)
»
64 Bits»
![Page 16: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/16.jpg)
Pré-geraçãoNão deixe o usuário disparar ações críticas do sistema!
16
django-staticgenerator
![Page 17: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/17.jpg)
Pré-geração
Necessita que alguém ou algo dispare o processo
17
![Page 18: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/18.jpg)
18
CACHE EVERYWHERE
CACHE
![Page 19: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/19.jpg)
Memcached
19
Diminui a quantidade de memória utilizada: podemos ter várias listas de ids de objetos mas apenas uma entrada para cada objeto
»Aumenta a performance do sistema: não é necessário expurgar todo o cache de uma lista na alteração de um objeto. Basta expurgar o objeto.
»
![Page 20: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/20.jpg)
Memcached
Faça o caching de objetos e não de resultados de queries!
20
![Page 21: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/21.jpg)
SSI - Server Side Include
21
<div class="glb-grid-8 widget-plantao-semantico"> <!--#include virtual="/_ssi/plantao/pe/plantao/1" --></div>
HTTP/1.1 200 OKServer: nginxDate: Fri, 11 Apr 2014 13:19:20 GMTContent-Type: text/html; charset=utf-8Connection: closeLast-Modified: Fri, 11 Apr 2014 13:19:06 GMTExpires: Fri, 11 Apr 2014 13:24:06 GMTCache-Control: max-age=300
globoesporte.globo.com/pe/noticia/plantao.html
![Page 22: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/22.jpg)
Ajax vs SSIMelhor para a indexação daquele conteúdo (SEO)
»
22
Diferentes tempos de cache para componentes em uma mesma página
»
Melhora a performance client-side»
Diminui o número de requisições ao servidor
»
![Page 23: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/23.jpg)
![Page 24: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/24.jpg)
Baixo consumo de recursos da máquina»
24
simples configuração»
alta performance»
free e open source»módulos essenciais: purge, ssi e proxy cache
»
![Page 25: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/25.jpg)
Minify de CSS e JS (módulo compressor do Django)
»
25
CSS Sprite»Tempo de expiração adequado para a sua view
»
gzip»estáticos servidos pelo Varnish em uma farm separada
»
Performance Client-Side
caching do browser»
![Page 26: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/26.jpg)
26
Performance Client-Side
Configure os headers da requisição corretamente!HTTP/1.1 200 OKAccept-Encoding: gzip,deflate,sdchContent-Type: application/jsonVary: Accept-EncodingCache-Control: max-age=30Expires: Tue, 10 Oct 2014 11:45:24 GMTDate: Tue, 10 Oct 2014 11:44:59 GMTX-Varnish: 1952793053 1952791896
![Page 27: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/27.jpg)
27
Pitomba Spriter
pitomba.org
![Page 28: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/28.jpg)
FerramentasApache HTTP server benchmarking tool»
28
Monit: ferramenta open source para gerencia e monitoramento de processos, programas, arquivos, diretórios, serviços etc
»
WebPageTest: webpagetest.org»
Avalanche QA2»
![Page 29: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/29.jpg)
Palavra Final
29
Escalabilidade não é uma questão de linguagem ou framework mas de arquitetura da aplicação!!
![Page 30: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/30.jpg)
/DEV
30
http://
globodev.tumblr.com/
Conteúdo criado pelos
desenvolvedores da
globo.com
![Page 31: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/31.jpg)
31
![Page 32: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/32.jpg)
32
https://github.com/globocom/IWantToWorkAtGloboCom
http://bit.ly/quero-trabalhar-
globo-com
![Page 33: Talk at QConSP](https://reader034.vdocuments.com.br/reader034/viewer/2022042512/54b550ba4a7959c22c8b45cf/html5/thumbnails/33.jpg)
Perguntas!!
@romulojales
33
@victorpantojagithub.com/
victorpantoja
github.com/romulojales