pythonbrasil8
TRANSCRIPT
![Page 1: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/1.jpg)
Mauro Baraldi - Python Brasil 2012
Como medir a sua aplicação
Introdução à conceitos de profiling, debug e tracing
![Page 2: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/2.jpg)
Quem sou eu?
![Page 3: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/3.jpg)
http://www.66batman.com/cgi-bin/yabb2/YaBB.pl?num=1296085129
![Page 4: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/4.jpg)
Mauro Navarro Baraldi
Desenvolvedor
Mais
Programando em Python desde 2001
![Page 5: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/5.jpg)
Pai da Maria Eduarda,
do João Fernando
e marido da Camila
![Page 6: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/6.jpg)
Tópicos - Conceitos - Por que monitorar uma aplicação - Debugging - Profiling - Tracing - Memory Profiling - Holly Grail - Dúvidas?
![Page 7: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/7.jpg)
Conceitos- Debug: Metodo de Investigação do sistema (em execução) em busca do problema/erro.
- Profiling: Análise dinâmica de um sistema para medir recursos usado por ele no sistema.
- Tracing: Processo de gravar o comportamento de runtime do sistema para análise (debug/profiling)
![Page 8: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/8.jpg)
Porque monitorar uma aplicação - Aprendizado
- Bugs
- Segurança
- Desempenho
![Page 9: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/9.jpg)
Premature optimization is the root of all evil.
Tony Hoare
http://www.turing100.manchester.ac.uk/index.php/speakers/invited-list/11-speakers/39
![Page 10: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/10.jpg)
Debugginghttp://static.ddmcdn.com/gif/myths-everyday-science-1.jpg
![Page 11: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/11.jpg)
pdbBiblioteca da padrão para debugging de código
Prós: - fácil - simples - nativa
Contras: - IMHO nenhum!
![Page 12: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/12.jpg)
debug_app.py#!/usr/bin/env python
from pdb import set_trace
def debug_me(value): set_trace() # Aqui ativa-se o debug
if value > 0: print 'positivo' elif value < 0: print 'negativo' else: print 'zero'
if __name__ == '__main__': debug_me(0)
![Page 13: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/13.jpg)
resultado do pdbmauro@r2d2:~/projetos/pythonbrasil8$ python pdb_app.py > /home/mauro/projetos/pythonbrasil8/pdb_app.py(9)debug_me()-> if value > 0:(Pdb) n> /home/mauro/projetos/pythonbrasil8/pdb_app.py(11)debug_me()-> elif value < 0:(Pdb) n> /home/mauro/projetos/pythonbrasil8/pdb_app.py(14)debug_me()-> return 'zero'(Pdb) n--Return--> /home/mauro/projetos/pythonbrasil8/pdb_app.py(14)debug_me()->'zero'-> return 'zero'(Pdb) n--Return--> /home/mauro/projetos/pythonbrasil8/pdb_app.py(17)<module>()->None-> debug_me(0)(Pdb) n
![Page 14: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/14.jpg)
Profiling
http://www.guardian.co.uk/books/booksblog/2012/oct/30/scary-stories-halloween-francis-marion-crawford
![Page 15: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/15.jpg)
Standard Library
http://aidsource.ning.com/page/resource-library
![Page 16: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/16.jpg)
cProfileBiblioteca padrão para profiling
Prós: - standard lib - baixo impacto na execução. - usa extensão em C Contras: - só funciona com metodos - não há suporte para gráficos (nativo)
![Page 17: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/17.jpg)
profile_app.py#!/usr/bin/env python
def profile_me(limit):
counter = 0
while counter <= limit: counter = counter + 1
if __name__ == '__main__': profile_me(10)
![Page 18: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/18.jpg)
resultado do profilemauro@r2d2:~/projetos/pythonbrasil8$ python -m cProfile profile_app.py 1 function calls in 0.000 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
8 function calls in 0.001 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.001 0.001 cProfile.py:132(run) 1 0.000 0.000 0.001 0.001 cProfile.py:137(runctx) 1 0.000 0.000 0.001 0.001 cProfile.py:14(run) 1 0.000 0.000 0.000 0.000 cProfile.py:5(<module>) 1 0.000 0.000 0.000 0.000 cProfile.py:66(Profile) 1 0.000 0.000 0.001 0.001 profile_app.py:3(<module>) 1 0.000 0.000 0.000 0.000 profile_app.py:5(profile_me) 1 0.001 0.001 0.001 0.001 {method 'enable' of '_lsprof.Profiler' objects
![Page 19: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/19.jpg)
resultado do profile - legendancalls: numero de chamados
tottime: tempo total gasto na função dada (excluindo o tempo feito nas chamadas para sub-funções)
percall: tottime/ncalls
cumtime: tempo total gasto nesta e todas as subrotinas
percall: cumtime/ncalls
filename:lineno(function): dados respectivos de cada função
![Page 20: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/20.jpg)
profileOutra biblioteca inspirada na cProfile (Pure Python)
Prós: - standard lib Contras: - os mesmo da cProfile - impacto significante na execução.
Usada para extender as funcionalidades do profiler default (cProfile).
![Page 21: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/21.jpg)
hotshotBiblioteca para profiling em alto desempenho
Prós: - standard lib - baixo impacto na execução, cProfile.
Contras: - mais lento na geração dos dados - não funciona bem com threads - será descontinuado em breve.
![Page 22: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/22.jpg)
Bibliotecas e Aplicações Externas
![Page 23: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/23.jpg)
line_profilerProfiling linha a linha
Prós: - simples e fácil - usa a cProfile - visão detalhada da execução do código.
Contras: - desempenho inferior ao cProfile - resultado resumido
![Page 24: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/24.jpg)
resultado do line_profilermauro@r2d2:~/projetos/pythonbrasil8$ kernprof.py -l line_profile_app.pyWrote profile results to line_profile_app.py.lprof
mauro@r2d2:~/projetos/pythonbrasil8$ python -m line_profiler line_profile_app.py.lprof Timer unit: 1e-06 s
File: line_profile_app.pyFunction: profile_me at line 3Total time: 1e-05 s
Line # Hits Time Per Hit % Time Line Contents============================================================== 3 @profile 4 def profile_me(limit): 5 6 1 1 1.0 10.0 counter = 0 7 8 12 5 0.4 50.0 while counter <= limit: 9 11 4 0.4 40.0 counter = counter + 1
![Page 25: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/25.jpg)
Análisehttp://blog.hubspot.com/blog/tabid/6307/bid/31619/How-to-Conduct-Competitive-Analysis-to-Step-Up-Your-Content-Strategy.aspx
![Page 26: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/26.jpg)
RunSnakeRunAnalise de dados gerados a partir do profiler.
http://mg.pov.lt/run-snake-run.png
![Page 27: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/27.jpg)
$ python -m cProfile -o snake.data runsnakerun_app.py
runsnakerun_app.py
gerando os dados de análise
#!/usr/bin/env python
def print_function(data): print data
def char2ord(char): return ord(char)
def profile_me(word):
for char in word: num = char2ord(char) print_function('Ordinal from %s is %i' % (char, num))
if __name__ == '__main__': profile_me("Python Brasil 8")
![Page 28: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/28.jpg)
visualização da análise
![Page 29: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/29.jpg)
http://cavemanreflections.blogspot.com.br/p/follow-yellow-brick-road.html
Tracing
![Page 30: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/30.jpg)
traceBiblioteca da padrão para tracing de código
Prós: - fácil - simples - nativa
Contras: - IMHO nenhum!
![Page 31: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/31.jpg)
trace_app.py#!/usr/bin/env python
def trace_me(value):
if value > 0: return 'positivo' elif value < 0: return 'negativo' else: return 'zero'
if __name__ == '__main__': trace_me(0)
![Page 32: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/32.jpg)
resultado do line_profiler$ python -m trace --trace trace_app.py --- modulename: trace_app, funcname: <module>trace_app.py(3): def trace_me(value):trace_app.py(12): if __name__ == '__main__':trace_app.py(13): trace_me(0) --- modulename: trace_app, funcname: trace_metrace_app.py(5): if value > 0:trace_app.py(7): elif value < 0:trace_app.py(10): return 'zero' --- modulename: trace, funcname: _unsettracetrace.py(80): sys.settrace(None)
![Page 33: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/33.jpg)
disptraceTracing de código exportando a saida para HTML
Prós: - exporta em um formato visual
Contras: - roda dentro de um contexto da app
![Page 34: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/34.jpg)
disptrace_app.pyimport disptrace
def browse_disptrace(dt): import tempfile, webbrowser, urllib, os
html = dt.render() tempfiledes, temppath = tempfile.mkstemp(suffix='.html') tempfile = os.fdopen(tempfiledes, "w") tempfile.write(html) tempfile.close() tempurl = "file://{}".format(urllib.pathname2url(temppath)) webbrowser.get(None).open_new(tempurl)
def if_test(a, b, c): if a > 10: print "this" elif b > 10: print "that" else: print "result is", b*c
t = disptrace.DispTrace()t.runfunc(if_test, 1, 2, 3)browse_disptrace(t)
![Page 35: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/35.jpg)
visualização da análise
![Page 36: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/36.jpg)
http://spectrum.ieee.org/semiconductors/memory/the-quest-for-a-universal-memory
Memory Profiling
![Page 37: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/37.jpg)
memory profilerProfiling de memoria
Prós: - facil - simples - integração com pdb
Contras: - Não encontrei
![Page 38: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/38.jpg)
memory_profiler_app.py@profiledef potencia(expoente): for i in range(expoente): print '2^%i = %i' % (i, pow(2,i))
if __name__ == '__main__': potencia(10)
![Page 39: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/39.jpg)
resultado do memory_profiler$ python -m memory_profiler memory_profile_app.pyFilename: memory_profile_app.py
Line # Mem usage Increment Line Contents================================================ 3 @profile 4 9.00 MB 0.00 MB def potencia(expoente): 5 9.00 MB 0.00 MB for i in range(expoente): 6 print '2^%i = %i' % (i, pow(2,i))
![Page 40: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/40.jpg)
http://janeaustenfilmclub.blogspot.com.br/2011/06/monty-python-and-holy-grail.html
All together
![Page 41: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/41.jpg)
http://www.shopping.com/Toy-Vault-Monty-Python-s-Holy-Grail-Plush/info
Holly Grail
![Page 42: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/42.jpg)
psutilsbiblioteca de monitoramento de recursos do sistema
Prós: - multi-plataforma - fácil - resultados em estruturas de dados Python - monitora: - cpu/processos - memoria/discos, - rede - usuarios
![Page 43: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/43.jpg)
ReferenciasMódulos e aplicaçõespdb: http://docs.python.org/2/library/pdb.html
cProfile: http://docs.python.org/2/library/profile.html
line_profiler: http://packages.python.org/line_profiler/
RunSnakeRun: http://www.vrplumber.com/programming/runsnakerun/
trace: http://docs.python.org/2/library/trace.html
disp_trace: https://github.com/atsuoishimoto/disptrace
memory_profiler: https://github.com/fabianp/memory_profiler
psutil: http://code.google.com/p/psutil/
MaisCode Profiling: http://en.wikipedia.org/wiki/Code_profiling
Python Profiling: http://wiki.python.org/moin/PythonSpeed/PerformanceTips#Profiling_Code
Profile Hooks (decorators for Python profiling): http://mg.pov.lt/profilehooks/
![Page 44: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/44.jpg)
http://fogsmoviereviews.com/2012/06/24/movies-that-everyone-should-see-monty-python-and-the-holy-grail/
Dúvidas?
![Page 45: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/45.jpg)
Source Codehttps://github.com/maurobaraldi/pybr8
![Page 46: Pythonbrasil8](https://reader034.vdocuments.com.br/reader034/viewer/2022052621/558568b3d8b42a3e6f8b48aa/html5/thumbnails/46.jpg)
http://spectrumculture.com/2010/01/film-dunce-monty-python-and-the-holy-grail.html/
Thank You