Como NÃO testar SW?

uma palestra da

@freire_da_silva

• Director of Product Safety @IndustrialLogic

• Agil desde 2001, lean desde a 1ª série

• Founder: AgilCoop, AgilBits e ).(

• Mestrado na USP: “Reflexões sobre o Ensino de Metodologias Ágeis na Academia, Indústria e Governo”

• Gosto de nadar e construir casas

• Experiência e sucesso com muitos amigos: alex@industriallogic.com

Você TESTA?

TODO SW TEM BUG?

TODO SW TEM BUG!

/1947

1999: !Imperial instead of metric units !cause Mars Climate Orbiter…

to disintegrate!

if((err=SSLFreeBuffer(&hashCtx))!=0) goto fail; if((err=ReadyHash(&SHA1,&hashCtx))!=0) goto fail; if((err=SHA1.update(&hashCtx,&rand))!=0) goto fail; goto fail; if((err=SHA1.update(&hashCtx,&sign))!=0) goto fail; if((err=SHA1.final(&hashCtx,&hsOut))!=0) goto fail; err = sslRawVerify(…);

sslKeyExchange.c

if((err=SSLFreeBuffer(&hashCtx))!=0) goto fail; if((err=ReadyHash(&SHA1,&hashCtx))!=0) goto fail; if((err=SHA1.update(&hashCtx,&rand))!=0) goto fail; goto fail; if((err=SHA1.update(&hashCtx,&sign))!=0) goto fail; if((err=SHA1.final(&hashCtx,&hsOut))!=0) goto fail; err = sslRawVerify(…);

sslKeyExchange.c

/* Read type & payload length first */

hbtype = *p++; n2s(p,payload); p1 = p; !/* Enter response type, length & copy payload */

*bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, p1, payload);

/* Read type & payload length first */

hbtype = *p++; n2s(p,payload); if (1+2+payload+16 > s->s3->rrec.length) return 0; /* silently discard per RFC 6520 sec. 4 */ p1 = p; !/* Enter response type, length & copy payload */

*bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, p1, payload);

Você AUTOMATIZA SEUS TESTES?

?

#1e5d91

#1e5d91

1describe “test” do 2 it “should do nothing” do 3 fail 4 end 5 end

1describe “test” do 2 it “should do nothing” do 3 fail 4 end 5 end

1describe “test” do 2 it “should do nothing” do 3 true 4 end 5 end

1describe “test” do 2 it “should do nothing” do 3 true 4 end 5 end ?

NON-FUNCTIONAL REQUIREMENTS

SCALABILITY

RELIABILITY

SECURITYUSABILITY

ACCESSIBILITY

MAINTAINABILITY

#1e5d91

por James Reason

RISCOS

FALHAS

Você APAGA seus testes?

?

100% COVERED CODE.

OF CODE YOU COVERLINE

60@Test 61public void setPrice() { 62 Item item = new 63 BasicItem(0.0, true); 64 65 assertEquals(0.0, 66 item.getPrice()); 67 68 //set new price 69 item.setPrice(1.0); 70 assertEquals(1.0, 71 item.getPrice()); 72}

74@Test 75public void isImported_true(){ 76 Item item = new 77 BasicItem(0.0, true); 78 79 assertTrue(item.isImported()); 80}

74@Test 75public void isImported_false(){ 76 Item item = new 77 BasicItem(0.0, false); 78 79 assertFalse(item.isImported()); 80}

28public Double getPrice(){ 25 return price; 26} 27 28public boolean isImported(){ 29 return imported; 30}

WHY DO WE TEST?

1while true do 2 print “vou parar?” 3end

!Murphy’s  law

PAIN

FLOW

Cost of Quality Assurance (QA)

Analysis Design Coding Unit Tests Acceptance Tests Production

freqüencia de Uso Das Funcionalidades

RARO 19%

Nunca 45%

Algumas VEZES 16%

Frequente 13%

Sempre 7%

Pair Programming

?

feedback vs

noise

Speed Robustness

non-deterministic

flakycode

Test  failed  21  -mes  locally  &  36  -mes  in  dev  build  in  the  last  6  months.  

@google If they fail we simply run flaky tests 3x (and keep statistics). Developer time is much more valuable than server time.

1@Test 2public void errorExerciseUploadTooBig { 3 goToExerciseIntroPage(); 4 goToNextPage(); 5 6 type(“labArchive”, path() + TOO_BIG); 7 pause(1000L); 8 assertText(“error: Upload too big!”); 9}

sleep() demais

How much QUALITY is ENOUGH?

Bugs/1KLOC

0

1

3

4

5

Indústria Nasa

0,004

Cost($/LOC)

0

225

450

675

900

Indústria Nasa

5

850

It will only run once

VS

<- costumer facing

back office ->

@martinfowler you’re doing enough testing if the following is true: ■You rarely get bugs that escape

into production ■You are rarely hesitant to change

some code for fear it will cause production bugs

@joshuakerievsky Test first/after misses the point that TDD is more about emergent design than it is about testing. Do you practice emergent design?

@kentbeck If I don’t typically make a mistake(...), I don’t test for it. !

Wish there were more examples of “what not to test and how not to test”.

2def create_name(fname, lname): 3 if not isInstance(fname, basestring): 4 raise TypeError(“fname must be a String”) 5 if not isInstance(lname, basestring): 6 raise TypeError(“lname must be a String”) 7 name = fname + lname

1 require ‘spec_helper‘ 2 describe Candidate do 3 context ‘associations‘ do 4 it { should have_many(:proposals) } 5 end 6 7 context ‘validations‘ do 8 it { should validate_presence_of :name } 9 10 it { should ensure_lenght_of(:phone). 11 is_at_least(7). 12 is_at_most(14) 13 } 14 15 it { should_not 16 allow_value(‘blah‘).for(:site) } 17 18 it { should 19 allow_value(‘http://www.blah.com‘) 20 .for(:site) } 21 end 22end

1 require ‘valid_url‘ 2 class Candidate < ActiveRecord::Base 3 has_many :proposals 4 5 validates :name, presence: true 6 7 validates :phone, :length => 8 {:in => 8..14}, 9 :allow_blank 10 11 validates :site, :url => true, 12 :allow_nil => true 13 14end

1 require ‘spec_helper‘ 2 describe Candidates do 3 4 let(:candidate) {double ‘candidate‘} 5 6 before :each do 7 Candidate 8 .should_receive(:find) 9 .with(1).and_return(candidate) 10 end 11 12 it ‘should find the candidate‘ do 13 c = Candidate.find(1) 14 c.should eql candidate 15 end 16end

1Feature: Create proposal 2 As a candidate 3 I want to post my proposals 4 So that voters can evaluate them 5 6 Scenario: 7 Given I am logged in 8 And I am posting a proposal 9 When 10 I fill all fields of the proposal 11 Then 12 I should see a success message

1Scenario: Client sees tooltip for plan 2 Given 3 I select the ‘light‘ plan 4 When 5 I mouse over ‘tooltip‘ 6 Then 7 I should see ‘tooltip‘ content 8 And 9 I mouse out ‘tooltip‘ 10 Then 11 I should not see ‘tooltip‘ content

1require ‘spec_helper‘ 2describe ShoppingCart do 3 4 let(:user) {double ‘user‘} 5 let(:product) {double ‘product‘} 6 7 before :each do 8 Authenticator.should_receive(:auth) 9 .and_return(true) 10 end 11 12 it ‘should addProduct & getTotal‘ do 13 Authenticator.auth(user) 14 cart = ShoppingCart.new 15 cart.add_product(product) 16 cart.get_total 17 end 18end

50@Test 51public void changeMarks() { 52 bot.leftClickAt(view, 53 800, 508); 54 addMarkAt(‘drama’, 1); 55 56 bot.leftClickAt(view, 57 900, 508); 58 addMarkAt(‘act’, 3); 59 60 bot.verifyTooltipAt(30, 190); 61}

PIXEL NAZI

1 require ‘spec_helper‘ 2 describe AddressController do 3 4 it ‘should calculate shipping‘ do 5 get :shipping, :zipcode => ‘90210‘ 6 assigns(:shipping).should == ‘8.2‘ 7 end 8 9 end

external dependencies:

DRY

WET

fixtures considered harmful?

JAVASCRIPT?

module('MultiSelectQuizTests',{ setup: function() { var container = document.getElementById("qunit-fixture"); var question = "Which foods are Mexican?"; var answers = [ { 'answerText': 'Tacos', 'value': true }, { 'answerText': 'Sushi', 'value': false } ]; ! this.myQuiz = new MultiSelectQuiz ( container, question, answers ); }, }); !test( "One correct", function() { checkRadio(0, true); checkRadio(1, true); deepEqual(this.myQuiz.grade(), 1, "just 1"); });

CSS?

it(‘centers logo at top of page’, function() { expect(isConTenteCenteredInPage(logo)).toBe(true); expect(elementPicelsFromTopOfPage(logo)).toBe(12); expect(fontSizeOf(logo)).toBe(22); expect(textColorOf(logo)).toBe(WHITE); });

Students(Can’t(Access(Service(

New(Produc5on(Server(

Ok(To(Toggle((Test(Passes(

Immune(System(Fails(

No(Auto?(Rollback(

No(SMS(

No(Policy(

Non?Standard(Tomcat(Runner(

Nginx(Points(To(Down(Service(

Non?Standard(Java(Version( JRE(Crash(

No(Policy(

And$

And$ And$ And$

análise de uma falha

Test journeys

NÃO É BUG É FEATURE

JAVASCRIPT

80% só até amanhã!código de desconto: DEVDAY2014

HTTP://industriallogic.com/shop

PYTHON