engenharia de tráfego na internet: entendendo o de tráfego...

37
PPGEE Programa de Pós-Graduação em Engenharia Elétrica Engenharia de Tráfego na Internet: Entendendo o processo de tomada de decisão para gerenciamento de tráfego na américa do sul. Internet Traffic Engineering: Understanding the decision- making process for traffic management in South America. Márcio De Deus

Upload: hadiep

Post on 16-Nov-2018

220 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

    PPGEE

Programa de Pós-Graduação em Engenharia Elétrica

Engenharia de Tráfego na Internet: Entendendo o processo de tomada de decisão para gerenciamento de tráfego na américa do sul. Internet Traffic Engineering: Understanding the decision-making process for traffic management in South America.

Márcio De Deus

Page 2: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Grupo  de  Pesquisa    Planejamento  e  análise  de  tráfego  em  redes  

 Reseach  Group  

Network  planning  and  traffic  analysis      

Paulo  H.  P.Carvalho  (Supervisor),    Márcio  A.  de  Deus,    João  Paulo  Leite.  

 [email protected],  [email protected],  [email protected]  

 Departamento  de  Engenharia  Elétrica  

LEMOM  Lab  Universidade  de  Brasilia  

Brasilia  –  DF  

Page 3: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

How  to  assure  the  effecQve  resource  usage?  

Network  OperaQonà  Traffic  engineering?  

Engineering  à  Capacity  Planning?  

SoluQon  not  based  in  a  math  model:  

Page 4: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

How  to  assure  the  effecQve  resource  usage?  

Network  OperaQonà  Traffic  engineering?  

Engineering  à  Capacity  Planning?  

SoluQon  not  based  in  a  math  model:  

or   =  ??  

Page 5: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

How  to  assure  the  effecQve  resource  usage?  

Network  OperaQonà  Traffic  engineering?  

Engineering  à  Capacity  Planning?  

SoluQon  based  in  a  math  model:  

Page 6: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

A traffic or network model can only exists if the existence of a math model could make possible to extract basic parameters to simulate a future utilization. [Takine e Masuyama]

Math  Model  

0

50

100

150

200

250

300

01 -

21:

00:0

0

02 -

03:

00:0

0

02 -

09:

00:0

0

02 -

15:

00:0

0

02 -

21:

00:0

0

03 -

03:

00:0

0

03 -

09:

00:0

0

03 -

15:

00:0

0

03 -

21:

00:0

0

04 -

03:

00:0

0

04 -

09:

00:0

0

04 -

15:

00:0

0

04 -

21:

00:0

0

05 -

03:

00:0

0

05 -

09:

00:0

0

05 -

15:

00:0

0

05 -

21:

00:0

0

06 -

03:

00:0

0

06 -

09:

00:0

0

06 -

15:

00:0

0

06 -

21:

00:0

0

07 -

03:

00:0

0

07 -

09:

00:0

0

07 -

15:

00:0

0

07 -

21:

00:0

0

08 -

03:

00:0

0

08 -

09:

00:0

0

08 -

15:

00:0

0

08 -

21:

00:0

0

Day - Time

Mb

ps

/ n VoIPP2PE-MailBrowsing

BANDWIDTH SUM

TIMESLOT

SERVICE

Parameters  PredicQon  

Current  ConfiguraQon  

New  configuraQon  

Self-similar Monofractal Multifractal

0100200300400500600700800900

1000

Sun A

pr

Sun A

pr

Mon A

pr

Mon A

pr

Tue A

pr

Tue A

pr

Wed A

pr

Wed A

pr

Wed A

pr

Thu A

pr

Thu A

pr

Fri A

pr

Fri A

pr

Sat A

pr

Sat A

pr

httpP2PStreamingVoIP

Page 7: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Decision-­‐making  process  

Collect  and  process  data  

Goal  (Best  SoluQon  

Ranking)  

Acquired  knowledge  

Decision-­‐making  

MulQcriteria  analysis  AlternaQve  a0  

AlternaQve  a1  

AlternaQve  an  

Criterion  c0  

Criterion  c1  

Criterion  cn  

Goal  

D={dji}  W  

{w1  ,…,  wn)  

30/$>?-/4$+4-/05/</45/4-$+4$3$A>=<./[$345$74A/0-3+4$H3E2$!>0/>D/0($-*/$D3.7/$>?$3$A0+-/0+>4$?>0$3$8+D/4$

3A-+>4$A34$K/$3??/A-/5$KE$=34E$?3A->0,$H*+A*$30/$/[-/043.$->$-*/$5/A+,+>4$,E,-/=$345$A344>-$K/$A>4-0>../5$

KE$ -*/$ 5/A+,+>4$ =3M/02$ \>0$ +4,-34A/($ +4$ 3$ ,7<<.+/0$ ,/./A-+>4$ <0>K./=$ +4$ 3$ ,7<<.E$ A*3+4$ =3438/=/4-$

,E,-/=($-*/$<0+A/($H*+A*$+,$3$A0+-/0+>4($A34$K/$+4?.7/4A/5$KE$/[-/043.$?3A->0,$,7A*$3,$,7<<.E$345$5/=345$

>?$ -*3-$ <0>57A-$ +4$ -*/$ =30M/-($ H*+A*$ 30/$ ?70-*/0$ 5/</45/4-$ >4$ >-*/0$ /A>4>=+A3.$ 345$ 4>4:/A>4>=+A3.$

?3A->0,$N/282($+4-/0/,-$03-/($></4+48$>?$4/H$=30M/-,($.3K>0$5+,<7-/($43-703.$5+,3,-/0($-/00>0+,-$3--3AM($/-A2O2$$

$

L*+,$<3</0$?>A7,/,$>4$=>5/.+48$74A/0-3+4-E$+4$!"#!$345$<0/,/4-,$>70$>48>+48$H>0M$>4$5/D/.><+48$3$

?03=/H>0M$K3,/5$>4$13E/,+34$4/-H>0M$N1]O$345$+-,$/[-/4,+>4$+4?.7/4A/$5+3803=,$NZ#O($?>0$!"#!2$$

$

1]$ 345$ Z#$ 30/$ 5/D/.></5$ +4$ -*/$ 30-+?+A+3.$ +4-/..+8/4A/$ N6ZO$ A>==74+-E$ 3,$ <0+4A+<./5$ ?>0=3.+,=,$ ?>0$

0/<0/,/4-+48$ 345$ 0/3,>4+48$ 745/0$ 74A/0-3+4-E$ +4$ +4-/..+8/4-$ ,E,-/=,2$ Z4$ 3$1]($ /4-+-+/,$ >?$ +4-/0/,-$ N/282($

5/A+,+>4$ A0+-/0+3$ 345$ ,7K:A0+-/0+3($ ?3A->0,$ -*3-$ +4?.7/4A/$ -*/=O$ 30/$ -0/3-/5$ 3,$ 0345>=$ D30+3K./,$ 345$

0/<0/,/4-/5$ 3,$ 4>5/,$ +4$ -*/$ 4/-H>0M($ A>44/A-/5$ KE$ 5+0/A-/5$ 30A,$ +45+A3-+48$ <0>K3K+.+,-+A$ 5/</45/4A+/,$

K/-H//4$ -*/=2$L*/$4/-H>0M$,-07A-70/($ ->8/-*/0$H+-*$A>45+-+>43.$<0>K3K+.+-E$ -3K./,$3,,>A+3-/5$H+-*$/3A*$

4>5/($ <0>D+5/$ 3$ A>=<3A-$ 0/<0/,/4-3-+>4$>?$ -*/$ ^>+4-$ <0>K3K+.+-E$ 5+,-0+K7-+>4$ >?$ 3..$ D30+3K./,2$6$ ,7+-/$ >?$

3.8>0+-*=,$*3D/$K//4$5/D/.></5$?>0$<0>K3K+.+,-+A$+4?/0/4A/$H+-*$1]($/,</A+3..E$-*>,/$H*+A*($H*/4$,>=/$

D30+3K./,_$D3.7/,$*3D/$K//4$>K,/0D/5($ A>=<7-/$ -*/$<>,-/0+>0$<0>K3K+.+-+/,$>?$>-*/0,2$ Z#$/[-/45,$1]$KE$

355+48$7-+.+-E$4>5/,$345$5/A+,+>4$4>5/,($3..>H+48$>4/$->$</0?>0=$D30+>7,$5/A+,+>4$0/.3-/5$-3,M,($+4A.75+48$

A>=<7-+48$-*/$/[</A-/5$7-+.+-E($8+D/4$>K,/0D3-+>4,$345$5/A+,+>4$A*>+A/,($345$?+45+48$-*/$><-+=3.2$1]$345$

Z#$ -*7,$<0>D+5/$3$ -*/>0/-+A3..E$H/..:?>745/5$345$></03-+>43.$K3,+,$ ?>0$=>5/.+48$!"#!$345$<0>K./=:

,>.D+482$$

$

L*/$<0/,/4-3-+>4$>?$>70$H>0M$>4$5/D/.><+48$3$1]$K3,/5$?03=/H>0M$?>0$!"#!$+4$-*+,$<3</0$+,$>0834+Q/5$

3,$?>..>H,2$Z4$@/A-+>4$%($-*/$K3AM80>745$>?$!"#!$345$K0+/?$A0+-+A3.$0/D+/H$>?$/[+,-+48$!"#!$=/-*>5,$

30/$<0>D+5/52$13E/,+34$4/-H>0M,$345$ +4?.7/4A/$5+3803=,$30/$5+,A7,,/5$ +4$@/A-+>4$`2$@/A-+>4$'$>7-.+4/,$

-*/$<0><>,/5$?03=/H>0M($->8/-*/0$H+-*$3$0744+48$/[3=<./2$\+43..E($A>4A.7,+>4,$345$?70-*/0$0/,/30A*$30/$

5+,A7,,/5$+4$@/A-+>4$F2$

!!ON!0@K=6!4F6=CF6;!.CI6:689!0;<69D!P04.0Q!$

!"#!$0/?/0,$->$=3M+48$5/A+,+>4,$+4$-*/$<0/,/4A/$>?$=7.-+<./$A0+-/0+32$Z4$-*/$/,,/4A/($3$!"#!$<0>K./=$

+,$ ?>0=/5$ +4->$ *+/030A*E$ A>=<>,/5$ >?$ ?>70$ /./=/4-,a$ -*/$ 2/*#($ -*/$ /8E)-$%5).($ -*/$ -(%$)(%*($ 345$ -*/$*#$)(0*$%5).2$ L*/,/$ /./=/4-,$ A34$ K/$ <0/,/4-/5$ +4$ 3$ =3-0+[$ ?>0=3-2$ b/-$ I((G

; 9**M �� K/$ 3$ ,/-$ >?$

5/A+,+>4$3.-/043-+D/,$345$ I((G; 0--' �� $3$,/-$>?$A0+-/0+3$3AA>05+48$->$H*+A*$5/,+03K+.+-E$>?$34$3A-+>4$+,$

^758/52$ 6$ 5/A+,+>4$ =3-0+[$ ,$ +,$ 34$ 09� $=3-0+[($ +4$ H*+A*$ /./=/4-$ <%E$ +45+A3-/,$ -*/$ </0?>0=34A/$ >?$3.-/043-+D/$*%:+/D3.73-/5$383+4,-$-*/$5/A+,+>4$A0+-/0+>4$-E>$Z-$ +,$>?-/4$3,,7=/5$-*3-$ -*/$5/A+,+>4$=3M/0$*3,$5/-/0=+4/5$-*/$H/+8*-,$>?$0/.3-+D/$+=<>0-34A/$>?$-*/$5/A+,+>4$A0+-/0+3($ I((G

; 0;;N �� $Nc+==/0=344($

;VV9O2$L*/$->-3.$,A>0/$?>0$/3A*$3.-/043-+D/$+,$>K-3+4/5$KE$-*/$?>..>H+48$?>0=7.3a$

%EEE% <;O �� $

)*/4$-*/$>D/03..$,A>0/,$30/$A3.A7.3-/5$?>0$3..$-*/$3.-/043-+D/,($-*/$>4/$H+-*$-*/$*+8*/,-$,A>0/$+,$A*>,/42$$

$

#70+48$ -*/$ <3,-$ -*0//$ 5/A35/,($ 3.>48$ H+-*$ -*/$ 03<+5$ 5/D/.><=/4-$ >?$ +4?>0=3-+>4$ -/A*4>.>8+/,($

A>4,+5/03K./$<0>80/,,$*3,$K//4$=35/$+4$=7.-+<./$A0+-/0+3$5/A+,+>4$343.E,+,X-*/$=>5/.+48$345$,>.D+48$>?$

!"#!$ <0>K./=,($ 345$ =34E$!"#!$=/-*>5,$ *3D/$ K//4$ 5/D/.></5$ N,//$ SH348$ T$ U>>4($ ;VW;$ 345$

@-/H305($;VV%$?>0$3$,70D/EO2$S>H/D/0($,>=/$>?$-*/$/[+,-+48$=/-*>5,$*3D/$K//4$A0+-+A+Q/5$3,$35$*>A$345($

->$A/0-3+4$5/80//($74^7,-+?+/5$>4$-*/>0/-+A3.$345d>0$/=<+0+A3.$80>745,2$L*/,/$=/-*>5,$A34$K/$5+D+5/5$+4->$

-*0//$A3-/8>0+/,a$=7.-+<./$3--0+K7-/$7-+.+-E$-*/>0E$N!6CLO($>7-034M+48$=/-*>5,($345$+4-/03A-+D/$=/-*>5,2$

L*/$ ?+0,-$ A3-/8>0E($ !6CL($ A>4,+,-,$ >?$ =/-*>5,$ -*3-$ 3880/83-/$ 5+??/0/4-$ <>+4-,$ >?$ D+/H$ +4->$ 3$ 7-+.+-E$

?74A-+>4($ H*+A*$ =7,-$ ,7K,/Y7/4-.E$ K/$ ><-+=+Q/52$ )*/4$ 3--/=<-+48$ ->$ ><-+=+Q/$ -*/$ 74+?+/5$ 7-+.+-E$

?74A-+>4($>?-/4$3$,-0+A-$>05/0$>D/0$-*/$,/-$>?$3..$<>,,+K./$5/A+,+>4,$+,$5/?+4/5$K3,/5$>4$,>=/$3,,7=<-+>4,2$

L*/$<0>K./=$?>0$=/-*>5,$+4$-*+,$A3-/8>0E$+,$-*3-$+4-/05/</45/4A/$>?$3--0+K7-/,$+,$4>-$=>5/./52$@+4A/$3..$

Page 8: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

CooperaQve   Inter-­‐Domain   Traffic   Engineering   using   Nash  Bargaining  and  DecomposiQon  SoluQon  criteria:  1.  Minimum  informaQon  to  be  exchange  between  ASes.  2.  The  efficiency  is  based  on  the  cooperaQon  approach  on  a  Pareto  basis.  3.  Fairness  is  the  main  approach  for  the  best  soluQon.  4.  Based  on  a  50/50  usage  criteria  

assume that ISPs can improve their local performance bybargaining (or negotiating) about the traffic flow distributionon their peering links. Our first insight is that we can usethe well-known concept of Nash bargaining [10],[11] to doso. Under this scheme, the ISPs agree to jointly optimize asocial cost function, known as the Nash product, which isessentially the product of the utility functions of the twoISPs. The key advantage of using this approach is thatthe solution is guaranteed to provide Pareto efficiency andfairness. When the ISPs’ utilities (measures of performance,such as average delay or maximum load on a link) aredirectly comparable, this solution is min-max fair, i.e.,the gains from cooperation are equal. However, when theutilities are not comparable, it still provides a Pareto efficientsolution that is proportionally fair [12]. By this, we meanthat the gains from cooperation to individual ISPs are equalafter some (automatic) suitable scaling of the utilities. Thisscaling is endogenous to the solution and, therefore, ishighly desirable.

This leaves us with the issue of not revealing criticalinternal information. Our insight here is that we can use dualdecomposition [13] to transform the joint optimization of theNash product into a procedure with precisely this property,as follows. The global optimization problem can be decom-posed into two independent sub-problems by recognizingthe coupling flows (these are the flows crossing between ISPdomains) and introducing appropriate Lagrange multipliers(or shadow prices) [13]. These sub-problems can now beassigned to the ISPs to be solved in a decentralized manner.These have the critical feature that they are completely local– an ISP’s assigned sub-problem depends only on its ownnetwork – thus, the ISPs don’t have to share critical internalinformation. Relying on this insight, we develop an iterativeprocedure based on the sub-gradient method [14] where,given Lagrange multipliers, the ISPs independently optimizetheir local sub-problems to come up with their requiredcoupling flows. The Lagrange multipliers are then updatedusing the sub-gradient method, which uses the difference inthe two sets of required flows to determine the magnitudeof the update. The update can be done in a decentralizedmanner. After the update, the ISPs again try to optimize theirlocal sub-problems. We show that this process converges infinite time to a fair and Pareto-efficient allocation.

We evaluate the effectiveness of our approach using sim-ulations over real ISP topologies. Through simulations, weshow that our approach significantly out-performs unilateralapproaches such as the commonly used hot potato or shortestpath routing (where ISPs route to nearest peering locationin terms of link weights) as well as the Nash equilibriumsetting (where ISPs optimize local objectives while playingbest responses to each other [9]). For the case where ISPsemploy similar utilities, we compare our solution againstcentralized optimal routing (where a central arbitrator op-timizes the common objective across both ISP networks).We also confirm the proportional fairness guarantees of oursolution via simulation experiments.

ISP2ISP1

yd21

yd12

sd21 − y

d21

sd12 − y

d12

Fig. 1. The Model

II. INTER-DOMAIN TRAFFIC ENGINEERING USING NASHBARGAINING AND DECOMPOSITION

A. The ModelWe model the interaction between two ISPs: ISP1 and

ISP2 as shown in Figure 1. These ISPs are optimizingutilities u1 and u2, respectively. As mentioned in Section I,these utilities are related to some measure of performance.These utilities could mean different things to the ISPs. Forexample, for one ISP the utility could be related to theaverage delay in the network, and for the other ISP theutility could be related to the maximum load on a linkin the network. The ISPs optimize these utilities under theflow conservation constraints, i.e., flows from all sources toall destinations must be routed. To simplify exposition, inthe following description, we assume that the ISPs employMPLS-like routing. We believe the approach we describebelow can be easily modified to yield a mechanism forsetting link weights for ISPs using OSPF in a way similarto [15].We make the common assumption that the utilities are

either convex or concave functions, and that the ISPs arerespectively minimizing or maximizing these utilities. Forexample, some convex utilities are the maximum load on alink and the average delay using convex link per unit delayfunctions (e.g., the per unit delay in an M/M/1 queue).We assume that ISP1 needs to send flows sd

12 to ISP2 ona per destination basis: sd

12 is a vector of flows to be sent toeach of the destinations in ISP2 from ISP1.2 Similarly, ISP2

needs to send flows sd21 to ISP1 on a per destination basis:

sd21 is a vector of flows to be sent to each of the destinationsin ISP1 from ISP2. Even though the ISPs may have multiplepeering links, to facilitate easier understanding, we explainour model using two bi-directional peering links (Figure 1).The model generalizes readily to multiple peering links. Weassume that ISP1 splits sd

12 so that yd12 goes on the upper

link and (sd12 − yd

12) goes on the lower link. Similarly ISP2

splits sd21 so that yd

21 goes on the upper link and (sd21 − yd

21)goes on the lower link.Optimizing for the utilities would be a no-brainer if the

ISPs had no interaction. Then they could optimize theirrespective utilities independently. What makes it compli-cated is the interaction of the ISPs through flows sent be-tween each other. These flows make the ISPs utilities inter-2We make the notation precise in Section II-C.1.

2

u2

u1

Pareto frontier

breakdown point

feasibleregion

Fig. 2. The feasible region with Pareto efficient frontier.

dependent. These flows between the ISPs are sometimesreferred to as coupling flows since they cause the ISPsoptimization problems to be coupled.If the ISPs are myopic, i.e., they employ unilateral ap-

proaches towards inter-domain TE, they would optimizewithout paying attention to how the coupling flows affectthe other ISPs optimization problem. For example, yd

12 isan output of ISP1’s optimization problem and is thus underits control. However, yd

12 is an input to ISP2’s optimizationproblem and thus affects its outcome. Now, if ISP1 ismyopic, it will optimize without paying attention to howmuch it may be hurting ISP2 by determining yd

12 myopically.Similarly, ISP2 could determine yd

21 myopically. Thus, inthis process, both ISPs may end up hurting each other.When both ISPs route myopically, we denote the ISPs

utilities as umyopic1 and umyopic

2 , respectively. The questionthen arises is: Is there any way that the ISPs could somehowcooperate on determining the coupling flows and improvetheir performance, i.e., achieve u1 ≥ umyopic

1 and u2 ≥umyopic

2 such that1) The gains from cooperation are equitable (or fair)while operating at a Pareto efficient operation point?

2) ISPs don’t have to divulge any critical informationabout their networks?

The answer to the first question lies in the idea of NashBargaining [11]. The answer to the second question lies inthe idea of decomposition [13]. We explain both of theseideas next.

B. Nash BargainingThe basic idea behind Nash bargaining is as follows. We

assume that the ISP utilities are inter-dependent, concaveand cardinal, where by cardinal we mean that the actualvalues of utilities matter – as opposed to ordinal utilitieswhere only the relative ordering of outcomes matters. Figure2 shows the feasible region for the two utilities, where thefeasible region is defined as the region where both ISPswould do better off compared to the myopic outcome. Themyopic outcome is also referred to as the breakdown point.A fair and Pareto efficient outcome, also referred to

as the Nash solution can be obtained by maximizing theNash product given by u1u2. Using the axiomatic theory

of cooperative games, it can be shown that when twoplayers (ISPs for us) with equal market power bargain, usingthreat strategies, they should arrive at the Nash solution.Referring to Figure 2, these threat strategies correspond tothe breakdown point, which is the outcome if the ISPs areunable to reach an agreement.In what follows, we provide a brief summary of the prop-

erties of the Nash solution, using the axiomatic bargainingapproach. The idea here is that a good bargaining solutionshould satisfy the following four axioms, which we simplystate as follows (see [11] for a detailed discussion):Pareto efficiency. This is obviously desirable since ISPsprefer more to less.Symmetry. This says that the solution should provideequal gains from cooperation when the feasible region issymmetric, where by symmetric we mean that the feasibleregion is agnostic of the player’s identities and that it wouldlook the same even if the ISPs utility axis were swapped.Independence of affine transformations. This requires thatthe solution should be agnostic of any affine transformations(that is, shifts and scalings) applied to any of the twoutilities. So, if the solution is given by (uNB

1 , uNB2 ) for some

utilities (u1, u2), and u1 is scaled and shifted to α1u1 + β1,then the solution should change to (α1uNB

1 + β1, uNB2 ).

Independence of irrelevant alternatives. This basicallysays that addition of irrelevant alternatives should not changethe solution. That is, for feasible regions F and G, if(uNB

1 , uNB2 ) ∈ solution(F ), G ⊂ F , and (uNB

1 , uNB2 ) ∈ G

then (uNB1 , uNB

2 ) ∈ solution(G).It turns out that the Nash solution is the only solution that

satisfies these four axioms [10]. In fact, the Nash solution isthe only solution that satisfies the following problem that issimultaneously utilitarian (Pareto efficient) and egalitarian(fair) [11]. That is, the Nash solution solves

maximize α1u1 + α2u2

subject to α1u1 = α2u2

(u1, u2) ∈ U

for some α1 ≥ 0 and α2 ≥ 0, where the optimization isover the bounded set U . Note that this scaling by α’s doesnot change the Pareto efficient frontier in Figure 2, i.e., thevalues of the choice variables resulting in Pareto efficientpoints remain the same. These α’s bring the usually un-comparable u1 and u2 on a common footing so that we cantalk about fairness in the first place. In particular, the Nashsolution is proportionally fair [12]. This means that movingaway from the Nash solution causes a negative cumulativepercentage change in utilities. That is, if (uNB

1 , uNB2 ) is the

Nash solution, and we move to another point (u∗1, u

∗2), then

2!

i=1

(u∗i − uNB

i )

uNBi

≤ 0 (1)

We next describe how decomposition can be used tojointly optimize the Nash product without revealing anysensitive information.

C. DecompositionThe idea of decomposition is not new. It has been success-

fully used to solve large scale optimization problems [13]

3

U1 e U2 are the optimization functions to ISP1 e ISP2

Nash Solution: Max (U1U2)

Decomposition Max {ln(u1) + ln(u2)}

and to solve separable problems in a decentralized manner.Moreover, in our case, decomposition allows separate enti-ties in the optimization problem to hide their internal criticalinformation. In what follows, we first develop a preciseoptimization framework, and then use this framework toexplain decomposition.1) Optimization Formulation: We denote the network

topology of ISPi, i ∈ {1, 2}, by a directed graph Gi =(Ni,Li) with ni = |Ni| nodes and li = |Li| internal links.We also denote by P the set of p directed peering links.We then define the incidence matrix for ISPi as matrixAi ∈ Rni×(li+p), with Ai,jk = +1 if link k leaves nodej, Ai,jk = −1 if link k enters node j, and 0 otherwise.We consider aggregate data flows through the network,

where we identify each flow by its destination node. Wedenote by D the set of all possible destination nodes. ForISPi, we denote the nonnegative amount of flow originatingat node j and destined to node d ∈ D by sd

i,j (j = d).When j = d, sd

i,d is the negative sum of the flows destinedto the node d, thus ensuring flow conservation. We refer tosd

i ∈ Rn as the source-sink vector. Note that sdi , i ∈ {1, 2}

include sd12 and sd

21, as described in Section II-A. Similarly,for ISPi, we denote the amount of nonnegative flow destinedto node d on each internal link k ∈ Li by xd

i,k . We callxd

i ∈ Rl the internal flow vector for destination d. Finally,we denote the amount of nonnegative flow destined to noded on each peering link k ∈ P by yd

k. We call yd ∈ Rp thepeering flow vector for destination d. This yd includes yd

12and yd

21, as described in Section II-A.Now, we are ready to define the optimization problems

in various scenarios. We first present the Nash productproblem, where ISP would jointly solve

maximize u1u2

subject to A1

!

xd1

yd

"

= sd1, A2

!

xd2

yd

"

= sd2

xd1 ≥ 0, xd

2 ≥ 0, yd ≥ 0,

(2)

for all d ∈ D, where the optimization variables are xd1,

xd2, and yd. Here the two equality constraints are the flowconservation constraints for ISP1 and ISP2, respectively, andthe last set of inequality constraints ensures that the choicevariables are non-negative.3A related problem to (2) is when both ISPs route myopi-

cally. The myopic routing schemes that we are particularlyinterested in are:1. Hot potato routing: Under this approach, each ISP routesinter-domain traffic originating in its network to the closestpeering point (i.e., least OSPF-cost). In a way, this attemptsto minimize the network resources consumed by inter-domain traffic within the source ISP network. This formof inter-domain traffic exchange is commonly used today.2. Nash equilibrium: Under this approach, ISPs myopicallyoptimize local objectives while iteratively playing best re-sponse to each other. Each ISP finds the optimal way to splitinter-domain traffic across peering links, given the trafficsplits of its neighbor, until no better traffic split can be

3Other constraints such as link capacity constraints can be readilyincluded.

found. This dynamic eventually finds an equilibrium, alsoknown as the Nash equilibrium [9], from which no ISP hasan incentive to deviate.Under these routing schemes, each ISP myopically solves

an optimization problem

maximize ui

subject to Ai

!

xdi

yd

"

= sdi

xdi ≥ 0, yd ≥ 0,

(3)

for i = {1, 2} and for all d ∈ D. Here we use sdi instead

of sdi to represent the fact that the myopic routing strategies

change the original flow vectors. For example, in hot-potatorouting, since each ISP routes the inter-domain flows to thenearest exit points, the flow vector reflects the source offlows being on peering points instead of being on internalnodes. In Nash equilibrium, where the ISPs iterate over themyopic problems based on current incoming flows, the flowvector reflects a similar transformation.2) Decomposition: Now we look into how we can cast

problem (2) in separable form, allowing for a decentralizedsolution. We face two challenges: first, as it stands, theobjective is not separable, and second, the ISPs utilities arecoupled through yd.We first transform problem (2) into an equivalent problem

by taking the logarithm of the objective function. Sincethe logarithm is an increasing and concave function, thistransformation does not change the solution to the originalproblem [14]. We then get the following equivalent problem

maximize ln(u1) + ln(u2)

subject to A1

!

xd1

yd

"

= sd1, A2

!

xd2

yd

"

= sd2

xd1 ≥ 0, xd

2 ≥ 0, yd ≥ 0,

(4)

We next introduce new nonnegative variables yd1 and

yd2 , which are local versions of yd for ISP1 and ISP2,respectively. Problem (4) can then be rewritten as

maximize ln(u1) + ln(u2)

subject to A1

!

xd1

yd1

"

= sd1, A2

!

xd2

yd2

"

= sd2

xd1 ≥ 0, yd

1 ≥ 0, xd2 ≥ 0, yd

2 ≥ 0

yd1 = yd

2 .

(5)

We still have a coupling constraint yd1 = yd

2 , which we dealwith using dual (or pricing) decomposition [13], which isoutlined next.We first write the partial Lagrangian of problem (5), with

respect to the coupling constraint, as

L(xd1, y

d1 , xd

2, yd2 , λd) = ln(u1)+ln(u2)+

#

d

(λd)T (yd1−yd

2),

where λd ∈ Rp are Lagrange multipliers associated with thecoupling constraint. This is a separable function in (xd

1, yd1)

and (xd2, y

d2). We now solve the dual problem of problem (5),

given byminimize g1(λ

d) + g2(λd), (6)

4

Page 9: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

CooperaQve  interdomain  traffic  engineering  using  Nash  negoQaQon  and  dual  decomposiQon  SoluQon  criteria:  1.  Minimum  informaQon  to  be  exchange  between  ASes.  2.  The  efficiency  is  based  on  the  cooperaQon  approach  on  a  Pareto  basis.  3.  Fairness  is  the  main  approach  for  the  best  soluQon.  4.  Based  on  a  50/50  usage  criteria  

assume that ISPs can improve their local performance bybargaining (or negotiating) about the traffic flow distributionon their peering links. Our first insight is that we can usethe well-known concept of Nash bargaining [10],[11] to doso. Under this scheme, the ISPs agree to jointly optimize asocial cost function, known as the Nash product, which isessentially the product of the utility functions of the twoISPs. The key advantage of using this approach is thatthe solution is guaranteed to provide Pareto efficiency andfairness. When the ISPs’ utilities (measures of performance,such as average delay or maximum load on a link) aredirectly comparable, this solution is min-max fair, i.e.,the gains from cooperation are equal. However, when theutilities are not comparable, it still provides a Pareto efficientsolution that is proportionally fair [12]. By this, we meanthat the gains from cooperation to individual ISPs are equalafter some (automatic) suitable scaling of the utilities. Thisscaling is endogenous to the solution and, therefore, ishighly desirable.

This leaves us with the issue of not revealing criticalinternal information. Our insight here is that we can use dualdecomposition [13] to transform the joint optimization of theNash product into a procedure with precisely this property,as follows. The global optimization problem can be decom-posed into two independent sub-problems by recognizingthe coupling flows (these are the flows crossing between ISPdomains) and introducing appropriate Lagrange multipliers(or shadow prices) [13]. These sub-problems can now beassigned to the ISPs to be solved in a decentralized manner.These have the critical feature that they are completely local– an ISP’s assigned sub-problem depends only on its ownnetwork – thus, the ISPs don’t have to share critical internalinformation. Relying on this insight, we develop an iterativeprocedure based on the sub-gradient method [14] where,given Lagrange multipliers, the ISPs independently optimizetheir local sub-problems to come up with their requiredcoupling flows. The Lagrange multipliers are then updatedusing the sub-gradient method, which uses the difference inthe two sets of required flows to determine the magnitudeof the update. The update can be done in a decentralizedmanner. After the update, the ISPs again try to optimize theirlocal sub-problems. We show that this process converges infinite time to a fair and Pareto-efficient allocation.

We evaluate the effectiveness of our approach using sim-ulations over real ISP topologies. Through simulations, weshow that our approach significantly out-performs unilateralapproaches such as the commonly used hot potato or shortestpath routing (where ISPs route to nearest peering locationin terms of link weights) as well as the Nash equilibriumsetting (where ISPs optimize local objectives while playingbest responses to each other [9]). For the case where ISPsemploy similar utilities, we compare our solution againstcentralized optimal routing (where a central arbitrator op-timizes the common objective across both ISP networks).We also confirm the proportional fairness guarantees of oursolution via simulation experiments.

ISP2ISP1

yd21

yd12

sd21 − y

d21

sd12 − y

d12

Fig. 1. The Model

II. INTER-DOMAIN TRAFFIC ENGINEERING USING NASHBARGAINING AND DECOMPOSITION

A. The ModelWe model the interaction between two ISPs: ISP1 and

ISP2 as shown in Figure 1. These ISPs are optimizingutilities u1 and u2, respectively. As mentioned in Section I,these utilities are related to some measure of performance.These utilities could mean different things to the ISPs. Forexample, for one ISP the utility could be related to theaverage delay in the network, and for the other ISP theutility could be related to the maximum load on a linkin the network. The ISPs optimize these utilities under theflow conservation constraints, i.e., flows from all sources toall destinations must be routed. To simplify exposition, inthe following description, we assume that the ISPs employMPLS-like routing. We believe the approach we describebelow can be easily modified to yield a mechanism forsetting link weights for ISPs using OSPF in a way similarto [15].We make the common assumption that the utilities are

either convex or concave functions, and that the ISPs arerespectively minimizing or maximizing these utilities. Forexample, some convex utilities are the maximum load on alink and the average delay using convex link per unit delayfunctions (e.g., the per unit delay in an M/M/1 queue).We assume that ISP1 needs to send flows sd

12 to ISP2 ona per destination basis: sd

12 is a vector of flows to be sent toeach of the destinations in ISP2 from ISP1.2 Similarly, ISP2

needs to send flows sd21 to ISP1 on a per destination basis:

sd21 is a vector of flows to be sent to each of the destinationsin ISP1 from ISP2. Even though the ISPs may have multiplepeering links, to facilitate easier understanding, we explainour model using two bi-directional peering links (Figure 1).The model generalizes readily to multiple peering links. Weassume that ISP1 splits sd

12 so that yd12 goes on the upper

link and (sd12 − yd

12) goes on the lower link. Similarly ISP2

splits sd21 so that yd

21 goes on the upper link and (sd21 − yd

21)goes on the lower link.Optimizing for the utilities would be a no-brainer if the

ISPs had no interaction. Then they could optimize theirrespective utilities independently. What makes it compli-cated is the interaction of the ISPs through flows sent be-tween each other. These flows make the ISPs utilities inter-2We make the notation precise in Section II-C.1.

2

u2

u1

Pareto frontier

breakdown point

feasibleregion

Fig. 2. The feasible region with Pareto efficient frontier.

dependent. These flows between the ISPs are sometimesreferred to as coupling flows since they cause the ISPsoptimization problems to be coupled.If the ISPs are myopic, i.e., they employ unilateral ap-

proaches towards inter-domain TE, they would optimizewithout paying attention to how the coupling flows affectthe other ISPs optimization problem. For example, yd

12 isan output of ISP1’s optimization problem and is thus underits control. However, yd

12 is an input to ISP2’s optimizationproblem and thus affects its outcome. Now, if ISP1 ismyopic, it will optimize without paying attention to howmuch it may be hurting ISP2 by determining yd

12 myopically.Similarly, ISP2 could determine yd

21 myopically. Thus, inthis process, both ISPs may end up hurting each other.When both ISPs route myopically, we denote the ISPs

utilities as umyopic1 and umyopic

2 , respectively. The questionthen arises is: Is there any way that the ISPs could somehowcooperate on determining the coupling flows and improvetheir performance, i.e., achieve u1 ≥ umyopic

1 and u2 ≥umyopic

2 such that1) The gains from cooperation are equitable (or fair)while operating at a Pareto efficient operation point?

2) ISPs don’t have to divulge any critical informationabout their networks?

The answer to the first question lies in the idea of NashBargaining [11]. The answer to the second question lies inthe idea of decomposition [13]. We explain both of theseideas next.

B. Nash BargainingThe basic idea behind Nash bargaining is as follows. We

assume that the ISP utilities are inter-dependent, concaveand cardinal, where by cardinal we mean that the actualvalues of utilities matter – as opposed to ordinal utilitieswhere only the relative ordering of outcomes matters. Figure2 shows the feasible region for the two utilities, where thefeasible region is defined as the region where both ISPswould do better off compared to the myopic outcome. Themyopic outcome is also referred to as the breakdown point.A fair and Pareto efficient outcome, also referred to

as the Nash solution can be obtained by maximizing theNash product given by u1u2. Using the axiomatic theory

of cooperative games, it can be shown that when twoplayers (ISPs for us) with equal market power bargain, usingthreat strategies, they should arrive at the Nash solution.Referring to Figure 2, these threat strategies correspond tothe breakdown point, which is the outcome if the ISPs areunable to reach an agreement.In what follows, we provide a brief summary of the prop-

erties of the Nash solution, using the axiomatic bargainingapproach. The idea here is that a good bargaining solutionshould satisfy the following four axioms, which we simplystate as follows (see [11] for a detailed discussion):Pareto efficiency. This is obviously desirable since ISPsprefer more to less.Symmetry. This says that the solution should provideequal gains from cooperation when the feasible region issymmetric, where by symmetric we mean that the feasibleregion is agnostic of the player’s identities and that it wouldlook the same even if the ISPs utility axis were swapped.Independence of affine transformations. This requires thatthe solution should be agnostic of any affine transformations(that is, shifts and scalings) applied to any of the twoutilities. So, if the solution is given by (uNB

1 , uNB2 ) for some

utilities (u1, u2), and u1 is scaled and shifted to α1u1 + β1,then the solution should change to (α1uNB

1 + β1, uNB2 ).

Independence of irrelevant alternatives. This basicallysays that addition of irrelevant alternatives should not changethe solution. That is, for feasible regions F and G, if(uNB

1 , uNB2 ) ∈ solution(F ), G ⊂ F , and (uNB

1 , uNB2 ) ∈ G

then (uNB1 , uNB

2 ) ∈ solution(G).It turns out that the Nash solution is the only solution that

satisfies these four axioms [10]. In fact, the Nash solution isthe only solution that satisfies the following problem that issimultaneously utilitarian (Pareto efficient) and egalitarian(fair) [11]. That is, the Nash solution solves

maximize α1u1 + α2u2

subject to α1u1 = α2u2

(u1, u2) ∈ U

for some α1 ≥ 0 and α2 ≥ 0, where the optimization isover the bounded set U . Note that this scaling by α’s doesnot change the Pareto efficient frontier in Figure 2, i.e., thevalues of the choice variables resulting in Pareto efficientpoints remain the same. These α’s bring the usually un-comparable u1 and u2 on a common footing so that we cantalk about fairness in the first place. In particular, the Nashsolution is proportionally fair [12]. This means that movingaway from the Nash solution causes a negative cumulativepercentage change in utilities. That is, if (uNB

1 , uNB2 ) is the

Nash solution, and we move to another point (u∗1, u

∗2), then

2!

i=1

(u∗i − uNB

i )

uNBi

≤ 0 (1)

We next describe how decomposition can be used tojointly optimize the Nash product without revealing anysensitive information.

C. DecompositionThe idea of decomposition is not new. It has been success-

fully used to solve large scale optimization problems [13]

3

U1 e U2 are the optimization functions to ISP1 e ISP2

Nash Solution: Max (U1U2)

Decomposition Max {ln(u1) + ln(u2)}

and to solve separable problems in a decentralized manner.Moreover, in our case, decomposition allows separate enti-ties in the optimization problem to hide their internal criticalinformation. In what follows, we first develop a preciseoptimization framework, and then use this framework toexplain decomposition.1) Optimization Formulation: We denote the network

topology of ISPi, i ∈ {1, 2}, by a directed graph Gi =(Ni,Li) with ni = |Ni| nodes and li = |Li| internal links.We also denote by P the set of p directed peering links.We then define the incidence matrix for ISPi as matrixAi ∈ Rni×(li+p), with Ai,jk = +1 if link k leaves nodej, Ai,jk = −1 if link k enters node j, and 0 otherwise.We consider aggregate data flows through the network,

where we identify each flow by its destination node. Wedenote by D the set of all possible destination nodes. ForISPi, we denote the nonnegative amount of flow originatingat node j and destined to node d ∈ D by sd

i,j (j = d).When j = d, sd

i,d is the negative sum of the flows destinedto the node d, thus ensuring flow conservation. We refer tosd

i ∈ Rn as the source-sink vector. Note that sdi , i ∈ {1, 2}

include sd12 and sd

21, as described in Section II-A. Similarly,for ISPi, we denote the amount of nonnegative flow destinedto node d on each internal link k ∈ Li by xd

i,k . We callxd

i ∈ Rl the internal flow vector for destination d. Finally,we denote the amount of nonnegative flow destined to noded on each peering link k ∈ P by yd

k. We call yd ∈ Rp thepeering flow vector for destination d. This yd includes yd

12and yd

21, as described in Section II-A.Now, we are ready to define the optimization problems

in various scenarios. We first present the Nash productproblem, where ISP would jointly solve

maximize u1u2

subject to A1

!

xd1

yd

"

= sd1, A2

!

xd2

yd

"

= sd2

xd1 ≥ 0, xd

2 ≥ 0, yd ≥ 0,

(2)

for all d ∈ D, where the optimization variables are xd1,

xd2, and yd. Here the two equality constraints are the flowconservation constraints for ISP1 and ISP2, respectively, andthe last set of inequality constraints ensures that the choicevariables are non-negative.3A related problem to (2) is when both ISPs route myopi-

cally. The myopic routing schemes that we are particularlyinterested in are:1. Hot potato routing: Under this approach, each ISP routesinter-domain traffic originating in its network to the closestpeering point (i.e., least OSPF-cost). In a way, this attemptsto minimize the network resources consumed by inter-domain traffic within the source ISP network. This formof inter-domain traffic exchange is commonly used today.2. Nash equilibrium: Under this approach, ISPs myopicallyoptimize local objectives while iteratively playing best re-sponse to each other. Each ISP finds the optimal way to splitinter-domain traffic across peering links, given the trafficsplits of its neighbor, until no better traffic split can be

3Other constraints such as link capacity constraints can be readilyincluded.

found. This dynamic eventually finds an equilibrium, alsoknown as the Nash equilibrium [9], from which no ISP hasan incentive to deviate.Under these routing schemes, each ISP myopically solves

an optimization problem

maximize ui

subject to Ai

!

xdi

yd

"

= sdi

xdi ≥ 0, yd ≥ 0,

(3)

for i = {1, 2} and for all d ∈ D. Here we use sdi instead

of sdi to represent the fact that the myopic routing strategies

change the original flow vectors. For example, in hot-potatorouting, since each ISP routes the inter-domain flows to thenearest exit points, the flow vector reflects the source offlows being on peering points instead of being on internalnodes. In Nash equilibrium, where the ISPs iterate over themyopic problems based on current incoming flows, the flowvector reflects a similar transformation.2) Decomposition: Now we look into how we can cast

problem (2) in separable form, allowing for a decentralizedsolution. We face two challenges: first, as it stands, theobjective is not separable, and second, the ISPs utilities arecoupled through yd.We first transform problem (2) into an equivalent problem

by taking the logarithm of the objective function. Sincethe logarithm is an increasing and concave function, thistransformation does not change the solution to the originalproblem [14]. We then get the following equivalent problem

maximize ln(u1) + ln(u2)

subject to A1

!

xd1

yd

"

= sd1, A2

!

xd2

yd

"

= sd2

xd1 ≥ 0, xd

2 ≥ 0, yd ≥ 0,

(4)

We next introduce new nonnegative variables yd1 and

yd2 , which are local versions of yd for ISP1 and ISP2,respectively. Problem (4) can then be rewritten as

maximize ln(u1) + ln(u2)

subject to A1

!

xd1

yd1

"

= sd1, A2

!

xd2

yd2

"

= sd2

xd1 ≥ 0, yd

1 ≥ 0, xd2 ≥ 0, yd

2 ≥ 0

yd1 = yd

2 .

(5)

We still have a coupling constraint yd1 = yd

2 , which we dealwith using dual (or pricing) decomposition [13], which isoutlined next.We first write the partial Lagrangian of problem (5), with

respect to the coupling constraint, as

L(xd1, y

d1 , xd

2, yd2 , λd) = ln(u1)+ln(u2)+

#

d

(λd)T (yd1−yd

2),

where λd ∈ Rp are Lagrange multipliers associated with thecoupling constraint. This is a separable function in (xd

1, yd1)

and (xd2, y

d2). We now solve the dual problem of problem (5),

given byminimize g1(λ

d) + g2(λd), (6)

4

But…  The  south  american  traffic  is  unbalanced!  

Page 10: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

 Some  definiCons  

Page 11: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Traffic  Asymmetry  

Path  asymmetry  –  ASYPATH  

S  

I1   I2  

D

Request  Response  

Load  asymmetry  –  ASYin-­‐out  

S  

I1   I2  

DRequest  

Response  

Page 12: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Long-­‐range  dependence  processes  •  Memory  process  •  Work  in  different  scales:  

•  The  incremental  fBm  -­‐  Fractal  Brownian  MoCon  –process  can  be  defined  as  

•  The  fBm  process  can  be  used  for  forecasQng.  •  Defined  with  few  parameters  (second  order)  •  The  Poisson  (Erlang  e.g.)   is  a  memory  less  process  and  is  not  

able  to  explain  the  Internet  traffic.  

Network Tra⇥c Modeling Using a Multifractal Wavelet Model 3

in the multifractal partition function T (q) , the burstiness as measured by themultifractal spectrum, the marginals, and the queueing behavior.

In this paper, we describe LRD and its relationship with fBm in section 2. Af-ter introducing the wavelet transform and describing the WIG model in section 3,we derive the MWM in section 4. Section 6 reports on the results of simulationexperiments with real data traces. We give an intuitive introduction to multifractalcascades in section 5 and close with conclusions in section 7.

2. fBm and LRD

Although we analyze fBm from a continuous-time point of view, for practicalcomputations and simulations, we often work with sampled continuous-time fBm.The increments process of sampled fBm

X[n] := B(n)�B(n� 1) (3)

defines a stationary Gaussian sequence known as discrete fractional Gaussian noise(fGn) with covariance behavior [10]

rX [k] ⇥ |k|2H�2, for |k| large . (4)

For 1/2 < H < 1, the covariance of fGn is strictly positive and decays soslowly that it is non-summable (i.e.,

⇤k rX [k] =⇤). This property is called LRD.

The LRD of fGn can be equivalently characterized in terms of how the ag-gregated processes

X(m)(n) :=1m

km⌅

i=(k�1)m+1

X(i) (5)

behave. It follows from (1) that X(n) fd= m1�HX(m)(n) .Hence, a log-log plot of the variance of X(m)(n) as a function of m —known

as a variance-time plot— will have a slope of 2H � 2. The variance-time plot cancharacterize LRD in non-Gaussian, non-zero-mean data as well [1].

3. Wavelets and LRD Processes

3.1. Wavelet transform

The discrete wavelet transform is a multi-scale signal representation of the form[11]

c(t) =⌅

k

uk 2�J0/2 ��2�J0t� k

⇥+

J0⌅

j=�⇥

k

wj,k 2�j/2 ⇥�2�jt� k

⇥, j, k ⌅ ZZ

Network Tra⇥c Modeling Using a Multifractal Wavelet Model 3

in the multifractal partition function T (q) , the burstiness as measured by themultifractal spectrum, the marginals, and the queueing behavior.

In this paper, we describe LRD and its relationship with fBm in section 2. Af-ter introducing the wavelet transform and describing the WIG model in section 3,we derive the MWM in section 4. Section 6 reports on the results of simulationexperiments with real data traces. We give an intuitive introduction to multifractalcascades in section 5 and close with conclusions in section 7.

2. fBm and LRD

Although we analyze fBm from a continuous-time point of view, for practicalcomputations and simulations, we often work with sampled continuous-time fBm.The increments process of sampled fBm

X[n] := B(n)�B(n� 1) (3)

defines a stationary Gaussian sequence known as discrete fractional Gaussian noise(fGn) with covariance behavior [10]

rX [k] ⇥ |k|2H�2, for |k| large . (4)

For 1/2 < H < 1, the covariance of fGn is strictly positive and decays soslowly that it is non-summable (i.e.,

⇤k rX [k] =⇤). This property is called LRD.

The LRD of fGn can be equivalently characterized in terms of how the ag-gregated processes

X(m)(n) :=1m

km⌅

i=(k�1)m+1

X(i) (5)

behave. It follows from (1) that X(n) fd= m1�HX(m)(n) .Hence, a log-log plot of the variance of X(m)(n) as a function of m —known

as a variance-time plot— will have a slope of 2H � 2. The variance-time plot cancharacterize LRD in non-Gaussian, non-zero-mean data as well [1].

3. Wavelets and LRD Processes

3.1. Wavelet transform

The discrete wavelet transform is a multi-scale signal representation of the form[11]

c(t) =⌅

k

uk 2�J0/2 ��2�J0t� k

⇥+

J0⌅

j=�⇥

k

wj,k 2�j/2 ⇥�2�jt� k

⇥, j, k ⌅ ZZ

Network Tra⇥c Modeling Using a Multifractal Wavelet Model 3

in the multifractal partition function T (q) , the burstiness as measured by themultifractal spectrum, the marginals, and the queueing behavior.

In this paper, we describe LRD and its relationship with fBm in section 2. Af-ter introducing the wavelet transform and describing the WIG model in section 3,we derive the MWM in section 4. Section 6 reports on the results of simulationexperiments with real data traces. We give an intuitive introduction to multifractalcascades in section 5 and close with conclusions in section 7.

2. fBm and LRD

Although we analyze fBm from a continuous-time point of view, for practicalcomputations and simulations, we often work with sampled continuous-time fBm.The increments process of sampled fBm

X[n] := B(n)�B(n� 1) (3)

defines a stationary Gaussian sequence known as discrete fractional Gaussian noise(fGn) with covariance behavior [10]

rX [k] ⇥ |k|2H�2, for |k| large . (4)

For 1/2 < H < 1, the covariance of fGn is strictly positive and decays soslowly that it is non-summable (i.e.,

⇤k rX [k] =⇤). This property is called LRD.

The LRD of fGn can be equivalently characterized in terms of how the ag-gregated processes

X(m)(n) :=1m

km⌅

i=(k�1)m+1

X(i) (5)

behave. It follows from (1) that X(n) fd= m1�HX(m)(n) .Hence, a log-log plot of the variance of X(m)(n) as a function of m —known

as a variance-time plot— will have a slope of 2H � 2. The variance-time plot cancharacterize LRD in non-Gaussian, non-zero-mean data as well [1].

3. Wavelets and LRD Processes

3.1. Wavelet transform

The discrete wavelet transform is a multi-scale signal representation of the form[11]

c(t) =⌅

k

uk 2�J0/2 ��2�J0t� k

⇥+

J0⌅

j=�⇥

k

wj,k 2�j/2 ⇥�2�jt� k

⇥, j, k ⌅ ZZ

Page 13: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

MulQfractal  Spectrum                              (Parameters)  fBm Fraclab H=0,7   Holder = Hurst em x0  

fBm f periodic   Holder  

fBm f log   Holder  

Page 14: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Parameter:    h(t)  =  (0,1  +  0,8t)  v  4096  samples  

H(t)  -­‐  Hölder  

Synthesized  

Spectrum  

SyntheQc  traffic    

Page 15: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Internet  Traffic:  CharacterizaQon  

         MulQfractal  

Ingress  

Egress  

Page 16: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

 USàVenezuela/Colombia  -­‐  CharacterizaQon  

Tráfego  Original  

Espectro  de  Legendre   Função  Hölder  Interpolada  

T1   T2   T3  

Page 17: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

South  America  

Some  issues    

Page 18: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Internet  Traffic:  The  real  issue    

0"

200"

400"

600"

800"

1000"

1200"01/13/2012"00:00:00"

01/13/2012"04:00:00"

01/13/2012"08:00:00"

01/13/2012"13:00:00"

01/13/2012"17:00:00"

01/13/2012"21:00:00"

01/14/2012"01:00:00"

01/14/2012"05:00:00"

01/14/2012"09:00:00"

01/14/2012"13:00:00"

01/14/2012"17:00:00"

01/14/2012"21:00:00"

01/15/2012"01:00:00"

01/15/2012"05:00:00"

01/15/2012"09:00:00"

01/15/2012"13:00:00"

01/15/2012"17:00:00"

01/15/2012"21:00:00"

01/16/2012"01:00:00"

01/16/2012"05:00:00"

01/16/2012"09:00:00"

01/16/2012"13:00:00"

01/16/2012"17:00:00"

01/16/2012"21:00:00"

01/17/2012"01:00:00"

01/17/2012"05:00:00"

01/17/2012"09:00:00"

01/17/2012"13:00:00"

01/17/2012"17:00:00"

01/17/2012"21:00:00"

01/18/2012"01:00:00"

01/18/2012"05:00:00"

01/18/2012"09:00:00"

01/18/2012"13:00:00"

01/18/2012"17:00:00"

01/18/2012"21:00:00"

01/19/2012"01:00:00"

01/19/2012"05:00:00"

01/19/2012"09:00:00"

01/19/2012"13:00:00"

01/19/2012"17:00:00"

01/19/2012"21:00:00"

01/20/2012"01:00:00"

01/20/2012"05:00:00"

01/20/2012"09:00:00"

01/20/2012"13:00:00"

01/20/2012"17:00:00"

01/20/2012"21:00:00"

Ingress"to"Venezuela/Colombia" Egress"from"Venezuela/Colombia"

Colombia  and  Venezuela  80%  is  Inbound:  Asymmetric  –  ASYin-­‐out    

Source:  N.D.A.  

Page 19: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Internet  Traffic:  The  real  issue    

Brazil  60ì70%  is  Inbound:  Asymmetric  ASYin-­‐out    

0"

200"

400"

600"

800"

1000"

1200"

01/13/2012"00:00:00"

01/13/2012"04:00:00"

01/13/2012"08:00:00"

01/13/2012"13:00:00"

01/13/2012"17:00:00"

01/13/2012"21:00:00"

01/14/2012"01:00:00"

01/14/2012"05:00:00"

01/14/2012"09:00:00"

01/14/2012"13:00:00"

01/14/2012"17:00:00"

01/14/2012"21:00:00"

01/15/2012"01:00:00"

01/15/2012"05:00:00"

01/15/2012"09:00:00"

01/15/2012"13:00:00"

01/15/2012"17:00:00"

01/15/2012"21:00:00"

01/16/2012"01:00:00"

01/16/2012"05:00:00"

01/16/2012"09:00:00"

01/16/2012"13:00:00"

01/16/2012"17:00:00"

01/16/2012"21:00:00"

01/17/2012"01:00:00"

01/17/2012"05:00:00"

01/17/2012"09:00:00"

01/17/2012"13:00:00"

01/17/2012"17:00:00"

01/17/2012"21:00:00"

01/18/2012"01:00:00"

01/18/2012"05:00:00"

01/18/2012"09:00:00"

01/18/2012"13:00:00"

01/18/2012"17:00:00"

01/18/2012"21:00:00"

01/19/2012"01:00:00"

01/19/2012"05:00:00"

01/19/2012"09:00:00"

01/19/2012"13:00:00"

01/19/2012"17:00:00"

01/19/2012"21:00:00"

01/20/2012"01:00:00"

01/20/2012"05:00:00"

01/20/2012"09:00:00"

01/20/2012"13:00:00"

01/20/2012"17:00:00"

01/20/2012"21:00:00"

Ingress"to"Brazil" Egress"from"Brazil"

Source:  N.D.A.  

Page 20: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Planning  &  traffic  engineering  soluQons  

Because:  The  South  America  problems  are  unique.    Obviously:  The   decision-­‐making   must   be   supported   by   local   data.   The  specific  math  models  must  match  the  real  network  behavior.  

Page 21: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Best  decision-­‐making    

Are  you  really  sure  about  it?    

Page 22: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

AS  1st  rule:  no  external  control  

“This   is   especially   true   for   the   inbound   direc+on   as   the   flow   of  traffic  depends  on  the  forwarding  decision  made  at  other  routers  in  other  Autonomous  Systems.  In  other  words,  in  order  to  engineer  the  way   traffic   enters   a   network,   the   route   selec+on   process   at  other  routers  has  to  be  influenced  remotely.”    Explicitly  Accommoda7ng  Origin  Preference  for  Inter-­‐Domain  Traffic  Engineering  Rolf  Winter,  ACM  Symposium  on  Applied  CompuQng,  2012    

Page 23: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Improving  the  prepend  decision-­‐making  Process  Start  

Network  CharacterizaQon  

Internet  full  rouQng  

Prepend  CharacterizaQon  

Internet  full  rouQng  

Success  Probability  to  use  Prepend  

Over  50%  

Resource  ForecasQng  

BGP  ModificaQon    2nd  Plane  opQon  or  Man2Man  negoQaQon  

Traffic  CharacterizaQon  

A

A

UQlizaQon  ForecasQng  

Best  Path  SelecQon  

Network  Change  TentaQve  

Monitoring  results  

End  

y  

n  

Stable  ?  

y  

n  Wait  

Next  Window  

Stability  Window  

Page 24: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

1st  –  Can  I  use  the  prepend  as  a  T.E.  tool?  

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99

1

0 5 10 15 20 25 30 35

CD

F

Difference in prepended ASN num. for same prefix

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99

1

0 5 10 15 20 25 30 35

CD

F

Difference in prepended ASN num. for same prefix

routing tablerouting updates

Fig. 8. Difference in number of prependedASNs for the same prefix

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

Fraction of routes with prepending paths per prefix

Fig. 9. Fraction of routes with prependingAS paths for the same prefix

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

Fraction of monitors select routes with ASPP

Fig. 10. Fraction of monitors adopting routeswith ASPP.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

first hop distribution among all routes

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

first hop distribution among all routes

prefixorigin AS

Fig. 11. First hop distribution among neighbors

the prepending AS paths are likely to be longer than otherpaths, thus less preferable. However, there are 20% of theroutes are still adopted by most of the monitors. With furtherinvestigation, we found that it’s the origin AS that prependsits ASNs multiple times consistently for announcements to allits neighbors. Therefore, even the prepending increases the ASpath length, it is an equal inflation for all monitors.

ASPP can be used to balance the traffic among all neighborsor to provision a backup route. We categorize routes from allmonitors for the same prefix at any time according to their firsthop ASes. We then calculate the fraction of routes associatedwith each first hop. The distribution of such fraction numbersare shown in Figure 11. Surprisingly, we see that 90% casesthere is only one dominant route (the fraction is 1) for eachprefix. But examining from all prefixes of same origin AS,we observe a much smaller fraction. It suggests that operatorscommonly use ASPP to make best routes go through differentneighbors for different prefixes.

C. Dynamic characterization

15000

20000

25000

30000

35000

40000

45000

1/066/06 1/076/07 1/086/08 1/096/09 1/106/101/11

Num

ber o

f pre

fixes

with

ASP

P (K

)

months

Fig. 12. Trends of routes with ASPP in the default-free routing table

0.5

0.6

0.7

0.8

0.9

1

1/06 1/07 1/08 1/09 1/10 1/11

Avg.

frac

tion

of ro

utes

per

nei

ghbo

rmonths

Fig. 13. Trends of route distribution amongst neighbors over time

We next demonstrate how the prepending AS path involvesover time. Figure 12 shows the number of prefixes withany prepending AS paths from January 2006 to March 2011from RouteView. It presents significant increases, suggestingthat ASPP as a traffic engineering method has been morecommonly used. We next depict the trends of the impactof ASPP. Figure 13 shows the average fraction per monthover five years. The trend is very stable, i.e., 70%-80%,suggesting that each prefix often has a primary carrier amongstall neighbors.

VI. VALIDATION

In the following, we first validate the accuracy of ourprediction on route selection. Then we evaluate the algorithms’effectiveness from three aspects, i.e., load balancing, backuproute provisioning and bypassing a particular AS. The topol-ogy is extracted from multiple BGP routing tables and thetopology reported by CAIDA [17].

A. Validation of route predictionSince we cannot directly modify the prepending behavior on

the Internet, we evaluate the simulation accuracy by comparingthe predicted routes with the adopted routes on the Internet.One of the key steps in our proposal is the prediction of routesseen and chosen by each AS in Figure 2. We validate itsaccuracy using observed best routes from RouteView. Fromone month of RouteView table and update files, we extractall the prefixes which is announced to multiple neighborsby the origin AS padding its ASN different times. We plugthe observed padding numbers for each neighbor into thealgorithm in Figure 2 to compute the best route observed oneach AS. For each prefix and padding vector, we compute thefraction of ASes where the predicted route matches with the

66

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99

1

0 5 10 15 20 25 30 35

CD

F

Difference in prepended ASN num. for same prefix

0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99

1

0 5 10 15 20 25 30 35

CD

F

Difference in prepended ASN num. for same prefix

routing tablerouting updates

Fig. 8. Difference in number of prependedASNs for the same prefix

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

Fraction of routes with prepending paths per prefix

Fig. 9. Fraction of routes with prependingAS paths for the same prefix

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

Fraction of monitors select routes with ASPP

Fig. 10. Fraction of monitors adopting routeswith ASPP.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

first hop distribution among all routes

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CD

F

first hop distribution among all routes

prefixorigin AS

Fig. 11. First hop distribution among neighbors

the prepending AS paths are likely to be longer than otherpaths, thus less preferable. However, there are 20% of theroutes are still adopted by most of the monitors. With furtherinvestigation, we found that it’s the origin AS that prependsits ASNs multiple times consistently for announcements to allits neighbors. Therefore, even the prepending increases the ASpath length, it is an equal inflation for all monitors.

ASPP can be used to balance the traffic among all neighborsor to provision a backup route. We categorize routes from allmonitors for the same prefix at any time according to their firsthop ASes. We then calculate the fraction of routes associatedwith each first hop. The distribution of such fraction numbersare shown in Figure 11. Surprisingly, we see that 90% casesthere is only one dominant route (the fraction is 1) for eachprefix. But examining from all prefixes of same origin AS,we observe a much smaller fraction. It suggests that operatorscommonly use ASPP to make best routes go through differentneighbors for different prefixes.

C. Dynamic characterization

15000

20000

25000

30000

35000

40000

45000

1/066/06 1/076/07 1/086/08 1/096/09 1/106/101/11

Num

ber o

f pre

fixes

with

ASP

P (K

)

months

Fig. 12. Trends of routes with ASPP in the default-free routing table

0.5

0.6

0.7

0.8

0.9

1

1/06 1/07 1/08 1/09 1/10 1/11

Avg.

frac

tion

of ro

utes

per

nei

ghbo

r

months

Fig. 13. Trends of route distribution amongst neighbors over time

We next demonstrate how the prepending AS path involvesover time. Figure 12 shows the number of prefixes withany prepending AS paths from January 2006 to March 2011from RouteView. It presents significant increases, suggestingthat ASPP as a traffic engineering method has been morecommonly used. We next depict the trends of the impactof ASPP. Figure 13 shows the average fraction per monthover five years. The trend is very stable, i.e., 70%-80%,suggesting that each prefix often has a primary carrier amongstall neighbors.

VI. VALIDATION

In the following, we first validate the accuracy of ourprediction on route selection. Then we evaluate the algorithms’effectiveness from three aspects, i.e., load balancing, backuproute provisioning and bypassing a particular AS. The topol-ogy is extracted from multiple BGP routing tables and thetopology reported by CAIDA [17].

A. Validation of route predictionSince we cannot directly modify the prepending behavior on

the Internet, we evaluate the simulation accuracy by comparingthe predicted routes with the adopted routes on the Internet.One of the key steps in our proposal is the prediction of routesseen and chosen by each AS in Figure 2. We validate itsaccuracy using observed best routes from RouteView. Fromone month of RouteView table and update files, we extractall the prefixes which is announced to multiple neighborsby the origin AS padding its ASN different times. We plugthe observed padding numbers for each neighbor into thealgorithm in Figure 2 to compute the best route observed oneach AS. For each prefix and padding vector, we compute thefraction of ASes where the predicted route matches with the

66To  acquire  the  needed  data  only    

To  have  the  needed  alternaQve  only    

To  establish  a  good  criteria  (minimum  price,  balancing,  …)    

Page 25: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Acquiring  data+  :Prepend  characterizaQon  

hwp://bgp.potaroo.net/as6447/bgptable.txt  

AS1   AS2   AS3   AS4   AS5   AS6   AS7   AS8  No  Prepend  

AS1   AS2   AS3   AS22   AS23  Prepend  

0  

500000000  

1E+09  

1,5E+09  

2E+09    Distância  0  

 Distância  1  

 Distância  2  

 Distância  3  

 Distância  4  

 Distância  5  

 Distância  6  

 Distância  7    Distância  8    Distância  9  

 Distância  10  

 Distância  11  

 Distância  12  

 Distância  13  

 Distância  14  

 Distância  15  

 Distância  16  

AS  distance  

Distância  Number  of  hops  

Page 26: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Improving  the  prepend  decision-­‐making  Process  Start  

Network  CharacterizaQon  

Internet  full  rouQng  

Prepend  CharacterizaQon  

Internet  full  rouQng  

Success  Probability  to  use  Prepend  

Over  50%  

Resource  ForecasQng  

BGP  ModificaQon    2nd  Plane  opQon  or  Man2Man  negoQaQon  

Traffic  CharacterizaQon  

A

A

UQlizaQon  ForecasQng  

Best  Path  SelecQon  

Network  Change  TentaQve  

Monitoring  results  

End  

y  

n  

Stable  ?  

y  

n  Wait  

Next  Window  

Stability  Window  

Page 27: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

BGP  Decision-­‐Making  parameters  

Atribute Priority

Local Preference Highest Priority

Shortest ASPATH

Traffic Engineering Tentative

Lowest MED

iBGP < eBGP

Lowest iGP Cost to BGP egress

Lowest Router ID Lowest Priority

III. THE BGP CHARACTERIZATION: 1ST STEP TO DECISION-MAKING PROCESS

The BGP Best Path Selection Process is defined with some external dependence, depending what kind of policy is used. According [8], [9] two different kind of policy can be found: external or internal, is shown that the Local attributes as shown in Fig. 4 have more priority than other attributes.

Atribute Priority

Local Preference Highest Priority

Shortest ASPATH

Traffic Engineering Tentative

Lowest MED

iBGP < eBGP

Lowest iGP Cost to BGP egress

Lowest Router ID Lowest Priority

Figure 4. BGP Condensed Priority Decision Making Preferences

In the same way the import routing policy (pin) or the (pout) can be defined as scalar filters applied on Graph on equation (1) as shown in [10] as a way to control the advertised routes inbound or outbound direction.

G=(V, E, B) (1)

The routing policy will define if a route will be announced or acquired in terms of table. The Gpol is the Graph after an inbound or outbound filter. The pin or pout can be written as a scalar.

Gpol=(V,E,B) * pin (G) for ingress (2)

Gpol=(V,E,B) * pout(G) for egress (3)

pin (G) assuming discrete values of “0” to block or “1” to accept a route set. pout(G) assuming discrete values of “0” to not advertise or “1” to advertise a route set.

This is a way to make a police controlled system with a possibility to insert filters. This makes an ingress traffic change by the destination a hard task, because AS_PATH attribute is the first external to be considered and is only the fourth in order of preference [11], [12]. This decision process can be divided in three different layers. The local preference is the highest priority parameter and internally controlled only. There are also the Traffic Engineering purpose parameters used as a tentative of traffic control by an external AS, these are always used when a someone wants to try to interfere in the internal decision making process. The lowest router ID is the lowest priority followed by iGP cost, iBGP learned, eBGP learned, MED, ASPATH Local Preference as the higher.

As described in [9],[12],[13] the hot potato routing can interfere in terms of routing convergence. By other side, if the

hot potato do not lead to new BGP update messages, some delay increment can occurs inside AS [4].

A. The Internet Radius Characterization. The Internet radius for decision-making process can be defined as the path count number. On the Bellmand criterion [13] a vertex ! ! ! lies on a shortest path between vertices !! ! ! !! !"!!"#!!"#$!!! !! ! ! !!! !! ! ! !! !! ! ! !Algorithm:

Get a full routing table T=dG(s,t)

For Subnet Sj

For PathSj={AS0, AS1; ASj-1; ASj}

Aggregate { PathSj}

Min {PathSj}

Count j

{Sj} ! ASk

rASk = j

end

end

Using this algorithm in weka project [weka] as the tool to find the AS Radius with a full table routing as shown in Fig.5

Fig.5. The Internet radius r. Each n ASn denotes the Internet radius using weka [14]

In Fig.6 Measured using internet full table (400k+ routes) information from [routeviews, potaroo] using weka [14] to filter. The (r + var) is shown in the ordinate axys variation

Fig.6. The Internet radius average and var.

!"

#"

$"

%"

&"

'"

("

)*+,-" ./$0(" ./1!#" ./1!#0" ./0'$" ./#$('&" ./%'&2" ./#'&(2"!"

#$%&#'()

%$*%+,#''

-%.*/0'

!12'

III. THE BGP CHARACTERIZATION: 1ST STEP TO DECISION-MAKING PROCESS

The BGP Best Path Selection Process is defined with some external dependence, depending what kind of policy is used. According [8], [9] two different kind of policy can be found: external or internal, is shown that the Local attributes as shown in Fig. 4 have more priority than other attributes.

Atribute Priority

Local Preference Highest Priority

Shortest ASPATH

Traffic Engineering Tentative

Lowest MED

iBGP < eBGP

Lowest iGP Cost to BGP egress

Lowest Router ID Lowest Priority

Figure 4. BGP Condensed Priority Decision Making Preferences

In the same way the import routing policy (pin) or the (pout) can be defined as scalar filters applied on Graph on equation (1) as shown in [10] as a way to control the advertised routes inbound or outbound direction.

G=(V, E, B) (1)

The routing policy will define if a route will be announced or acquired in terms of table. The Gpol is the Graph after an inbound or outbound filter. The pin or pout can be written as a scalar.

Gpol=(V,E,B) * pin (G) for ingress (2)

Gpol=(V,E,B) * pout(G) for egress (3)

pin (G) assuming discrete values of “0” to block or “1” to accept a route set. pout(G) assuming discrete values of “0” to not advertise or “1” to advertise a route set.

This is a way to make a police controlled system with a possibility to insert filters. This makes an ingress traffic change by the destination a hard task, because AS_PATH attribute is the first external to be considered and is only the fourth in order of preference [11], [12]. This decision process can be divided in three different layers. The local preference is the highest priority parameter and internally controlled only. There are also the Traffic Engineering purpose parameters used as a tentative of traffic control by an external AS, these are always used when a someone wants to try to interfere in the internal decision making process. The lowest router ID is the lowest priority followed by iGP cost, iBGP learned, eBGP learned, MED, ASPATH Local Preference as the higher.

As described in [9],[12],[13] the hot potato routing can interfere in terms of routing convergence. By other side, if the

hot potato do not lead to new BGP update messages, some delay increment can occurs inside AS [4].

A. The Internet Radius Characterization. The Internet radius for decision-making process can be defined as the path count number. On the Bellmand criterion [13] a vertex ! ! ! lies on a shortest path between vertices !! ! ! !! !"!!"#!!"#$!!! !! ! ! !!! !! ! ! !! !! ! ! !Algorithm:

Get a full routing table T=dG(s,t)

For Subnet Sj

For PathSj={AS0, AS1; ASj-1; ASj}

Aggregate { PathSj}

Min {PathSj}

Count j

{Sj} ! ASk

rASk = j

end

end

Using this algorithm in weka project [weka] as the tool to find the AS Radius with a full table routing as shown in Fig.5

Fig.5. The Internet radius r. Each n ASn denotes the Internet radius using weka [14]

In Fig.6 Measured using internet full table (400k+ routes) information from [routeviews, potaroo] using weka [14] to filter. The (r + var) is shown in the ordinate axys variation

Fig.6. The Internet radius average and var.

!"

#"

$"

%"

&"

'"

("

)*+,-" ./$0(" ./1!#" ./1!#0" ./0'$" ./#$('&" ./%'&2" ./#'&(2"

!"#$%&#'()

%$*%+,#''

-%.*/0'

!12'

Accept  or  not  ?  

Page 28: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

More  parameters:  The  Internet  Radius  

III. THE BGP CHARACTERIZATION: 1ST STEP TO DECISION-MAKING PROCESS

The BGP Best Path Selection Process is defined with some external dependence, depending what kind of policy is used. According [8], [9] two different kind of policy can be found: external or internal, is shown that the Local attributes as shown in Fig. 4 have more priority than other attributes.

Atribute Priority

Local Preference Highest Priority

Shortest ASPATH

Traffic Engineering Tentative

Lowest MED

iBGP < eBGP

Lowest iGP Cost to BGP egress

Lowest Router ID Lowest Priority

Figure 4. BGP Condensed Priority Decision Making Preferences

In the same way the import routing policy (pin) or the (pout) can be defined as scalar filters applied on Graph on equation (1) as shown in [10] as a way to control the advertised routes inbound or outbound direction.

G=(V, E, B) (1)

The routing policy will define if a route will be announced or acquired in terms of table. The Gpol is the Graph after an inbound or outbound filter. The pin or pout can be written as a scalar.

Gpol=(V,E,B) * pin (G) for ingress (2)

Gpol=(V,E,B) * pout(G) for egress (3)

pin (G) assuming discrete values of “0” to block or “1” to accept a route set. pout(G) assuming discrete values of “0” to not advertise or “1” to advertise a route set.

This is a way to make a police controlled system with a possibility to insert filters. This makes an ingress traffic change by the destination a hard task, because AS_PATH attribute is the first external to be considered and is only the fourth in order of preference [11], [12]. This decision process can be divided in three different layers. The local preference is the highest priority parameter and internally controlled only. There are also the Traffic Engineering purpose parameters used as a tentative of traffic control by an external AS, these are always used when a someone wants to try to interfere in the internal decision making process. The lowest router ID is the lowest priority followed by iGP cost, iBGP learned, eBGP learned, MED, ASPATH Local Preference as the higher.

As described in [9],[12],[13] the hot potato routing can interfere in terms of routing convergence. By other side, if the

hot potato do not lead to new BGP update messages, some delay increment can occurs inside AS [4].

A. The Internet Radius Characterization. The Internet radius for decision-making process can be defined as the path count number. On the Bellmand criterion [13] a vertex ! ! ! lies on a shortest path between vertices !! ! ! !! !"!!"#!!"#$!!! !! ! ! !!! !! ! ! !! !! ! ! !Algorithm:

Get a full routing table T=dG(s,t)

For Subnet Sj

For PathSj={AS0, AS1; ASj-1; ASj}

Aggregate { PathSj}

Min {PathSj}

Count j

{Sj} ! ASk

rASk = j

end

end

Using this algorithm in weka project [weka] as the tool to find the AS Radius with a full table routing as shown in Fig.5

Fig.5. The Internet radius r. Each n ASn denotes the Internet radius using weka [14]

In Fig.6 Measured using internet full table (400k+ routes) information from [routeviews, potaroo] using weka [14] to filter. The (r + var) is shown in the ordinate axys variation

Fig.6. The Internet radius average and var.

!"

#"

$"

%"

&"

'"

("

)*+,-" ./$0(" ./1!#" ./1!#0" ./0'$" ./#$('&" ./%'&2" ./#'&(2"

!"#$%&#'()

%$*%+,#''

-%.*/0'

!12'

Page 29: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

10.1.0.0/16  

BGP  Traffic  Engineering    

ASp  

AS2   AS3  

ASs   ASa  

AS4  

Outbound  Source:  ASS  

DesQnaQon:  ASp  

Inbound  Source:  ASS  

DesQnaQon:  Asp  Subnet  10.1.0.0/16  

 

Asp  Needs  to  control  the  inbound  traffic    to  balance:                              l2-­‐p;  l3-­‐p  e  l4-­‐p    

l2-­‐p  l3-­‐p  

l4-­‐p  

Page 30: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

ASp  

AS2   AS3  

ASs   ASa  

AS4  

Source:  ASS  DesQnaQon:  ASp  

Inbound  Source:  ASS  

DesQnaQon:  Asp  Subnet  10.1.0.0/16  

 

Prepend  with  ASPath  10.1.128.0/18  AS4  shortest  Path  

l2-­‐p  l3-­‐p  

l4-­‐p  

10.1.0.0/18  10.1.64.0/18  

10.1.192.0/18  

10.1.128.0/18  

BGP  Traffic  Engineering    

Page 31: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

ASp  

AS2   AS3  

ASs   ASa  

AS4  

Outbound  Source:  ASS  

DesQnaQon:  ASp  

Inbound  Source:  ASS  

DesQnaQon:  Asp  Subnet  10.1.0.0/16  

 

Prepend  with  ASPath  10.1.128.0/18  AS4  Shortest  Path  

l2-­‐p  l3-­‐p  

l4-­‐p  

10.1.0.0/18  10.1.64.0/18  

10.1.192.0/18  

10.1.128.0/18  

No  traffic  balance  guarantee  because  the  10.1.0.0  and  10.1.64.0  subnets  can  have  more  traffic  demand  per  user.  No  Inbound  100%  control.  

BGP  Traffic  Engineering    

Page 32: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

ASp  

AS2   AS3  

ASs   ASa  

AS4  

Outbound  Source:  ASS  

DesQnaQon:  ASp  

Inbound  Source:  ASS  

DesQnaQon:  Asp  Subnet  10.1.0.0/16  

 

Prepend  with  ASPath  10.1.128.0/18  AS4  shortest  Path  wins  

l2-­‐p  l3-­‐p  

l4-­‐p  

10.1.0.0/18  10.1.64.0/18  

10.1.192.0/18  

10.1.128.0/18  

The  soluQon  must  be  more  intelligent  than  use  BGP  parameters  only.  Other  parameters  extract  from  the  network  must  be  used.  

BGP  Traffic  Engineering    

Page 33: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Improving  the  prepend  decision-­‐making  Process  Start  

Network  CharacterizaQon  

Internet  full  rouQng  

Prepend  CharacterizaQon  

Internet  full  rouQng  

Success  Probability  to  use  Prepend  

Over  50%  

Resource  ForecasQng  

BGP  ModificaQon    2nd  Plane  opQon  or  Man2Man  negoQaQon  

Traffic  CharacterizaQon  

A

A

UQlizaQon  ForecasQng  

Best  Path  SelecQon  

Network  Change  TentaQve  

Monitoring  results  

End  

y  

n  

Stable  ?  

y  

n  Wait  

Next  Window  

Stability  Window  

Page 34: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Traffic  Engineering:  Decision-­‐making  

Traffic&Jam

&

Decision/Making&Process&

BGP&&

ATTRIBUTES&

BGP&&

ATTRIBUTES&

BGP&&

ATTRIB

UTES&

BGP&&

ATTRIBU

TES&

EUA&EUA&

EUA&South&America&

A?er&Decision/Making&Process&

US  

US  

US  

Page 35: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Conclusion  The  traffic  is  mostly  asymmetric  even  when  in  the  same  link,  because   the   huge   dependence   of   content   from   South  America  to  US.  This  also  can  be  explain  because  the  faciliQes  low   prices   in   US   hosQng   services   when   compare   to   South  America.   The   traffic   is   mulQfractal   and   because   that   the  forecasQng  must  to  use  this  model  to  represent  future  needs.  Anyway,  the  traffic  opQmizaQon  is  a  huge  task  even  when  the  task   is   internal   only,   considering   the   whole   Internet  environment   makes   this   task   much   difficult   because   by  definiQon   any   AS   is   itself   managed,   with   own   policies   and  procedures.  

Page 36: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

Obrigado  !    

Page 37: Engenharia de Tráfego na Internet: Entendendo o de tráfego ...ix.br/pttforum/7/doc/mdeusPTT_2013_Final.pdf · Engenharia de Tráfego na Internet: Entendendo o processo de tomada

Márcio De Deus

[email protected]

References  [1]  -­‐    Gireesh  Shrimali,  Aditya  Akella,    Almir  Mutapcic.  Stanford  University  &  University  of  Wisconsin-­‐Madison.  CooperaQve  Inter-­‐Domain  Traffic  Engineering  Using  Nash  Bargaining  and  DecomposiQon.    [2]  -­‐  R.  Mahajan,  D.  Wetherall,  and  T.  Anderson,  “Towards  coordinated  interdomain  traffic  engineering,”  in  HotNets-­‐III,  2004.  

[3]  -­‐  DEUS,  M.A.;  CARVALHO,  P.  H.  P.;  LEITE,  J.P.  BGP  Traffic  Engineering:  Understanding  South  America  Traffic  Pawerns  to  Improve  Decision-­‐Making.  In:  XXXI  Simpósio  Brasileiro  de  Telecomunicações,  2013,  Fortaleza.  SBrT  2013,  2013.  [4]    -­‐  DEUS,  M.  A.*  ;  CARVALHO,  P.  H.  P.;  SOLIS  BARRETO,  P.  S.  .  IP  and  3G  Bandwidth  Management  Strategies  applied  to  Capacity  Planning.  In:  OrQz,  Jesus  H..  (Org.).  TelecommunicaQons  Networks  -­‐  Current  Status  and  Future  Trends.  1ed.CroaQa:  In  Tech  Open  Access,  2012,  v.  01,  p.  1-­‐16.