Upload
totvs
View
424
Download
0
Embed Size (px)
Citation preview
CaseTailTarget:infraestruturaemodelodedesenvolvimento
FernandoBabadopulos@babadopulos
NuvemAplicaçãoEscalável
Automa4zarTarefasDeployCon=nuo
IntegraçãoCon=nuaTestesAutoma4zados
ArquiteturasEscaláveis
Arquiteteparapelomenos3
Assumaquealgumacoisairáfalhar
Isoleosseusserviços
Architec4ngforScalability
SHAREDNOTHING
SharedNothing
AppServer1 AppServer2 AppServer3
Cache1 Cache2 Cache3
DB1 DB2 DB3
SharedNothing
AppServer1 AppServer2 AppServer3
Cache1 Cache2 Cache3
DB1 DB2 DB3
Cache4
- Semserver-sidesession
- SemsMckysessions
SharedNothing
PlayFramework PlayFramework PlayFramework
RedisShard1/Replica1
RedisShard2/Replica1
RedisShard1/Replica2
MongoDBMaster
MongoDBSlave1
MongoDBSlave2
RedisShard2/Replica2
Escalando
UseopoderdaNuvem
masuseosseusDADOS!
PrediçãodeTráfego–Porque?
Podedemorarmuitosminutosparaterumamáquinanoar,atéhoras.Dáparaesperartudoisso?Evitefalsasquedasdetráfego
PrediçãodeTráfego
700
800
900
1000
1100
1200
1300
1400
1500
1600
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61
Requests
Requests
PrediçãodeTráfego
700
800
900
1000
1100
1200
1300
1400
1500
1600
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61
Requests
Requests
Linear(Requests)
PrediçãodeTráfego
700
800
900
1000
1100
1200
1300
1400
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61
Requests
Requests
Linear(Requests)
PrediçãodeTráfego
700
800
900
1000
1100
1200
1300
1400
1500
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61
Requests
PredictedTraffic
Linear(Requests)
Muitasvezesvocênãoprecisaprocessartodososseusdados
705.166
1.288.822.081
62ms
1,5h
6.931
12.678.931
x1.190.642.440651.448
= 12.667.692
Erro≈0.088%
Totalderegistrosconhecido
amostragem
Totalderegistrosamostrados
REDIS01 REDIS02 REDIS03 REDIS04 REDIS05
for (Jedis jedis : cluster.getAllShards()) { }
long dbSize = jedis.dbSize();
String key = jedis.randomKey();
String value = jedis.get(key);
.
.
.
.
.
F 0.728 OUT
Tamanhodaamostrafinal
A
0.461
B
0.352
C
0.831
D
0.314
E
0.816F 0.728
IN
ReservoirSamplingDistribuído
1 2 3 4 5
A:0.1B:0.3C:0.2D:0.7E:0.9F:0.11G:0.4H:0.6I:0.76
J:0.8K:0.2L:0.54M:0.4N:0.21O:0.33P:0.56Q:0.32R:0.23
S:0.21T:0.32U:0.22V:0.7X:0.12Y:0.23W:0.3Z:0.76
private SortedMap<Double, MyObject> reservoir; ... if (reservoir.size() < SAMPLE_SIZE) { reservoir.put(score, myObject); } else if (score > reservoir.firstKey()) { reservoir.remove(reservoir.firstKey()); reservoir.put(score, myObject); }
ReservoirSamplingDistribuído
O L P I Z
1 2 3 4 5
A:0.1B:0.3C:0.2D:0.7E:0.9F:0.11G:0.4H:0.6I:0.76
J:0.8K:0.2L:0.54M:0.4N:0.21O:0.33P:0.56Q:0.32R:0.23
S:0.21T:0.32U:0.22V:0.7X:0.12Y:0.23W:0.3Z:0.76
H:0.6D:0.7E:0.9F:0.11I:0.76 R:0.23Q:0.32O:0.33L:0.54P:0.56 S:0.21U:0.22Y:0.23T:0.32Z:0.76
COMBINER
ReservoirSamplingDistribuído
import org.apache.crunch.lib.Sample; Sample.reservoirSample(PCollection<T> input, int sampleSize)
ApacheCrunch:
CaseTailTarget:infraestruturaemodelodedesenvolvimento
FernandoBabadopulos@babadopulos