desmistificando tecnologias

Mobile Development

Desmistificando tecnologias

Juliano Marcos Martins [email protected]

http://jmmwrite.wordpress.com

Objetivo

Apresentar tecnologias e meios de forma objetiva e simples, enriquecendo a bagagem de conhecimento dos presentes.

Agregar conhecimento que permita capitalizar.

Disclaimmer

Esta apresentao NO FOI cuidadosamente revisada e por isso no esta livre de erros.

Toda critica ser muito bem aceita ;)

As opinies apresentadas durante a palestra so de minha responsabilidade e no tem relao com as empresas/Instituies nas quais trabalhei/trabalho ou tive qualquer tipo de relacionamento.

Agenda

Eu

Desmistificando TecnologiasDevOps

BigData

IoT

Desenvolvimento Mobile

TroubleShooting e Performance

Open Source

Mercado de Trabalho

Perfil do Profissional

Q & A

Eu

1999 Estagirio professor

2000 - Estagirio Programador

2003 - Programador

nica

2004 Analista de I.

2005 IT Specialist

2008Lab DB2

2011Arquiteto

2007Arquiteto

2014IT Spec

2005

2015

Professor

Profissional

Projeto Atual

PMP Professional Market Place

Big Data

SSD App 6 BC-H with 72 data blades, 2 admin, 1 mgmt blade + 1 Cisco 6509 switchesD5100 6 Data modules, 1 admin, 1 mgmt, 1 RSM, 2 standby + 3 SMC switches + 4 san switches

Raw Data6 TB6 TB

CPU77 Intel Core i7 940 (308 cores)11 Intel E5450 (44 cores)

Memory576 GB192 GB

Total disk cap10.8 TB13.2 TB

Raw storage cap22.8 TB21 TB

MB/s84 + GB/s7.8 GB/s

IOPs4,320,000+48,000+

Network72 x 1G Ethernet (FCM)+72 x 1G replication + corp + backup12 x 1G Ethernet (FCM)+ (12 FC/G Eth for backups)

SANN/A4 Gbps SAN infras.

Power21,606 Watts*12,930 Watts*

Footprint69U48U

Sweet Child o'mine

Big Data

Data is a new Class of economic asset, like currency or gold. Davos, Suia, 2012 World Economic Summit

Intro

Tecnologia + Metodologia para Capturar, Armazenar e Analisar grandes volumes de dados

Volume, Velocidade e Variedade (+ veracidade e valor)

Finalidade: dar subsidios para tomadas de deciso

Temos muitos dados, pouca informao

Intro

Anlise de sentimento com dados da Web e sociaisDepartamentos de marketing usam feeds do Twitter para realizar anlise de sentimento e determinar o que os usurios esto falando sobre a empresa e seus produtos ou servios, especialmente aps o lanamento de um novo produto ou release.O sentimento do cliente deve ser integrado aos dados de perfil do cliente para derivar resultados significativos. O feedback do cliente pode variar de acordo com seus aspectos demogrficos.

Fraudes em plano de sade, abertura de contas

O case Abertura de conta

SciCrop

METODOLOGIA

Identificao

ColetaAlgoritmo
prdeterminadoResposta

Identificao

ColetaAlgoritmoResposta

VARIVEL INCERTACRESCENTE RPIDA
TERABYTES??Se no responder rpido, nem adianta responder.ISSO BIGDATA!IDENTIFICAO, PREDIO E RECOMENDAOO que acontece quando os problemas variam seus padres em condies incertas?Resoluo de problemas determinsticos.FORECASTING
&
NOWCASTING

ONDE, O QU, QUANDO?Dados EstruturadosDados No Estruturados

Armaz. DistribudoMachine Learning, Data MiningClassificao: SVM, Naive-Bayes, logistic regression, K-nearestRegresso: Linear (lasso, ridge), polinomial, etc Clustering: K-means, Spectral clustering, etcReduo Dimensional: Isomap, kernel aprox, PCA, singular value decompositionrvores de DecisoRandom ForestsAnlise de Associao: Apriori, FP-growthModelo Escondido de MarkovNo supervisionadosSupervisionadosContnuoCategrico

ETL / MRDADOSANLISE

ETL: Extract, Transform and Load

MR: Map Reduce

Data WareHouse, Sistemas Legados, Main FrameDispositivos, SMS, Redes Sociais, udio, Vdeo, Geo, Logs

DATA VISUALIZATION - SEARCHCopiado livremente da Apresentao de Jos Damico FEMA Set, 2015

Tecnologias

Datasources: Facebook, Twitter, Blogs, Aplicaes, bancos, Devices (IoT), etc...Banco de dados NoSQL (Not only Structured Query Language) MongoDB, Cassandra, Hbase

Processamento distribuido: Apache Hadoop/HDFS, Hive, Impala, Pig

Anlise: R, Pentaho, Python

Recursos na nuvem (sob demanda elstico)

Solr

O Profissional

O profissional big data (Data Scientist) deve ser multidisciplinar (matemtica, estatstica, modelagem, reconhecimento de padres, IA, DW, ETL, High Performance Computation, etc).

DevOps

What is?

DevOps is a software development method that stresses collaboration between Software Developers (Development) and Information Technology professionals (Operations).

DevOps brings together Development and Operations toImprove collaborations and productivity

Automate InfraStructure/Workflows

Measure app performance / code testing

Meu slide resumo

ALM - Rational Team Concert, Microsoft Team Foundation, Jira

Versionamento/SCM: Git, SVN, CVS

Jenkins, Hudson, LuntBuild CI Integrao continua

BugZilla

Jmeter, FindBugs, PMP, CheckStyle, Estilo de codificao.

SOAP UI

Linux / Windows Ambiente do Cliente (verses)

Instrumentalizar criar ferramentas

Geradores Trabalho repetitivo

History of Development and Operations

Software

Admin

In the beginning, Development and Operations were typically performed by the same person.

As time went on, that changed, and Development and Operations separated.

Id like to share a brief history of DevOps. In the beginning of software development, the development and operations person was the same. If you, for example, ran a punchboard, then the business logic pretty much went from paper immediately into what wed call production today, as you punched different wires within a punchboard.

As time went on and we got a bit more sophisticated, a schism started to unfold between development teams and operations teams. Im probably dating myself, Back in the day, coders would write Assembler, Fortran, or Cobol programs and have the programs punched out on a punch card. Then you would take that deck of punch cards and hand it over to the operator who would order them, make sure that they made sense, and then feed them into the system that they operated.

As the years went by, systems changed, the schism widened, and operations and development became further and further estranged from each other. When developing applications and JCL, which was the request of how to configure the system so an application could run, a developer would send both to the operations team. From there, the operations team would determine whether or not they could accommodate the request in JCL, and then run or release the program.

Solution production: The Legacy Model

In this model, Development and Operations are separate, and do not interact until the end of the process.

The legacy process of developing applications and promoting them into production typically went like this: development teams developed, they did some unit testing, but then at the end of the day, they simply created deployment plans or deployment requests, as well as configuration management request and plans.

After that, they turned it over sometimes to a release coordinator who reviewed the plan and made sure that all the required elements were there, before handing it over to the operations team as a deployment request. If there was testing, then the configuration management plan and the use cases were given to the testers to build their test plans. Then, they handed that over to the operations team so that they could test the cost in user acceptance testing. Operations really held the key here.

In the mainframe days, an application would only get implemented in production if operations released it. In the client server days, certainly our development teams kind of turned the page back a bit and at first configured and genned some of their servers before releasing the code directly, but typically their knowledge was limited. Later, as server farms were introduced, development teams knew little about what else was running on the servers that they were deploying to, and depended on the operations teams to ensure that their new applications do not crash multiple systems. So again, the standard became two separate teams that do not interact with each other until the end of the process.

Pre-DevOps processes

Operations First:

Charter / Fund: 2 weeks to 3 months

Acquire Infrastructure (CAPEX): 6 weeks to 6 months

Integrate Infrastructure (OPEX): 4 weeks to 8 weeks

Dry runs, validation, acceptance testing (OPEX): 2 weeks to 4 weeks

Development Second:

Workstation SCM

SCM Dev

Dev Test

Test Stage, Prod

Issues:

Patch Level / Security

Demand, Scale to Max Peak

Release schedules per Quarter / Year

Governance Process alignment

Consistency, Speed, Reliability

A new project can take between 6 and 18 months before release 1.0 is in production!

Issues with the Legacy Model

Your customers find major defects.

Major defects take a long time to fix.

You cannot do anything until you have everything.

Legacy model leads to unique infrastructure dependencies.

People do not talk to each other.

This leads to broken processes and overbearing governance.

Even if you are agile, the delivery process is still broken.

Going live still takes as long as ever.

Any problems lead to finger pointing.

Development and Operations are not able to operate as one team.

Introduction to DevOps: Overview

The Application Deployment Model Must Change

DevOps change is as much a culture change as it is a technology change.

DevOps is married to an agile process and originally DevOps was also a very rugged process. Rugged DevOps should be repeatable, with no manual steps. It should be reliable, reviewable, fast, and be able to stand an audit. All of this this helped create rapid deployment, rapid building, and rapid restoration. Resilience should be automated, along with the configuration process. It should also reduce or limit the attack surface, therefore reducing the risk of failure.

All of this is critical, but it also means that DevOps with agile as much a cultural change as it is a change in the technology approach. The cultural change is increased collaboration throughout the entire lifecycle.


Solution production: The DevOps Model who is involved?

Business function owners, developers, and operations all are involved early on in the process.

Heres the first of two diagrams that help to define the DevOps model, and how it differs from the legacy model. First, whos involved?

The players involved span the lifecycle of the application development process. They include the business function owners: those who write up the storyboard or the business need and then translate them into use cases or translate them into a storyboard for business goals to be developed. Development needs to review this, get an understanding of whats in the realm of possible and whats not, and then prioritize what can be done to meet a rapid and agile schedule. Through all of this and through this collaborative process with the business owners, development, and operations, operations primary role is to make sure that they have an environment that will be able to run and scale the new changes. They also need to do the same for the testing environments and enable rapid development, rapid release, and a continuous release process.


Solution production: The DevOps Model swimlane view

The DevOps software delivery process introduces transparency and culture change.

Now, a typical agile development software delivery process looks like what you see here. This is a bit different from the legacy model, and certainly cleaner. It is certainly more of a waterfall type flow, moving from one event to the next, and is certainly a more agile process as well. In addition, this is clearly more of a collaborative process across the board and across the lifecycle between business development and operations.

Where will you typically see DevOps solutions?

DevOps Solutions are commonly found in:

Web sites (mobile, desktop)

Mobile apps using backends as a server component for information resources

Mobile application front-end development

DevOps solution are not found as commonly in:

Typical packaged / platform-based services (BI Systems, ERP systems)

Back-end Systems (ETL Systems, Systems of Record data sources)

DevOps lends itself to situations requiring rapid changes in response to functionality and scale.

Key DevOps attributes

Startups, quick initiatives, exploring / prototyping:

More systems of engagement involved

Mobile on the front-end

MBaaS middleware to API catalog

Back end APIs or mashable APIs are used

Common unit of deployment (Operations) allows for:

Scalability(demand management)

Movement

Recovery

Feature deployment

Scripting / automation is critical


Post-DevOps implementation

Infrastructure is more consistent and predictable.

Build / deploy processes are typically done outside of the bastion of enterprise.

Development

Operations

QA

Frequent Builds

Automated Builds

Known Deploy Units

Known Accounting

Fast Setup & Teardown

Automated Monitoring

Consistent Scripts

Peer Code Reviews

Automated Tests

Provisioning is fast.

On-demand provisioning is both good and bad at the same time, from a financial standpoint.

The post-DevOps implementation world looks a bit different. Development has frequent builds. It automates those builds and certainly knows how to release on a faster, more agile basis. From an operations perspective, deploy units are known and certainly the deploy units are smaller as well. Automation is monitored. Theres fast setup and teardowns of environment. The scripting for these environments is standardized, and so is testing, which is also automated. There are also peer reviews that include both development and operations to make sure that what gets deployed in an agile fashion will meet the necessary business goals and objectives.


Sample DevOps maturity model

Define release with business objectivesMeasure to customer value

Optimize applicationsUse enterprise issue resolution procedures

Fully Achieved

Initiative Goals

Partially Achieved

Centralize requirements managementMeasure to project metrics

Document objectives locallyManage department resources

Manage lifecycle artifactsSchedule SCM integrations and automated builds Test following construction

Plan and manage releases Standardize deployments

Monitor resources consistentlyCollaborate DevOps informally

Plan and source strategically

Dashboard portfolio measures

Link lifecycle information Deliver and build with testCentralize and automate test management

Monitor using business and end user contextCentralize event notification and incident resolution

Automate problem isolation and issue resolutionOptimize to customer KPIs continuously

Improve continuously with development intelligence Test continuously

Manage environments through automationProvide self-service build, provision, and deploy

Manage data and virtualize services for test Deliver and integrate continuously

Standardize and automate cross-enterpriseAutomate patterns-based provision and deploy

Plan departmental releases and automate statusAutomated deployment with standard topologies

Here, lets look at some environments and we can determine their maturity level for DevOps using a DevOps maturity model. As you see, that maturity models has a four-column by four-row matrix, where you can look towards a DevOps environment, or a DevOps development lifecycle. Here you can examine four key parts of the software process: plan and measure, development and test, release and deploy, and then the continuous monitor and optimize that starts the continuous loop all over again.

As you can see in this model, or heat map, you can help a customer identify their particular maturity level under any one of the four silos or any one of the four columns by going through and asking targeted questions.


Example IBM deployment tooling

Line of Business

IBM UrbanCode Build

Rational Team Concert

Rational Quality ManagerRational Test WorkbenchRational Test Virtualization Server

SmartCloud Application Performance Management

Rational Focal PointRational Requirements Composer

SmartCloud OrchestratorIBM Pure Application System

IBM UrbanCode Deploy

IBM UrbanCode Release

And of course, from a deployment tooling perspective, our UrbanCode products help to streamline our build, release, and deploy processes.

DevOps aligns Operations and Development, stressing collaboration between the two areas.

DevOps is a pattern, not necessarily a product.

DevOps is as much about processes, culture, and tools, as it is about a methodology.

Think on how you can impact your customers business by helping them:

Accelerate

Fail fast (and recover!)

Transform the business

Meet business goals and objectives

DevOps - Summary

IoT

Intro

Prototipao

Sensoriamento

Interligao

Novas idias

O cu o limite!

Performance & Tunning

Intro

Profissionais no se preparam antes que o problema aconteca

SQLIndices

Where

Limite

Select com FOR READ

Threads

Paralelismo

Exemplo simples

SELECT * FROM TABELA;

SELECT * FROM TABELA FETCH FIRST 10 ROWS ONLY FOR READ ONLY WITH UR; (where? where? where?)

FETCH FIRST 10 ROWS ONLY -> quer dizer que pra trazer somente as 10 primeiras linhas encontradas

FOR READ -> fala pro banco que esta trazendo dados somente para LEITURA, de forma superficial, quando voc no usa isso, o banco deixa CURSORES posicionados na posio dos registros que voc trouxe, ja preparando pra um update/delete isso, consome recursos.

WITH UR -> quer dizer with uncommited records, ou seja, j traz os dados mais atualizados do banco, que sofreram um update, mesmo no tendo sofrido commit ainda.

Troubleshooting

Intro

Erros e problemas comuns

Exemplo Softwares legados que param de funcionar

Falta de planejamento

Ler Logs

Reproduzir comportamentos

Open Source

Intro

Open Source Drives the world!LinuxMicrosoft InitiativesThe Apache FoundationOpen Source != Free Software

Mobile Development

IOs

IPhone, IpadApp Store

http://www.apple.com

Google Android

80% of the marketGoogle Play

https://www.android.com/

Windows Phone, Ubuntu, Tizen, BlackBerry, etc...

NATIVE

Hybrid Mobile App

Angular + Cordova (IONIC)

Appcelerator + Titanium IDE

Xamarin

Python + Kivy

Tecnologies

HTML

CSS

JavaScript

Json

Angular

BootStrap

Hybrid Mobile App

Built with HTML, CSS and JavaScript and is contained in a native wrapper so that it can be installed on a mobile device.

This allows web developers to build mobile apps without having to learn the native programming languages (e.g., Objective-C, Swift, Java).

It also means that you can have one codebase for all the different platforms.

Frameworks...

Attention Points:Maybe you can loose some native functions that use specific OS/Hardware resources.

Maybe the performance can be lower than the native development.

Hybrid PROS

1- Easy to learn and Develop2- Fast to create a prototype3- Easy to find workforce4- Easy to create a beautiful app5- No need to learn native technology6- Its faster to develop than Native (supposing that you are good in both technologies)7- Write once! (keep in mind that there are differences from Platforms)

Hybrid CONS

1- Performance can be lower (supposing that you are expert in Native and Hybrid)2- You can loose native functions specific for a OS/Device

When go Native or Hybrid*

IF ( NEED HIGH PERFORMANCE or NEED A LOT NATIVE RESOURCES or NEED A NATIVE RESOURCE NOT SUPPORTED) { goNative();} ELSE { goHibrid();}

* this is what I THINK. Please, research before start your project. Many fail because there are no planning, just code write.

Adobe buy NitobeNitobe PhoneGapOpen Source

2009

2011

Adobe PhoneGapApache Cordova

Apache Cordova

Apache Cordova is a platform for building native mobile applications using HTML, CSS and JavaScript.

http://cordova.apache.org/

Amazon Fire OS

Android

BlackBerry 10

Firefox OS

iOS

Ubuntu

Windows Phone 8

Windows

Tizen

PhoneGap

PhoneGap is Adobes productized version and ecosystem on top of Cordova.

http://phonegap.com/

IONIC

Free and open source, Ionic offers a library of mobile-optimized HTML, CSS and JS components, gestures, and tools for building highly interactive apps. Built with Sass and optimized for AngularJS.

http://ionicframework.com

Installation

Install Git - https://git-scm.com/

Install Node Package Manager (npm) - https://nodejs.org/

Install Cordova/Ionic - npm install -g cordova ionic

Install Android SDK - http://developer.android.com/sdk/index.html

Create Environment Variable ANDROID_HOME , Example: ANDROID_HOME=/home/julianom/Android/Sdk Install Java JDK 7

Create Environment Variable JAVA_HOME , Example (Linux):JAVA_HOME=/opt/java Its a good idea put at your Path:

$JAVA_HOME/bin and $ANDROID_HOME/tools, Example (Linux): PATH=$JAVA_HOME/bin:$ANDROID_HOME/tools:$PATH Install Genymotion (to emulate Android) - https://www.genymotion.com/#!/

Create a virtual device at Genymotion and test it. I recommend a not powerful device, like a Galaxy S3.

Install a good Editor, I recommend Brackets (Its free) - http://brackets.io/

Create a user at Git Hub - https://github.com/

Take note about the users that you will create, you will use along the course.

Hands on IONIC

ionic start myApp tabs

cd myApp

ionic serve

ionic serve --lab

Hands on IONIC

ionic platform list list available platforms

Android

ionic platform add android

ionic emulate android

or

ionic run android

IOs

ionic platform add ios

ionic emulate ios

or

ionic run ios

Testing

Android We recommend test with Genymotion, by running the command ionic run android. Also, if you plug your Android phone at your computer, the run command should install it at your device.

IOs You can test in the simulator running ioinic emulate ios.

You can open your project in at XCode by going to the folder platforms/PLATFORM of your project, and run this as a normal XCode project. Also, you can publish your app from here.

Any time that you change content inside www folder, run the command cordova prepare ios in order to update the XCode project. Have in mind that this will overwrite any change that you have done at XCode.

More about XCode/Cordova:http://cordova.apache.org/docs/en/3.3.0/guide_platforms_ios_index.md.html

Thank you

for your dedication and patience!

All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 3.0 License (unless otherwise specified). Adobe, Google, Apple, Apache, and other are registered trademarks. Their respective logos and icons are subject to international copyright laws.

24/10/15

24/10/15

desmistificando tecnologias

Education