Conférence 'Déployez votre site PHP / MariaDB, simplement et rapidement dans Azure' présentée au TechDays 2015 à Paris par Christophe Villeneuve et Serge Frezefond
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Les 4 axes du jour
● Les fondamentaux du déploiement
● Les notions à connaître
● Les différentes possibilités de communiquer
● Automatisation
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Déploiement dev → Azure
● git init
● git add .
● git commit -m "initial commit"
● git remote add azure [URL for remote repository]
● git push [URL repository] master
PC Websites
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Déploiement BDD MySQL
Sites web configurer → chaînes de connexion
● Database=remote_db_name;
Data Source=remote_host;
User Id=remote_username;
Password=remote_password
● mysqldump -u local_username --password=local_password
drupal |
mysql -h remote_host -u remote_username
--password=remote_password remote_db_name
PC
SQL
Database
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Vérification
● URL : http://hello-drupal-demo.azurewebsites.net/
2 solutions :
● Mise à jour de
settings.php
git commit -m 'update
setting'
● Visual Studio Online
Option PHP
Websites
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Visual Studio online (1/2)
● Sites web configurer
● Sites web tableau de bord
Websites
Visual Studio
Online
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Visual Studio Online (2/2)
● https://hello-drupal-demo.scm.azurewebsites.net/dev/wwwroot/
Websites
Visual Studio
Online
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Mode terminal
Vérification distribution linux
$ apt-get install git
$ apt-get install apache2 libapache2-mod-php5 mysql-server libapache2-mod-auth-
mysql php5-mysql phpmyadmin
$ sudo /etc/init.d/apache2 restart
..etc...
Installation AMP
Virtual
Machines
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MySQL → MariaDB
● Vérification
$ mysql -u root -p
show databases;
● Repositories
https://downloads.mariadb.org/mariadb/repositories
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MySQL → MariaDB (1/2)
https://downloads.mariadb.org/mariadb/repositories
sudo apt-get install software-properties-common
sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80
0xcbcb082a1bb943db
sudo add-apt-repository 'deb http://ftp.igh.cnrs.fr/pub/mariadb/repo/10.0/ubuntu
utopic main'
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MySQL → MariaDB (2/2)
sudo apt-get update
sudo apt-get install mariadb-server
$ mysql -u root -p
<enter password>
show databases;
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Test → prod (1/4)
● Sites web tableau de bord → Ajouter un nouvel emplacement
● Saisir un nom
Virtual
Machines
Virtual
Machines
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Test → prod (2/4)
● Résultat
● Echanger (swap)
Virtual
Machines
Virtual
Machines
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Test → prod (3/4)
● Résultat
● Echanger (swap)
Virtual
Machines
Virtual
Machines
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Performance
● Sites web Mettre à l'échelle → capacité
Virtual
Machines
Virtual
Machines
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB
●
A MySQL fork
●
Developed by original creators of MySQL
●
Adopted by Linux Distributions (Now default for RedHat 7
and Suse)
●
100% open source
●
Supported by big names that actively contribute:
Google, Booking.com,...
MariaDB 10.0
Scalability
● Advanced parallel replication
● Sharding
● MaxScale proxy (future)
Performance
● Enhanced optimization
● Improved and special purpose storage engines
● Carefully tuned and enhanced server internals
● Advanced performance monitoring
Availability
● HA clustering - integrating Galera cluster
● More online operations, less planned downtime
NoSQL
● Interoperable storage engines such as Cassandra and Connect
● Dynamic columns and JSON processing
● HandlerSocket API
Operations
● Comprehensive diagnostics built-in to the DB
● APIs and open architecture for easier integration
Security
● Role-based access control
● Authentication plugins
● Sophisticated auditing capabilities
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Multi-Source Replication
●
Collects data for analytics
using built-in replication.
●
Aids in administration
example: consolidated
backups of multiple databases.
●
Uses MariaDB 10’s improved
Global Transaction ID (GTID).
Online E-
Commerce
Application
Master
S S S S
Content
Management
System
Click-stream data
Data WarehouseSlave
ETL
Master
S S S S
Master
S S S S
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Dynamic (& Virtual) Columns
●
Store unstructured data in MariaDB tables with a simple API.
●
Use MariaDB’s indexing and transactions to manipulate
“document” style data fast and consistently.
●
Nest sets of dynamic columns inside of other dynamic columns -
hierarchical structuring.
●
Include multiple rows with dynamic columns in transactions.
●
Virtual Columns allows to create function based columns
Cust ID Account Balance Dyn_Col_BLOBs
2035 $154.04 NAME: John Smith|LOC: 45.35243, -74.98348|IMAGE: x27A8B8C ...
2036 $929.10 NAME: Jane Doe|LOC: 45.35243, -74.98348|AGE: 32| GENDER: F...
2037 $377.53 NAME: Carol Jones|AGE: 43|GENDER: F||IMAGE: xA9674DE678 ...
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Dynamic Column Example
MariaDB [test]> create table t1 (id int auto_increment primary
key,
-> name varchar(40),
-> type enum ("shirt", "phone", "computer"),
-> price decimal(10,2),
-> dynstr mediumblob);
MariaDB [test]> insert into t1 (name, type, price, dynstr)
values ("Funny shirt", "shirt", 10.0, COLUMN_CREATE(1, "blue",
10, "XL")),
…
MariaDB [test]> select id, name, type, price, length(dynstr) …
+----+-------------------------+----------+--------+------|
1 | Funny shirt | shirt | 10.00 | 17 | 1,10 |
…
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Dynamic Column Example
MariaDB [test]>
MariaDB [test]> select name, COLUMN_GET(dynstr, 1 as char(10))
from t1 where COLUMN_EXISTS(dynstr, 1);
+-------------------------+-----------------------------------+
| name | COLUMN_GET(dynstr, 1 as char(10)) |
+-------------------------+-----------------------------------+
| Funny shirt | blue |
| nokia | black |
| htc Desire hd | black |
| BM/Lenovo Thinkpad X60s | black |
+-------------------------+-----------------------------------+
4 rows in set (0.00 sec)
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Security
● Pluggable authentication (5.2)
– PAM Plugin
– Allow other authentication methods
● Audit Plugin (10.0)
– Track user access to data in real time
● Roles (10.0)
– Assign users to roles
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB 10 Interoperability : CONNECT Storage Engine
●
From 3rd party developer.
●
Maps diverse data
to tables.
●
JOIN mapped data
to DB tables.
●
Flat files including CSV.
●
Tables in external Dbs.
●
Generated tables
(PIVOT etc.)
●
Plug-in API for your own mappings.
Application
Spi
der
MariaDB Parser/Optimizer/Connection Pool
CONNECT
Engine
Other
Engines
Database
Tables
Powerful tool for data integration, federation.
.log
XML
CSV
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
JSON Table Type (1/2)
[
{
"ISBN": "9782212090819",
"LANG": "fr",
"SUBJECT": "applications",
"AUTHOR": [
{
"FIRSTNAME": "Jean-Christophe",
"LASTNAME": "Bernadac"
},
… ],
"TITLE": "Construire une application XML",
"PUBLISHER": {
…
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
JSON Table Type (2/2)
create table jsampall (
ISBN char(15),
Language char(2) field_format='LANG',
Subject char(32) field_format='SUBJECT',
Author char(128) field_format='AUTHOR:[" and "]',
Title char(32) field_format='TITLE',
Publisher char(20) field_format='PUBLISHER:NAME',
Location char(16) field_format='PUBLISHER:PLACE',
Year int(4) field_format='DATEPUB')
engine=CONNECT table_type=JSON File_name='biblio3.jsn';
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
JSON Table Type : Query Result
select title, author, publisher, location from jsampall;
Title author publisher location
Construire application XML Jean Bernadac and François Knab
Eyrolles Paris
XML en Action William J. Pardi Microsoft Press Paris
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
JSON Table Type : The Jpath Specification
Specification Array Type Description
[n] All Take the nth value of the array. Ignore it if n is 0.
[X] or [x] All Expand. Generate one row for each array value.
["string”] StringConcatenate all values separated by the specified string.
[+] Numeric Make the sum of all the array values.
[*] Numeric Make the product of all array values.
[!] Numeric Make the average of all the array values.
[>] or [<] All Return the greatest or least value of the array.
[#] All Return the number of values in the array.
[] All Sum if numeric, else concatenation separated by “, “.
All Take the first value if an array.
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Galera Cluster
Synchronous Replication Cluster
●
Read & Write access to any node
●
Client can connect to any node
●
There can be several nodes
●
Automatic node provisioning
●
Replication is synchronous Galera Replication
MariaDB MariaDB MariaDB
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Galera Cluster and Standard replication
Mgc1
Mgc2
Mgc3
Srv1 Srv2 Srv2
Galera Cluster
MariaDB Multi Source
Asynchronous replication
Synchronous replication
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB Galera Cluster and Standard replication
Mgc1
Mgc2
Mgc3 Slave
Asynchronous replication
Synchronous replication
Slave can easily bind to new master with GTID
Galera Cluster
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB MaxScale Proxy
• Connection Load Balancing
• RW splitting
• Galera Connection Load Balancing
• MaxScale monitors the backends status
• Other usages : filtering, auditing, query
rewriting …
MaxScale
Technology Preview
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
MariaDB MaxScale BinLog Server
●
A binlog server carries binlog / no databases
●
Small lag / Small drain on master
●
Easy reconfiguration of topology
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
●
VM Creation from image Galery
●
VM Disks / Affinity group
●
Virtual Network
●
End Points / Load Balancer
●
Storage / Containers
●
Image Creation in Gallery
Azure actions can be fully scripted in IaaS mode
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
Azure actions can be fully scripted in IaaS mode
$ azure network vnet create
--address-space 10.0.0.0 --cidr 8
--subnet-name mariadb --subnet-start-ip 10.0.0.0
--subnet-cidr 24
--affinity-group galeraag galeravnet
…
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
$ azure vm create -v --vm-name sfmariadb5
--virtual-network-name galeravnet2
--subnet-names mariadbsn2 --affinity-group galeraag
--vm-size large
--ssh 22 --ssh-cert ./myCert.pem --no-ssh-password
sfmariadb5 5112500ae3b842c8b9c604889f8753c3__OpenLogic-
CentOS-66-20150128 azureuser
…
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
// Adding load balancing
$ azure vm endpoint create --lb-set-name mysql mariadb1 3306
$ azure vm endpoint create --lb-set-name mysql mariadb2 3306
$ azure vm endpoint create --lb-set-name mysql mariadb3 3306
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
$ sudo yum install MariaDB-Galera-server MariaDB-client galera
$ sudo service mysql start
$ sudo /usr/bin/mysql_secure_installation
// create user for gal era cluster
$ mysql -u root -pmanager1 <<EOF
DELETE FROM mysql.user WHERE user='';
GRANT ALL ON *.* TO 'root'@'%' IDENTIFIED BY 'dbpass';
GRANT USAGE ON *.* to sst_user@'%' IDENTIFIED BY 'dbpass';
GRANT ALL PRIVILEGES on *.* to sst_user@'%';
FLUSH PRIVILEGES;
EOF
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
// /etc/my.cnf.d/server.cnf
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://10.0.0.9,10.0.0.11,10.0.0.13"
wsrep_cluster_name='galera_cluster'
wsrep_node_address='10.0.0.9'
wsrep_node_name=’sfmariadb1'
…
$ sudo /etc/init.d/mysql start --wsrep-new-cluster
$ sudo /etc/init.d/mysql start
$ sudo /etc/init.d/mysql start
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Azure Load Balancing : Web & MariaDB
Network Load Balancer
Windows Azure Datacenter
← HTTP load balancing
Network Load Balancer ← MariaDB load balancing
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure
●
Using configuration tools
●
Instead of manual install :
yum/apt install Pupet / Chef / Ansible
●
Vagrant provider for Azure :
create VM, deploy and configure software
●
Docker : can create VM
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure : Ansible
●
hosts: db
user: root
tasks:
– name: get epel-repo
…
– name: copy mariadb repo file
copy: src=/etc/ansible/files/MariaDB.repo dest=/etc/yum.repos.d/MariaDB.repo
– name: install mariadb galera server
yum: name=MariaDB-Galera-server state=latest
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure : Ansible
●
hosts: cluster1
user: root
tasks:
– name: is mysql running?
…
– name: bootstrap by starting mysql with gcomm://
action: shell service mysql start –wsrep_new_cluster
when: is_mysql_running.rc > 0
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
How to deploy MariaDB in Azure : Ansible
●
hosts: cluster2:cluster3
tasks:
– name: if the cluster is not started already, start it
action: service name=mysql state=started
– hosts: cluster1
user: root
tasks:
– name: Create user for MaxScale
…
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Deploy more complex MariaDB architectures
Configuration can be automated
●
Master Slave (multi levels)
●
Galera Cluster (with replication for more HA)
●
Load Balancer (Azure, HaProxy, MaxScale)
●
Sharded configuration(MariaDB Spider)
tech.days 2015#mstechdaysDéployez votre site PHP / MariaDB, simplement et rapidement dans Azure
Operate MariaDB in the cloud
●
Backup
– Mysqldump, xtrabackup
●
Monitoring
– Open Source solutions : Nagios, ZABBIX,
– Proprietary tools :Monyog
– Use insight ? To test
●
Azure Storage / Archive can be used to push
MariaDB backups
tech.days 2015#mstechdays
Activez vos bénéfices Azure jusqu’à
115€ de ressources mensuelles
offertes
115€ /mois
x5 membres
x3 ans
= 4 175€ de ressources offertes
http://azure.com http://aka.ms/azurepourmsdn
150€ de ressources offertes
Sans engagement
Pour tous
Un mois d’essai offert
http://www.microsoft.com/bizspark/
Les Ressources Azure
Pour les startups
Bizspark
= 49 000€ de ressources offertes
pendant un an
Pour les abonnés
MSDN