Technical

Symfony cache: Redis vs Memcache vs APC/APCu

TLDR: Go with Redis!

REDIS VS MEMCACHE VS APCU
Redis and memcache are more or less interchangeable. Redis is considered the newer/faster option but real world performance hasn’t always reflected that sentiment. I naturally would try Redis first and see if it’s super fast, no weird issues. If Redis isn’t totally smooth, go straight to memcache. (Or for a totally hassle-free option, just go memcache first.) And then if you want to venture further, go with APCu. APCu is faster than both Redis/memcache but can be unstable or total memory-hog.
https://wpjohnny.com/redis-vs-memcache-vs-apcu-object-cache-comparisons/
Existing Symfony Bundles:

Articles:

  • Caching in Microservices – A Highly Effective way to Maximize Performance

    Below are the 4 things to consider when deciding to use the cache.

    1. What to cache.
    2. When to cache.
    3. Where to cache.
    4. How long to cache.
  • Why Redis beats Memcached for caching

    Memcached or Redis? These renowned cache engines share a number of similarities, but they also have important differences. Redis, the newer and more versatile of the two, is almost always the superior choice.

  • Memcached vs Redis, Which One to Pick?

    The performance and memory usage of Redis compared to memcached is relatively similar. Unless you already have a large investment in memcached, going forward redis is the obvious solution.Not only Redis is better option, it enables whole new types of use cases and usage patterns.

APC (Alternative PHP Cache) is opcode cache and data store and it is discontinued in favour of APCu (APC User Cache: “userland” data caching) which is only data store.
Since Zend Opcache (is only opcode caching) was included in core PHP >5.5 we can set it up altogether with Redis.. APCu is faster than both Redis/memcache but can be unstable or total memory-hog.

We have to differentiate two things:

  • The OPCode cache (formerly APC, xcache, etc…) : only serves the purpose of storing compiled version of PHP code into memory, and make it run faster – this has nothing to do with the later
  • The applicative cache : purely business stuff that Symfony will store into a raw storage in order to avoid to do some business computations too often and be faster

OPCode cache and applicative cache are really very different things, you should in order to understand ignore the word “cache” in the “OPCode cache” term, and replace it with “compiled PHP binary” which makes much more sense.

Considering the APCu (formerly APC user cache) user cache, it’s a storage for applicative cache, you can tell Symfony to use it the same way as Redis. Main problem with APCu is that it will store cache in the PHP shared memory, which means that you need to have *a lot* of dedicated RAM to the PHP processes to storeSymfony data into it. Plus, if you multiple PHP frontends, each frontend will carry its own shared memory, data in those frontends may differs: this means that you can have a cache desync between PHP frontends; That’s why we often prefer to use a Memcache, or Redis or Mongodb.
https://www.drupal.org/u/pounard

 

Redis use case:

Redis is a NoSQL key-value data store.

Pub/Sub: You can keep track of subscribers and what information they are interested in. You can use SUBSCRIBE, UNSUBSCRIBE, and PUBLISH.

Queues: You can use normal queue data structure, Redis has blocking queue commands as well.

Real time analysis of events(stats, anti spam etc) – Using Redis primitives it’s really simple to implement a real-time tracking filtering system or a spam filtering system.

Order by user votes and time: Leaderboard type functionality(Quora/Reddit/Digg), where the score is changes over time.

Transactional support: Provides transactional support. See Transactions – Redis.

Caching: Can be used as a general purpose cache. keys can strings, or more complex types (lists, sets, hashes etc). Also allows LRU eviction of items.

As a conclusion:
The microservice should include a straight forward setup for Redis: by default it contains database caching. If the dev wants to add more “areas” to cache then it is just a matter of configuring the bundle for:

  • Sessions
  • Monolog logging
  • SwiftMailer spooling
  • Profiler storage

That’s about data caching. For opcode caching we enable Opcache by default and set it up as explained by Symfony docs.

PhpStorm as diff tool in Sourcetree

I’am using JetBrain toolbox and this is the config I made for Sourcetree to do diff operations on PhpStorm:

First open PhpStorm and create a command line launcher: Tools -> Create command line launcher..

In my case It suggested: /usr/local/bin/pstorm

Next open Sourcetree preferences and in the Diff tab set those values for diff:

  • Visual Diff Tool: Custom..
  • Diff Command: /usr/local/bin/pstorm
  • Arguments: diff $LOCAL $PWD/$REMOTE

Set those values for Merge:

  • Merge Tool: Custom..
  • Merge Command: /usr/local/bin/pstorm
  • Arguments: merge $PWD/$LOCAL $PWD/$REMOTE $PWD/$BASE $PWD/$MERGED

Enjoy!

Inspect CPU usage in Linode

Linode Alert – CPU Usage
Your Linode, linode4496636, has exceeded the notification threshold (90) for CPU Usage by averaging 199.2% for the last 2 hours. The dashboard for this specific Linode is located at…

htop

Htop is a free (GPL) ncurses-based process viewer for Linux. It is similar to top, but allows you to scroll vertically and horizontally, so you can see all the processes running on the system, along with their full command lines, as well as viewing them as a process tree, selecting multiple processes and acting on them all at once.Tasks related to processes (killing, renicing) can be done without entering their PIDs.

Docker the cause!

sudo systemctl stop docker

Customize Metabse UI without modifying the core code

Metabase is a tool that allows everyone in your company to ask questions and learn from data: handy when it comes to generate reports, charts and export them into Excel etc..

While it is a good tool but I lacks easy basic ways to customize its UI without forking the repository.. Here I show a workaround to bypass the Content-Security-Policy header in Metabase that restricts loading external style sheets.

DISCLAIMER: Metabase license doesn’t allow this so please use only on your dev environment for education purposes only or buy a licence.

We will be using the Apache headers module for that so lets enable it in Apache

sudo a2enmod headers

Next we need to configure the virtual host for our Metabase, mine was in:

sudo nano /etc/apache2/sites-available/reports.example.com-le-ssl.conf

Then we need to see the current header set by Metabase by using PostMan and making a get request to: https://reports.example.com/questions

Note down the current “Content-Security-Policy” header and append your style source as in:

# Allow loading styles from my server
Header set Content-Security-Policy "default-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'self' https://maps.google.com https://apis.google.com https://www.google-analytics.com https://*.googleapis.com *.gstatic.com ; child-src 'self' https://accounts.google.com; style-src 'unsafe-inline' 'self' fonts.googleapis.com app.example.com; font-src 'self' fonts.gstatic.com themes.googleusercontent.com ; img-src * 'self' data:; connect-src 'self' metabase.us10.list-manage.com"

Check this list for parameters and their meanings.

And insert it to your vhost conf under:

<VirtualHost *:443>
ServerName reports.example.com
<HERE>

Now we will hijack one of the metabase style and append our custom style to it: since/app/dist/styles\.bundle\.css wis the last stye in Metabase I will use it for my hijacking.

Add this to the vhosts:

# RedirectPermanent Stylesheet
RewriteEngine on
RewriteRule "^/app/dist/styles\.bundle\.css$" "https://example.com/resources/css/reports.min.css" [R]

Test the configuration

sudo apache2ctl configtest

Restart the server

sudo service apache2 restart

PHP_CodeSniffer: Code Beautifier and Fixer, Symfony and PhpStorm

PHP_CodeSniffer is a set of two PHP scripts; the main phpcs script that tokenizes PHP, JavaScript and CSS files to detect violations of a defined coding standard, and a second phpcbf (PHP Code Beautifier and Fixer) script to automatically correct coding standard violations. PHP_CodeSniffer is an essential development tool that ensures your code remains clean and consistent.

In this step by step tutorial I show you how I use this tools to setup my PhpStorm development environment.

Requirement

Make sure composer bin is in your $PATH directory: check your current path with echo $PATH

Install PHP_CodeSniffer

If you use Composer, you can install PHP_CodeSniffer system-wide with the following command:
composer global require "squizlabs/php_codesniffer=*"

This will install PHP_CodeSniffer and all its dependencies into the ~/.composer/vendor/ directory and, most importantly, the PHP_CodeSniffer CLI tools are installed into ~/.composer/vendor/bin/.

See if the package is installed globally: composer global show

If you didn’t do it before, simply add ~/.composer/vendor/bin/ to your PATH in your ~/.bash_profile (or ~/.bashrc) like this:

export PATH=~/.composer/vendor/bin:$PATH

Now phpcs and phpcbf are now available on your command line:

phpcs -h

phpcbf -h

To keep your tools up to date, you simply do this:

composer global update

To remove a package, you edit ~/.composer/composer.json and then run composer global update.

Install Symfony code style

As described on the instructions here: Install the coding standard also system-wide:

composer global require --dev escapestudios/symfony2-coding-standard:3.x-dev

Add Symfony code style:

~/.composer/vendor/bin/phpcbf --config-set installed_paths ~/.composer/vendor/escapestudios/symfony2-coding-standard

Check if code style installed:

~/.composer/vendor/bin/phpcbf -i

You should get something like:
The installed coding standards are PEAR, Zend, PSR2, MySource, Squiz, PSR1 and Symfony

Configure PhpStorm

Open the PhpStorm settings and go to Tools => External Tools, and add a new one with this config settings (change your username!):

Name: phpcbf single file
Description: Fix PHP Codesniffer warnings using PHP Code Beautifier and Fixer
Program: /Users/aidrissi/.composer/vendor/bin/phpcbf
Parameters: –report=full –report-file=$ProjectFileDir$/var/logs/phpcbf.log –standard=Symfony $FileDir$/$FileName$
Working directory:  $ProjectFileDir$

PhpStorm settings window: add new external tool

You can assign a keyboard shortcut for this new tool to format the current open document.

Open the settings again and navigate to: Keymap and search for phpcbf-single file and assign your preferred shortcut: In my case I use Command + Shift + B

PhpStorm run window: results of the run tool

Enjoy 🙂

Metabase as a service for Ubuntu

This is how I installed Metabase in Ubuntu using systemd and services: I am using MySQL and configured the service to get started at system boot time and have the possibility to stop it when I want..

Install Metabase

Downloaded the jar file from metabase.com and put it inside /var/www/metabase

wget http://downloads.metabase.com/v0.229.0/metabase.jar -P /var/www/metabase

 

Create the service

sudo nano /etc/systemd/system/metabase.service

 

Paste the following inside this file:

[Unit]
Description=Metabase service
[Service]
User=root
# The configuration file application.properties should be here:
#change this to your workspace
WorkingDirectory=/var/www/metabase
#path to executable: executable is a bash script which calls jar file
ExecStart=/var/www/metabase/metabase
SuccessExitStatus=143
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

 

Create a Bash Script to call the service

I created the bash file inside my metabase folder:

sudo nano /var/www/metabase/metabase

 

I prefer to use MySQL instead of H2 default database so: I added the necessary environment variables for that in my script.

#!/bin/sh
sudo /usr/bin/java -DMB_DB_TYPE=mysql -DMB_DB_DBNAME=metabase -DMB_DB_PORT=3306 -DMB_DB_USER=<username> -DMB_DB_PASS="<password>" -DMB_DB_HOST=localhost -jar metabase.jar

 

Give it execute permission:

sudo chmod u+x /var/www/metabase/metabase

 

Start the Metabase service

sudo systemctl daemon-reload
sudo systemctl enable metabase.service
sudo systemctl start metabase
sudo systemctl status metabase

Upgrading Metabase

Suppose I want to upgrade to version 0.29.0 then I do the following:

sudo wget http://downloads.metabase.com/v0.29.0/metabase.jar -P /var/www/metabase -O/var/www/metabase/metabase-new.jar && sudo service metabase stop && sudo mv metabase.jar metabase-old.jar && sudo mv metabase-new.jar metabase.jar && sudo service metabase start

Ansible role to generate unlimited SSL certificates

This is the task for SSL generation:

- name: Create ssl directory
  file: path={{params['ssl'].folder}} state=directory group={{ apache_group }} owner={{ apache_user }}
  become: yes

# generate your own: https://deliciousbrains.com/ssl-certificate-authority-for-local-https-development/
- name: Copy CA certificate and CA private key
  copy:
    src: "{{ item }}"
    dest: "{{params['ssl'].folder}}"
    mode: u+rw,g+r,o+r
  with_fileglob:
    - files/*
  become: yes

- name: Creating private keys for hosts
  shell: openssl genrsa -out {{params['ssl'].folder}}/{{ item.host }}.key 2048
  become: yes
  args:
    executable: "/bin/bash"
  with_items: "{{params['vhosts']}}"
  when: item.ssl|default(true)|bool == true

- name: Creating CSR for hosts
  shell: openssl req -new -key {{params['ssl'].folder}}/{{ item.host }}.key -nodes -out {{params['ssl'].folder}}/{{ item.host }}.csr -subj "/C={{params['ssl'].country_name}}/ST={{params['ssl'].state}}/L={{params['ssl'].locality}}/O={{params['ssl'].organization}}/CN={{ item.host }}"
  become: yes
  args:
    executable: "/bin/bash"
  with_items: "{{params['vhosts']}}"
  when: item.ssl|default(true)|bool == true

- name: Generate the Subject Alternative Name (SAN) extension per host
  template:
    src: ssl/extfile.ext
    dest: "{{params['ssl'].folder}}/{{ item.host }}.ext"
  become: yes
  with_items: "{{params['vhosts']}}"
  when: item.ssl|default(true)|bool == true

- name: Create the certificate per host
  shell: openssl x509 -req -in {{params['ssl'].folder}}/{{ item.host }}.csr -CA {{params['ssl'].folder}}/root_certificate_authority.pem -CAkey {{params['ssl'].folder}}/ca_key.key -passin pass:{{params['ssl'].pass_phrase}} -CAcreateserial -out {{params['ssl'].folder}}/{{ item.host }}.crt -days 1825 -sha256 -extfile {{params['ssl'].folder}}/{{ item.host }}.ext
  become: yes
  args:
    executable: "/bin/bash"
  with_items: "{{params['vhosts']}}"
  when: item.ssl|default(true)|bool == true

 

This is the parameters I used:

vhosts:
  - { projectFolder: "mylocalfolder", host: "azul.dev", framework: "symfony", alias: "az" }
ssl:
    pass_phrase: "vagrant"
    folder: "/etc/apache2/ssl"
    country_name: "MA"
    state: "Ouarzazate"
    locality: "Ouarzazate"
    organization: "Tamazgha Ltd"