Friday, October 7, 2016

Web SSO - part 2 - integration with LDAP

In my previous article I presented how to create a web SSO system using SimpleSAMLphp and Symfony. The users where declared directly in simpleSAMLphp using "exampleauth:UserPass".

In many companies a LDAP server is the source from where information about user authentication is taken. I will install  OpenLDAP and configure my applications to use it.

1. Install OpenLDAP and phpLDAPadmin

For installing OpenLDAP and phpLDAPadmin I followed this tutorials from DigitalOcean:

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-openldap-and-phpldapadmin-on-an-ubuntu-14-04-server

You will need to edit also the ldap.conf file, see this thread on StackOverflow

Also you may get an error when trying to login with phpLDAPadmin  "Notice: Undefined variable: _SESSION in ..". 
For me this solution from StackOverflow solved the problem:

"Just add the user wich is running Apache2 (or php5-fpm!) to the system group "www-data" (debian) and restart services apache AND if used php5-fpm both.
Get the User apache is running as:

~# sed -rn 's/^User (.+)/\1/p' /etc/apache2/apache2.conf"

Using phpLDAPadmin I've created two groups "admin" and "regular_users" and also I've created some users allocated to these two groups.

2. Modify SimpleLDAPphp to use OpenLDAP


The documentation for using LDAP authentication is found here: https://simplesamlphp.org/docs/stable/ldap:ldap

My settings are:


Select LDAP authentication to be used from   /metadata/saml20-idp-hosted.php

 /* 
         *Authentication source to use. Must be one that is configured in
* 'config/authsources.php'.
*/
'auth' => 'example-ldap',

3. Modify Symfony app

In the current Symfony application I am expecting an attribute roles containing an array of roles. From LDAP I will receive different attributes, one of them is gidNumber, which is a number identifying a group. My current groups: admin and regular_users have gidNumber 500 and 501.
I will be using these gidNumbers to correctly create roles in the Symfony application.

The changes to be made are done in the UserCreator class:


Of course you need to change these mappings to fit your situation.

4. Test

First make sure to delete any sessions and cookies. After that try to access the secure route from consumer1.local, login with any user from LDAP and you should be redirected to secure area. Check in database if the user and user roles were created correctly.

Web SSO with Symfony and SimpleSAMLphp

1. Single sign on (SSO)


Sooner or later web development teams face one problem: you have developed an application at domain X and now you want your new application at domain Y to use the same login information as the other domain. In fact, you want more: you want users who are already logged-in at domain X to be already logged-in at domain Y. This is what SSO is all about. (source: https://auth0.com/blog/what-is-and-how-does-single-sign-on-work )

2. Concepts:


Identity Provider (IdP)  - is responsible for (a) providing identifiers for users looking to interact with a system, (b) asserting to such a system that such an identifier presented by a user is known to the provider, and (c) possibly providing other information about the user that is known to the provider.

ServiceProvider (SP) - it can be any application. In order to access the secure areas of the application the user need to be authenticated and authorized by a IdP

SAML  - Security Assertion Markup Language (SAML, pronounced sam-el) is an XML-based, open-standard data format for exchanging authentication and authorization data between parties, in particular, between an identity provider and a service provider. SAML is a product of the OASIS Security Services Technical Committee.

3. The flow


- the USER  requests a secure area from ServiceProvider
- if the user is not authenticated it is redirected to IdP with some information about SP
- the USER fills his credentials on IdP and after successful authentication is redirected back to SP
- based on the answer from IdP the SP creates the a new user or identify an existing one in his own database and creates a session.

4. Tools


SimpleSAMLphp - it can be used both as IdP and SP, I will be using it only as IdP
lightsaml/SP-Bundle - Symfony Bundle implementing the ServiceProvider

5. Implementation


5.1. Create and configure IdP

Install simpleSAMLphp following the documentation. I've created a virtualhost idp.local and the simpleSAMLphp is available at : http://idp.local/simplesamlphp

Next step is to configure simleSAMLphp as IdentityProvider following the quick guide using the exampleauth:UserPass authentication method.

I'll modify the defined users in config/authsources.php, by replacing the attribute "eduPersonAffilication" with 'roles':

      'student:studentpass' => array(
             'uid' => array('student'),
             'roles' => array('ROLE_USER', 'ROLE_SEF'),
         ),

To make it clear, you have just created a user "student" with password "studentpass". 
Skip the step 6 from the quick guide.
At the step 7 you need to enter the information about ServiceProviders. At this moment they do not exist, but they will, so you can fill the following 2 service providers:

$metadata['http://consumer1.local/saml'] = array(
    'AssertionConsumerService' => 'http://consumer1.local/app_dev.php/saml/login_check',
    'SingleLogoutService'      => 'https://consumer1.local/app_dev.php/logout',
    'simplesaml.nameidattribute' => 'uid',
);

$metadata['http://consumer2.local/saml'] = array(
    'AssertionConsumerService' => 'http://consumer2.local/app_dev.php/saml/login_check',
    'SingleLogoutService'      => 'https://consumer2.local/app_dev.php/logout',
    'simplesaml.nameidattribute' => 'uid',
);

I called them consumers because they consume the authentication service provided by IdP.
Skip steps from 8 to 11.

5.2 Create and configure SPs

Install Symfony and create a virtual host consumer1.local
We will be following the lightsaml/SP-Bundle documentation found here.

At step 7 configure your own entity_id and use the previously created IdP.

 -  First go to your IdP, it should be idp.local/simplesamlphp. Click on the Federation tab, you should see somewhere  SAML 2.0 IdP Metadata. Click on [Show metadata], copy the XML and createa file in your Symfony app: /src/AppBundle/Idp/idp.local.xml and paste the XML.
-  edit app/config.yml



At step 9 (and 11), please see below my security.yml:





Where /secure is a  just route I created for testing purpose. 

Step 10. Basically if the user logged in IdP does not exist in the database of the SP, in this case consumer1.local, it needs to be created.  For this reason a UserCreator class is made. In the documentation this class is able to identify the user id (uid) using a "username mapper" service. I will enhance that by adding an "attributes" mapper which will create an array with the other attributes passed from IdentityProvider. I want to pass a list of roles from IdP to SP.  If you remember I've added a list of 'roles' to my 'student' user. Below you can find a gist with the code for the AttributeMapper










 


Declare the attribute mapper service in app/services.yml:


attribute_mapper:
   class: AppBundle\Security\User\AttributeMapper

Modify the UserCreator class:



Declare the User Creator service injecting in it our Attribute Mapper service:

    user_creator:
        class: AppBundle\Security\User\UserCreator
        arguments:
            - "@=service('doctrine').getManager()"
            - "@lightsaml_sp.username_mapper.simple"
            - "@inno.attribute_mapper"
Also inject the attribute mapper service into the security.authentication.provider.lightsaml_sp service:



    security.authentication.provider.lightsaml_sp:
        class: LightSaml\SpBundle\Security\Authentication\Provider\LightsSamlSpAuthenticationProvider
        arguments:
            - ~ # provider key
            - ~ # user provider
            - ~ # force
            - "@security.user_checker"
            - "@lightsaml_sp.username_mapper.simple" # username mapper
            - ~ # user creator
            - "@inno.attribute_mapper" # attribute mapper
            - ~ # token factory
        abstract: true

5.3 Put your application at work

In browser try to access consumer1.local/app_dev.php/secure, because you are not authenticated
 you should be redirected to http://consumer1.local/app_dev.php/saml/discoveryClick on your IdP,
 you will be redirected to the IdP site where you will fill your user and password (student and 
studentpass. After that you are redirected to the secure page on cosumer1.local.You can check your
 database to see the newly created user. Now copy your Symfony installation and make another 
virtual host consumer2.local. Open the browser and try to access: consumer2.local/app_dev.php/securePick
 your IdP from discovery page, because you are already logged on IdP you will not be asked for 
password again, instead you are redirected to the secure page. Victory!In the next blog post I will
 investigate integrating LDAP with SimpleSAMLphp.







Wednesday, September 14, 2016

Using Silex and Symfony together in same application

I am testing the idea of having an application using both Silex and Symfony.

Why ?

Maybe because I want for some parts of the application to be really fast, and for the most of the application to enjoy the productivity of using a full stack framework and have access to all the third party bundles.

Of course having 2 application may introduce other kind of problems.

How my application should work:

- receive request
- fast Silex kernel handles the request.
     - if the URL request is not matching any of the routes from Silex, it will return a response with status 404
    -  if the URL is matching a route, execute controller and get response
- check what is the response from Silex
    - if the response has status 404 pass it to the Symfony kernel
   -  otherwise return response

Using the 2 frameworks together is possible because they both use the same abstraction for HTTP Request and Response and the HttpKernel.
There is a project called StackPHP that promotes interoperability between applications based on the HttpKernelInterface.

Implementation:


Create a probject folder under /var/www/public called double: /var/www/public/double
I will be using Symfony Standard Edition and Silex Skeleton project by Fabien Potencier: https://packagist.org/packages/fabpot/silex-skeleton

Install using Composer in two different subfolders:

   composer create-project fabpot/silex-skeleton  silex  ~2.0@dev
   composer create-project symfony/framework-standard-edition symf


Create a virtual host that will point to /var/www/public/double/symf/web
You may want to first save Symfony app.php and app_dev.php before editing them:
app_dev.php :

<?php

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Debug\Debug;
use Silex\Application;



/**
 * @var Composer\Autoload\ClassLoader $loader
 */
$loader = require __DIR__.'/../app/autoload.php';
require_once __DIR__.'/../../silex/vendor/autoload.php';

Debug::enable();

//create request object
$request = Request::createFromGlobals();

//initialize Silex app
$app = require __DIR__.'/../../silex/src/app.php';
require __DIR__.'/../../silex/config/dev.php';
require __DIR__.'/../../silex/src/controllers.php';
//handle request with Silex app
$response = $app->handle($request); if ($response->getStatusCode() === 404) {
    //initialize Symfony app and handle request
    $kernel = new AppKernel('dev', true);
    $kernel->loadClassCache();
    $response = $kernel->handle($request);
    $response->send();
    $kernel->terminate($request, $response);

} else {
    $response->send();
    $app->terminate($request, $response);
}


Into app.php I've added microtime() function to get the loading time in production as I do not have access to the Web Profiler.


<?php

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Silex\Application;

$time_start = microtime(true);
/**
 * @var Composer\Autoload\ClassLoader $loader
 */
$loader = require_once __DIR__.'/../app/autoload.php';
require_once __DIR__.'/../../silex/vendor/autoload.php';


//create request object
$request = Request::createFromGlobals();

//initialize Silex app
$app = require __DIR__.'/../../silex/src/app.php';
require __DIR__.'/../../silex/config/prod.php';
require __DIR__.'/../../silex/src/controllers.php';

$response = $app->handle($request);

if ($response->getStatusCode() === 404) {


    $kernel = new AppKernel('prod', false);
    $kernel->loadClassCache();
    $response = $kernel->handle($request);
    $response->send();
    $kernel->terminate($request, $response);

    $time_end = microtime(true);
    $time = $time_end - $time_start;

    echo "Execution time: $time seconds\n";

} else {
    $response->send();
    $app->terminate($request, $response);

    $time_end = microtime(true);
    $time = $time_end - $time_start;

    echo "Execution time: $time seconds\n";
}


I did some simple tests using the production env (app.php) and generally Silex is 3 times faster than Symfony, which is no surprise as it is lighter.
In Silex I used Doctrine DBAL (not ORM), the provider it is included in the Silex Skeleton project, just needs to be configured in src/app.php:



$app->register(new DoctrineServiceProvider(), array(
    'dbs.options' => array (
        'localhost' => array(
            'driver'    => 'pdo_mysql',
            'host'      => 'localhost',
            'dbname'    => 'symfony',
            'user'      => 'root',
            'password'  => 'root',
            'charset'   => 'utf8',
        )
    ),
));

Thursday, September 8, 2016

Multilingual website with Symfony

If you want to create a multilingual website, Symfony together with the community bundles have the tools for a fast set up.

When creating a multilingual website we need to take in account translating:

  1. Specific routes per language
  2. Menus, labels and forms
  3. The content of the website
  4. Extra: translate FOSUserBundle

1. Routes 

It may be tempting to use the same URL to display a resource in different languages based on the user's locale. For example, http://www.example.com/contact could show content in English for one user and French for another user. Unfortunately, this violates a fundamental rule of the Web: that a particular URL returns the same resource regardless of the user. To further muddy the problem, which version of the content would be indexed by search engines?
A better policy is to include the locale in the URL. This is fully-supported by the routing system using the special _locale parameter, see documentation:

The term locale refers roughly to the user's language and country. It can be any string that your application uses to manage translations and other format differences (e.g. currency format). The ISO 639-1 language code, an underscore (_), then the ISO 3166-1 alpha-2 country code (e.g. fr_FR for French/France) is recommended.

In config.yml add the following:

parameters:    locale: en_GB
    app.locales: en_GB|es_ES|fr_FR

framework:
    translator:  { fallbacks: ["%locale%"] }

By declaring the list of accepted as parameter you can easily use it in all routes declarations:

/** 
 * @Route("/{_locale}/product",  
 *     name="product",  
 *     requirements={ "_locale" = "%app.locales%" }) 
 */
public function productAction(Request $request)

By doing this, a route will look like this : www.mysite.com/en_GB/product

 2. Translations - labels, menus etc.

Translation of text is done through the translator service (Translator). To translate a block of text (called a message), use the trans() method, or in Twig using trans and transchoice tags.
The recommended way is to have message placeholders which will be translated using a translation file (one per each language).  An example would be a menu button that in english should show "About". We can create a placeholder named:  "menu.about" which will be translated in Twig:

     {{ 'menu.about'|trans }}

The translation files can have different formats (like YAML of XLIFF) and they live usually in app/Resources/translations, or under Resources/translations in your bundle.

I will be using YAML,  so I will create the files "messages.en_GB.yml" , "messages.es_ES.yml" etc.
Inside I add the placeholder and the translation:

button.product.view_detail : View detail

3. Content translation

One solution would be to create yourself the entities and handle the process. Imagine having a Product entity containing fields that do not need translation, like Id, Price and another entity ProductTranslation containing translatable fields like Name and Description. These two will be in a relation (example: OneToMany).

Another way is to use some community bundles that handle these process for us. I will be using KnpLabs/DoctrineBehaviors  and A2Lix TranslationBundle

Install DoctrineBehaviors and add translatable: true to config.yml  :

knp_doctrine_behaviors:    ...
    translatable:   true


Create your entities following the tutorial from documentation. Be careful that there are two different traits to be used in Entity and EntityTranslation classes.Do not add id field to the EntityTranslation class as it will be taken care by the traits. Update database structure:  php bin/console doctrine:schema:update  --force
I am using DoctrineFixturesBundle to add some data. For adding an apple(fruit) product the code is:


$product = new Product();
$product->translate('en')->setName('Apple ');
$product->translate('es')->setName('Manzana ');
$product->translate('en')->setDescription('Sweet apple');
$product->translate('es')->setDescription('Manzana dulce ');
$product->setImage('apple.jpeg');
$product->setCurrency($currency);
$product->setPrice(1.00);
$product->setActive('1');
$manager->persist($product);
$product->mergeNewTranslations();



In the database, I have a product_translation table where the information is saved. 
In controller I am retrieving the product object just as usual:


    $product = $em->getRepository('AppBundle:Product')->find($id);

In order to display the translatable fields in the current language in Twig I use:

    {{ product.translate(app.request.locale).name }}


For handling translation from the website interface you can use A2Lix TranslationBundle which will create a form with tabs for each translation (I've installed the version 3.x)

I've generated a ProductType class and added another line in code after adding my non translatable fields:  


use A2lix\TranslationFormBundle\Form\Type\TranslationsType;
...
$builder    ->add('active')
    ->add('price')
    ));
$builder->add('translations', TranslationsType::class);


4. Extra: translate FOSUserBundle

FOSUserBundle is one of the most popular Symfony bundles. You can easily activate translation by following the instructions from the link below:

https://codereviewvideos.com/course/getting-started-with-fosuserbundle/video/translations-and-internationalisation-in-fosuserbundle

Friday, September 2, 2016

Replace Symfony reverse proxy with Varnish

In my last post I've presented how to start caching pages with the reverse proxy shipped with Symfony Standard Edition. But as it is mentioned in the documentation the built in PHP proxy is not as fast as Varnish for example. So today I will do a small test with Varnish instead of the Symfony reverse proxy (AppCache).

First install Varnish on virtual machine with Ubuntu (I am using https://box.scotch.io ):

    sudo apt-get install varnish -y

Edit Varnish configuration file found at: /etc/varnish/default.vcl. I have a virtual host called myshop.dev, I will put that as host.

Example of Varnish configuration file for Symfony:

backend default {
    .host = "myshop.dev";
    .port = "80";
}

sub vcl_fetch {
    /* By default, Varnish3 ignores Cache-Control: no-cache and private
      https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html#cache-control
     */
    if (beresp.http.Cache-Control ~ "private" ||
        beresp.http.Cache-Control ~ "no-cache" ||
        beresp.http.Cache-Control ~ "no-store"
    ) {
    return (hit_for_pass);
    }

}

sub vcl_recv {
     unset req.http.Forwarded;

   if (req.http.Cookie) {

        set req.http.Cookie = ";" + req.http.Cookie;
        set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
        set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
        set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
        set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
        if (req.http.Cookie == "") {
            unset req.http.Cookie;
        }
    }


    // Add a Surrogate-Capability header to announce ESI support.

    set req.http.Surrogate-Capability = "abc=ESI/1.0";
}

sub vcl_fetch {

    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        unset beresp.http.Surrogate-Control;
        set beresp.do_esi = true;
    }
}

sub vcl_deliver {

  if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS
    set resp.http.X-Cache = "HIT";
  } else {
    set resp.http.X-Cache = "MISS";
  }
  # show how ofthe the object created a hit so far (reset on miss)
  set resp.http.X-Cache-Hits = obj.hits;
}

Configure Symfony config.yml file:

framework:
    esi:   { enabled: true }
    trusted_proxies: [127.0.0.1]
    fragments:  { path: /_fragment }

With these configuration in place you can open the browser and  call your page via Varnish on port 6081:

http://myshop.dev:6081/

If is not working you can try to restart Varnish service:  sudo service varnish restart


One source of information:  http://by-examples.net/2014/12/19/speeding-up-symfony-with-varnish.html

Introduction to Symfony HTTP Cache

You can start using the HTTP Gateway Cache that is shipped with Symfony by editing app.php file:

    $kernel = new AppKernel('prod', true);
    $kernel->loadClassCache();
    // wrap the default AppKernel with the AppCache one
    $kernel = new AppCache($kernel);

I've put the debug mode to be able to see the X-Symfony-Cache header in Firebug. You may want to delete the cache for prod environment before  refreshing the page. When loading the page, you can see in Firebug the Response headers:

    Cache-Control       no-cache
    X-Symfony-Cache   GET /: miss

So we do not have any Cache control directives set and our reverse proxy says it cannot find the path "/" in the list of cached information.

Let's make the response cacheable using Cache Expiration strategy.

In controller:

    $response =  $this->render('default/index.html.twig');
    $response->setSharedMaxAge(30);

    // (optional) set a custom Cache-Control directive
    $response->headers->addCacheControlDirective('must-revalidate', true);
    return $response;

Delete the cache and reload the page:

 First time: loading time is 4.28s according to FireBug
 Response headers:

     Age 0
     X-Symfony-Cache GET / : miss, store

 Reload second time: loading time is 82ms

     Age 9
     X-Symfony-Cache GET/ : fresh

 Reload third time loading time is 1.86 s

    Age 0
    X-Symfony-Cache GET / : stale, invalid, store

Things work as expected, the response time lowered to 82ms.


Now I will be testing the HTTP Cache Validation strategy.

     /**
     * @Route("/", name="homepage_symfony")
     */    
    public function indexAction(Request $request)
    {
        $text = "Welcome!";
        $response = new Response(); 
        $response->setETag(md5($text)); 
        $response->setPublic(); // make sure the response is public/cacheable
        if ($response->isNotModified($request)) { 
            return $response; 
        };

        $response->setContent($this->renderView('default/index.html.twig', array('content'         => $text)));
        return $response;
   }


First delete cache. Than reload page in browser:

First time:
X-Symfony-Cache  GET /: miss, store

Second time:
X-Symfony-Cache  GET /: stale, valid, store

At this moment there is not a big improvement as we are not avoiding any complex logic, just that instead of rendering the page with Twig we are serving it from cache.


Expiration and Validation 

You can of course use both validation and expiration within the same Response. As expiration wins over validation, you can easily benefit from the best of both worlds. In other words, by using both expiration and validation, you can instruct the cache to serve the cached content, while checking back at some interval (the expiration) to verify that the content is still valid. (from Symfony doc)

I will add setSharedMaxAge(30)to the controller resulting:

public function indexAction(Request $request)
{
    $text = "Bun venit pe pagina";
    $response = new Response();
    $response->setETag(md5($text));
    $response->setPublic(); // make sure the response is public/cacheable
    $response->setSharedMaxAge(30);
    if ($response->isNotModified($request)) {  
          return $response;
    }; 
   $response->setContent($this->renderView('default/index.html.twig', array('content' => $text)));
    return $response;
}

For 30 seconds the cache will be fresh and the page will be served from cache. After that using the ETag the freshness of the cache will checked and if it is still fresh the controller will return a 304 status with an empty content to the HTTP Cache which will serve again the response from cache.

First time loading the page:

Age               0
Cache-Control  public, s-maxage=30
Etag    "9c8751de832e5a472655e87731c49419-gzip"
X-Symfony-Cache  GET /: miss, store

Second Time:

Age   5
X-Symfony-Cache   GET /: fresh

After more than 30s

Age     5 
X-Symfony-Cache GET /: stale, valid, store


Monday, August 22, 2016

Symfony magic: generated PHP files

1.  bootstrap.php.cache


In versions 3.x is found at var/bootstrap.php.cache before was in app/ directory.

Loading all of classes from separate files on each request can result in some overhead. To reduce this overhead, the Symfony Standard Edition provides a script to generate a so-called bootstrap file, consisting of multiple classes definitions in a single file. By including this file (which contains a copy of many of the core classes), Symfony no longer needs to include any of the source files containing those classes. This will reduce disc IO quite a bit.

If you're using the Symfony Standard Edition, then you're probably already using the bootstrap file. To be sure, open your front controller (usually app.php) and check to make sure that the following line exists:

include_once __DIR__.'/../var/bootstrap.php.cache'

Note that there are two disadvantages when using a bootstrap file:

 - the file needs to be regenerated whenever any of the original sources change (i.e. when you update the Symfony source or vendor libraries)
 - when debugging, your breakpoints are never reached because the code is executed from bootstrap.php.cache. I wrote in this article about how to handle this.

If you're using the Symfony Standard Edition, the bootstrap file is automatically rebuilt after updating the vendor libraries via the composer install command.

See: http://symfony.com/doc/master/performance.html#use-bootstrap-files


2.  classes.php and classes.map


From Alan Storm blog:  "The generation of classes.php is a bit complicated. First, this file is not generated as part of Symfony's cache warmup. You can call

php app/console cache:clear

all you like, but that won't regenerate classes.php. Instead, you need to load an actual application page, or otherwise boot your application kernel.

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
public function boot()
{
    //...
    if ($this->loadClassCache) {
        $this->doLoadClassCache($this->loadClassCache[0], $this->loadClassCache[1]);
    }
    //...
}    

It's the call to doLoadClassCache which triggers a call to ClassCollectionLoader::load

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
protected function doLoadClassCache($name, $extension)
{
    if (!$this->booted && is_file($this->getCacheDir().'/classes.map')) {
        ClassCollectionLoader::load(include($this->getCacheDir().'/classes.map'), $this->getCacheDir(), $name, $this->debug, false, $extension);
    }
}

The work of generating classes.php is done in ClassCollectionLoader::load. Simple enough — except you need to pass in a list of classes you want to combine. These classes come from the classes.map file.

app/cache/dev/classes.map
app/cache/prod/classes.map

Of course, now our question is "where does classes.map come from?". The classes.map file is generated at the same time as the app container file, right after the call to $container->compile() in initializeContainer.

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
protected function initializeContainer()
{
    //...
    if (!$cache->isFresh()) {
        //...
        $container->compile();
        //...
    }
    //...
}

3.  The Dependency Injection Container(DIC) class file


In a simplified way you can think about the  DIC as an array, having as keys the names of the services, and as values, anonymous functions that create a new instance of the needed class ( have a look at Pimple ). Symfony DIC is more complex, it allows services to interact using tagged services, and this interaction is implemented using some classes called Compiler Passes. Compiler Passes manipulate the services, and they are executed when the DIC is compiled.

In Symfony you do not edit a class to register your services directly into an array, instead you edit different configuration files (YML, XML). Each extra bundle that you install may add its own services that need to be registered. Also Compiler Passes need to be executed. 
All these haveto be "compiled" in order to result a PHP class file containing a big array of services and anonymous functions that instantiate new objects. The compilation process has loaded the services from the configuration, extensions and the compiler passes, the result is dumped so that the cache can be used next time. The dumped version is then used during subsequent requests as it is more efficient. The generated file can be found in the following locations for the dev and prod environments:
var/cache/dev/appDevDebugProjectContainer.php
var/cache/prod/appProdProjectContainer.php

If you're creating new services in Symfony, you'll often need to re-generate you container file. Doing so is as simple as clearing your Symfony cache — the Symfony kernel will regenerate these files automatically. If you're curious, the code that does this starts here

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
protected function initializeContainer()
{
    $class = $this->getContainerClass();
    $cache = new ConfigCache($this->getCacheDir().'/'.$class.'.php', $this->debug);
    $fresh = true;
    if (!$cache->isFresh()) {
        $container = $this->buildContainer();
        $container->compile();
        $this->dumpContainer($cache, $container, $class, $this->getContainerBaseClass());

        $fresh = false;
    }

    require_once $cache;

    $this->container = new $class();
    $this->container->set('kernel', $this);

    if (!$fresh && $this->container->has('cache_warmer')) {
        $this->container->get('cache_warmer')->warmUp($this->container->getParameter('kernel.cache_dir'));
    }
}

4.  Twig templates


Before inspecting how Twig files are generated, you can read this article from Fabien Potencier on why do we need Twig in first place.

How does Twig work?

   1. Load the template: If the template is already compiled, load it and go to the evaluation step, otherwise:
        1.1. First, the lexer tokenizes the template source code into small pieces for easier processing;
        1.2. Then, the parser converts the token stream into a meaningful tree of nodes (the Abstract Syntax Tree);
        1.3. Eventually, the compiler transforms the AST into PHP code. This step is generating PHP  c    classes that extends Twig_Template class and contain a function called "doDisplay()".
The files are found under cache/twig.
    2. Evaluate the template: It basically means calling the display() method of the compiled template and passing it the context.

Compilation is a costly process, so the result is cached in the directory defined by this configuration option. You can set option "cache" to null to disable Twig template compilation. However, this is not recommended; not even in the dev environment, because the auto_reload option ensures that cached templates which have changed get compiled again.

auto_reload

If true, whenever a template is rendered, Symfony checks first if its source code has changed since it was compiled. If it has changed, the template is compiled again automatically. (source: http://symfony.com/doc/current/reference/configuration/twig.html).

5. Routes


In Symfony you can declare routes in different ways YAML, XML, Annotation, all these are in the end compiled to PHP. You can have a look at the file appDevUrlMatcher.php under /app/cache directory for Symfony 2. In dev environment this file regenerated if you are doing changes to routes, in prod environment you need to clear the cache in order to regenerate it. Have a look at this presentation for more info: http://davidbu.ch/slides/2015-12-03-symfony-routing.html

Tuesday, August 9, 2016

Consuming a SOAP web service with PHP

I started programming with PHP at the beginning of 2015, and when is about web services, everything I've read is about RESTful services.

But guess what, is 2016 I will have to interact with a SOAP web service. Doing a research I've discovered this very informative article from Benjamin Eberlei:

http://www.whitewashing.de/2014/01/31/soap_and_php_in_2014.html

Take your time to read it.

OK, now that we know what are we talking about, short example on how to consume a SOAP service. I will use a free SOAP web services for testing:  http://www.webservicex.net

I will test a simple request to this particular service: http://www.webservicex.net/New/Home/ServiceDetail/19 called Periodic table.

The link to the WSDL file is: http://www.webservicex.net/periodictable.asmx?WSDL

With this link and using the SoapClient class from PHP I will write:



<?php

$url = "http://www.webservicex.net/periodictable.asmx?WSDL";
$client = new SoapClient($url);

$fcs = $client->__getFunctions();
$types = $client->__getTypes();
var_dump($fcs);
//var_dump($types);

I do not really need to call __getFunctions() and __getTypes() but is useful for learning purpose.

The output should look like this:

array(8) {
  [3]=>
  string(71) "GetElementSymbolResponse GetElementSymbol(GetElementSymbol $parameters)
 

Where the types of data GetElementSymbolResponse and GetElementSymbol are data types defined in the WSDL file given as parameter to SoapClient constructor.
Let's look to the GetElementSymbol which needs to be passed to the SOAP server.


<s:element name="GetElementSymbol"> 
    <s:complexType>
        <s:sequence>
            <s:element minOccurs="0" maxOccurs="1" name="ElementName" type="s:string"/>
        </s:sequence>
    </s:complexType>
</s:element>

It expect an item called "ElementName". Make sense, let's say I want to find out the symbol for Iron in the periodic table.
Back to my PHP program:


<?php

$res = $client->GetElementSymbol(array('ElementName' => 'Iron'));
//var_dump($res);
echo "Symbol for Iron: ";
echo $res->GetElementSymbolResult;

Evrika! The server answered to our question, the symbol for Iron is "Fe". If you uncomment //var_dump($res); you can see that the response is an object from (stdClass) with one property.
 
 




Thursday, July 21, 2016

Using the Symfony Validator component as a standalone library

Sometimes I need to build some scripts without using a full framework like Symfony. The good news is that you can use individual components from Symfony. Most often I will be using the HTTPFoundation component, but for today I want to build a CLI script so no need for HTTPFoundation.
I  will be writing a small command line script to read and validate a  CSV file. For the  validation part I will be using the Symfony Validator Component,

Source of inspiration: https://blog.tinned-software.net/using-the-symfony-validator-as-a-standalone-component/


First step, install Symfony Validator using Composer:

                          composer require symfony/validator

and require the autoloader created by Composer.


<?php

require_once 'vendor/autoload.php';
require_once 'Loader.php';

use Symfony\Component\Validator\Validation;

$file = "someFile.csv";


$outputFile = "errorsLog" . "_" . time() . ".txt";
  
$validator = Validation::createValidatorBuilder()
 ->addMethodMapping('loadValidatorMetadata')
 ->getValidator()
;

$loader = new Loader($file, $outputFile, $validator);
$loader->load();

I will pass to my Loader class constructor the CSV file to be read, the file where I want to save the output and an instance of the Symfony validator.

The CSV file is read line by line, and from each line I will be creating a Row object. The validator will validate the Row object against the rules (constraints).
In the Row class I will add the method "loadValidatorMetada" mentioned when instantiating the Validator. I've added several validations for better examplification:


<?php

use Symfony\Component\Validator\Constraints as Assert;
use Symfony\Component\Validator\Mapping\ClassMetadata;


class Row
{
    private $id;         
    private $name;   
    private $personalNumber;    
    private $CountryCode;  

    public function __construct($rowNumber, $data)
    {
        $this->id = $rowNumber;
        $this->name= $data[0];
        $this->personalNumber = $data[2];
        $this->countryCode = $data[1];
    }
 
 
    /**
     * This method is where you define your validation rules.
    */
    public static function loadValidatorMetadata(ClassMetadata $metadata)
    {
        //Name 
        $metadata->addPropertyConstraint('name', new Assert\NotBlank());
 
        //Personal number
        $metadata->addPropertyConstraint('personalNumber', new Assert\NotBlank());
        $metadata->addPropertyConstraint('personalNumber', new Assert\Type(array(
            'type'    => 'digit',
            'message' => 'The value {{ value }} is not a valid personal number.',
         )));
  
         //Country code
         $metadata->addPropertyConstraint('countryCode', new Assert\NotBlank());
         $metadata->addPropertyConstraint('countryCode', new Assert\Choice(array(
            'choices' => array('DE', 'AT', 'LU', 'ES', 'FR', 'BE', 'NO', 'SI', 'SE', 'IT', 'DK'),
            'message' => '{{ value }} is not a valid country code!',
          )));
  
     }
}


When validating a Row object against these constraints an array of Errors will be returned by the Validator. Below is the Loader class.



<?php

require_once 'Row.php';


class Loader
{
 private $fileName;
 private $validator;
 private $outputFile;
 
 public function __construct($fileName, $outputFile, $validator)
 {
  $this->fileName = $fileName;
  $this->outputFile = $outputFile;
  $this->validator = $validator;

 }
 
 public function load()
 {
  $file_handle = fopen($this->fileName, "r");
  $fileOutputHandle = fopen($this->outputFile, "w");
  
  /* 
   * Keep track of the current row in .csv file
   */
  $rowNumber=1;
  
  /* 
   * Read the file line by line
   */
  while (!feof($file_handle) ) {

   $line_of_text = fgetcsv($file_handle, 0);
   
   if ($rowNumber === 1) {
    
      /*
       * Ignore the first row from file as it contains headers
       */
     
   } else {
       $row = new Row($rowNumber, $line_of_text);    
       $this->validateRow($row, $fileOutputHandle); 
   }
   
   $rowNumber++;
  }
  fclose($file_handle);
  fclose($fileOutputHandle);
 }
 
 
 public function validateRow(Row $row, $fileOutputHandle)
 {
  $errors = $this->validator->validate($row);
  
  foreach ($errors as $error) {
   $errorMsg = "At row:" 
      . $error->getRoot()->getId() 
      . "- Property: " 
      . $error->getPropertyPath()
      . ' - Message: ' 
      . $error->getMessage()
      . "\n";
   
   fwrite($fileOutputHandle, $errorMsg);

  }
 }
}