Doctrine Events: on Flush - get entity changes and persist new object

My situation is like this: I have an entity Ticket (like a support ticket) and an entity Action. I am using the Action objects to track any updates on Ticket objects.

Each time a Ticket is updated I want to create and persist a new Action object. The newly created Action object should contain the Ticket (there is a ManyToOne relation between Action and Ticket), and a note saying "The property X changed. The new value is: Y".

preUpdate

Looking over the available Doctrine events, my first thought was that preUpdate should be right for the job:

 preUpdate - The preUpdate event occurs before the database update operations to entity data. It is not called for a DQL UPDATE statement nor when the computed changeset is empty.

Reading more in detail the documentation I found that preUpdate event has many limitations:

- PreUpdate is the most restrictive to use event, since it is called right before an update statement is called for an entity inside the EntityManager#flush() method
- Changes to associations of the passed entities are not recognized by the flush operation anymore.
- Any calls to EntityManager#persist() or EntityManager#remove(), even in combination with the UnitOfWork API are strongly discouraged and don’t work as expected outside the flush operation.

According the documentation I cannot persist a new object in this event, which is exactly what I need, to persist a new Action object.

onFlush 

 

After some more reading the right event for job appeared:  onFlush

From documentation:

OnFlush is a very powerful event. It is called inside EntityManager#flush() after the changes to all the managed entities and their associations have been computed. This means, the onFlush event has access to the sets of:
      ...
     Entities scheduled for update 
     ...
If you create and persist a new entity in onFlush, then calling EntityManager#persist() is not enough. You have to execute an additional call to $unitOfWork->computeChangeSet($classMetadata, $entity).

So in this event I have access the UnitOfWork, which means I get access to all entities scheduled to be updated and also I can persist a new object doing that additional call. Great!

Implementation

 In my Symfony project I've created a new listener class called TicketListener with a method onFlush:

<?php 

 use Doctrine\ORM\Event\OnFlushEventArgs;

 class TicketListener {

    public function onFlush(OnFlushEventArgs $args)
    {

    }

And registered it as a service with Doctrine tag:

doctrine.ticket_listener:
        class: MyBundle\EventListeners\TicketListener
        arguments: []
        tags:
            - { name: doctrine.event_listener, event: onFlush }         

Using the OnFlushEventArgs we can access the Entity Manager and the UnitOfWork.

$em = $args->getEntityManager();
$uow = $em->getUnitOfWork();

$entities = $uow->getScheduledEntityUpdates();

From the entities scheduled to be updated I am interested only on those who are instance of Ticket entity.

if ($entity instanceof Ticket) { ..

 Using the UnitOfWork API we have access to the changes which happend to the Ticket object:

$changes_set = $uow->getEntityChangeSet($entity);

The method getEntityChangeSet($entity)   returns an array, where the keys are the name of the properties who changed.
When accessing an array key, you get another array with 2 positions [0] and [1], [0] contains the old value, [1] contains the new value.

In order to persist the new object, additionally to $em->persist() the following code need to be excuted

$classMetadata = $em->getClassMetadata('MyBundle\Entity\Action');
$uow->computeChangeSet($classMetadata, $action);

Below is the complete example:

    public function onFlush(OnFlushEventArgs $args)
    {
        $em = $args->getEntityManager();
        $uow = $em->getUnitOfWork();
        // get only the entities scheduled to be updated
        $entities = $uow->getScheduledEntityUpdates();

        foreach ($entities as $entity) {
            
            //continue only if the object to be updated is a Ticket
            if ($entity instanceof Ticket) {
                
                //get all the changed properties of the Ticket object
                $changes_set = $uow->getEntityChangeSet($entity);
                $changes = array_keys($changes_set);

                foreach ($changes as $changed_property) {
                    
                    $action = new Action();
                    $action->setTicket($entity);
                    $text = ucfirst($changed_property) . ' changed! New value: ' . $changes_set[$changed_property][1];
                    $action->setDescription($text);

                    $em->persist($action);
                    $classMetadata = $em->getClassMetadata('MyBundle\Entity\Action');
                    $uow->computeChangeSet($classMetadata, $action);
                }
            }
        }

This article was very helpful for me:  http://vvv.tobiassjosten.net/symfony/update-associated-entities-in-doctrine/

Serving protected files with Symfony2

If you just want to serve a file without any restriction you just generate a  path like this:

<a href="http://myvirtualhost.localhost/uploads/flower.jpeg" download> Nice flower</a>

Where:
 -  the virtual host directory is pointing to  ...www\symfony_prj\web
 - "uploads" directory is found under Symfony's  "web" directory

In this approach anyone can access the files. In many situations this is not the desired behavior.

Serving protected files with Symfony2

First we should let Apache know that access to the files should be blocked, so in the "uploads" file I will add a .htaccess file with just one row:

deny from all

 If you try again to access the link you get an error that your browser cannot find it.

If I would use plain PHP the solution for serving the files could be the one described here http://php.net/manual/en/function.readfile.php

<?php
$file = 'monkey.gif';

if (file_exists($file)) {
    header('Content-Description: File Transfer');
    header('Content-Type: application/octet-stream');
    header('Content-Disposition: attachment; filename="'.basename($file).'"');
    header('Expires: 0');
    header('Cache-Control: must-revalidate');
    header('Pragma: public');
    header('Content-Length: ' . filesize($file));
    readfile($file);
    exit;
}

But I am using Symfony, and I have access to the HttpFoundation component:

The HttpFoundation component defines an object-oriented layer for the HTTP specification.

The official documentation  about serving files can be found here:
http://symfony.com/doc/current/components/http_foundation/introduction.html#serving-files

One of the options described is using a BinaryFileResponse.

The link from our page will not point direct to the file, instead will be a regular Symfony route to a controller.

The controller looks like this:

 
    /**
     * Serve a file
     *
     * @Route("/download/{id}", name="file_download", requirements={"id": "\d+"})
     * @Method("GET")
     */
    public function downloadFileAction(Request $request, File $file)
    {

        /*
         * $basePath can be either exposed (typically inside web/)
         * or "internal"
         */
        $filename= $file->getName();
        $basePath = $this->container->getParameter('my_upload_dir');
        $filePath = $basePath.'/'.$filename;
        // check if file exists
        $fs = new FileSystem();

        if (!$fs->exists($filePath)) {
            throw $this->createNotFoundException();
        }

        // prepare BinaryFileResponse
        $response = new BinaryFileResponse($filePath);
        $response->trustXSendfileTypeHeader();
        $response->setContentDisposition(
            ResponseHeaderBag::DISPOSITION_ATTACHMENT,
            $filename,
            iconv('UTF-8', 'ASCII//TRANSLIT', $filename)
        );
        return $response;
    }

I have a File class mapped with Doctrine, Symfony is smart enough to transform the route "id" parameter to the actual File object instance. In the File object only the name of the file is saved, and it can be retrieved with the method getName().

The function "setContentDisposition"  can receive as parameter the constant:

ResponseHeaderBag::DISPOSITION_ATTACHMENT and is asking for download, or you can force download with ResponseHeaderBag::DISPOSITION_INLINE

This article is inspired from the above mentioned documentation links and from this blog post:  http://symfonybricks.com/en/brick/how-to-force-file-download-from-controller-using-binaryfileresponse

PHP Errors and Exception handling

When learning  PHP is very important to find the correct source of information.

I found a very good article on the topic "PHP Error handling" that  worth being shared:

http://alanstorm.com/php_error_reporting

So in PHP there are 2 systems, one for Error handling, and one for Exceptions.

Errors are real errors from PHP code, like missing  ";" at the end of  statement or trying to access an undefined variable.

Exceptions are some objects crafted by programmers. You could check if a variable has  a value bigger than 10, and if not you can throw an Exception with a message describing why  this makes no sense in your application logic.

There is also a difference in that Error handling is old school and Exceptions are OOP. You can create your own Exception class to match your needs: http://php.net/manual/en/language.exceptions.extending.php

Roughly speaking, errors are a legacy in PHP, while exceptions are the modern way to treat errors. The simplest thing then, is to set up an error-handler, that throws an exception. That way all errors are converted to exceptions, and then you can simply deal with one error-handling scheme. The following code will convert errors to exceptions for you:

function exceptions_error_handler($severity, $message, $filename, $lineno) {
  if (error_reporting() == 0) {
    return;
  }
  if (error_reporting() & $severity) {
    throw new ErrorException($message, 0, $severity, $filename, $lineno);
  }
}
set_error_handler('exceptions_error_handler');
error_reporting(E_ALL ^ E_STRICT);

Text from http://stackoverflow.com/questions/134331/error-handling-in-php

This answer from Stackoverflow pretty much clarifies how things work now, you set an error handler function and in that function you throw an Exception. You make sure to catch that Exception in your code because otherwise you will get a Uncaught Fatal Error.

Errors and exceptions in Symfony

The Debug componenet from Symfony works on the same principle:

The ErrorHandler class catches PHP errors and converts them to exceptions (of class ErrorException or FatalErrorException for PHP fatal errors)

 From Symfony doc: http://symfony.com/doc/current/components/debug/introduction.html

In the development environment, Symfony catches all the exceptions and displays a special exception page with lots of debug information to help you quickly discover the root problem.

Since these pages contain a lot of sensitive internal information, Symfony won't display them in the production environment. Instead, it'll show a simple and generic error page.

Information on how to override these error pages can be found in the Symfony documentation here: http://symfony.com/doc/2.8/cookbook/controller/error_pages.html

CPANAHierarchyBundle - Symfony and Neo4j

I've just uploaded to Github my last project: HierarchyBundle


This Symfony bundle manages a company hierarchy retrieving and stroring information to a Neo4j database. The hierarchy is based on groups: PHP group is part of Software Developement group, which is part of IT, etc. The users are members of these groups having different kind of roles (manager, employee etc).

I've got the idea for this bundle from the Neo4j documentation: http://neo4j.com/docs/stable/examples-user-roles-in-graphs.html . Basically it says that relational database are not really suited for representing hierarchical data :)

HierarchyBundle is using the Neo4jPHP library https://github.com/jadell/neo4jphp . On top of this I've created a class called GroupHierarchyManagement which contains specific functions for this scope, like retrieving the complete hierarchy above a user:

    /**
  * Return the hierarchy of of groups above one user node
  *
  * @param  Everyman\Neo4j\Node   node
  * @return row| null
  */
    public function getGroupHierarchyOfUser($node)
    {
        $id = $node->getId();

        $queryString =  ' MATCH n-[rel:'. $this->defRelTypeUserToGroup .']->group '.
                        ' WHERE id(n)='.$id.' '.
                        ' WITH group '.
                        ' MATCH p=group-[r:PART_OF*..]->d '.
                        ' WHERE ID(d)='.$this->rootGroupId.' '.
                        ' RETURN nodes(p) as nodes';

        $query = new Query($this->client, $queryString);
        $result = $query->getResultSet();

        if ($result->count() != 0) {
            return $result->current()->current();
        } else {
            return;
        }
    }

This class is configured as a service with the Symfony Dependency Injection Container. There are several parameters to be configured in order to work correctly:

#app/config/config.yml

cpana_hierarchy:
    group_hierarchy_manager_neo4j:
        neo4j_user:  'neo4j'
        neo4j_password:  'parola'
        def_rel_type_group_to_group: 'PART_OF'
        def_rel_type_user_to_group: 'MEMBER_OF'
        root_group_id: '69374'
        manager_role_property: 'manager'
        default_property_group:  'name'
        default_property_user: 'name'

Th first two are the user and password in order to login to Neo4j database. Next are the types (names) of the relations between Groups and Groups or between Users and Groups.
We can say that the team "PHP " is "PART_OF" departmenet "Software developement".
Also we can say employee Kenny is "MEMBER_OF" team "PHP".

You need to specify which is the root group node of your hierarchy by prodiving the Neo4j Id of that node in parameter "root_group_id".

The relation "MEMBER_OF" between Users and Groups has a property called "role", you can define which is the value of this property for users which are managers of a certain group: it can be a number, or explicitly  "manager", or "master" :) etc.

A Group or a User node can have many properties, like name, description for Groups, or age, salary for Users, from these we need to configure which to be displayed by default when we talk about that node, the name should be the most obvious value.

With this service called " group_hierarchy_manager_neo4j" I've built an application with a front side where regular users can browse the data and an Admin area to edit  the data.
Below some photos to get the idea:
Front side:

  

Admin side :

Symfony: How to embed a Collection of Forms and customize the form field Prototype

This article is based on the following doc articles:

http://symfony.com/doc/current/cookbook/form/form_collections.html
http://symfony.com/doc/2.7/reference/forms/types/collection.html
http://toni.uebernickel.info/2012/03/15/an-example-of-symfony2-collectiontype-form-field-prototype.html

In my example below I will be using Doctrine's  One-To-Many Bidirectional relation. Let's say we have a Feedback form to which you can add none or many Actions to be taken. Feedback form contains information like name and email of the person and Action form contains information like action description, deadline date, importance level. I will eliminate any details which are not relevant to the subject (like all the mappings, validations etc)/

Entities:

Feedback

<?php

class Feedback
{
    
    /**
     * @ORM\OneToMany(targetEntity="\MyBundle\Entity\Action", mappedBy="fkFeedback", cascade={"persist","remove"})
     */
    private $actions;
    
    private $name;
    
    private $email;


    public function __construct() {
        $this->actions = new ArrayCollection();
    }

    public function getActions()
    {
        return $this->actions;
    }

    public function addAction(\MyBundle\Entity\Action $action) 
    {
        $action->setFkIdFeedback($this);
        $this->actions->add($action);
        return $this;
    }
 
    public function removeAction($action) 
    {
        if ($this->actions>contains($action)) {
            $this->actions->removeElement($action);
        }
        return $this;
    }

}

Action

<?php

class Action

{
    private $actionDesc;

    private $importance;

    private $deadlineDate;

    /**
     * @ORM\ManyToOne(targetEntity="Feedback", inversedBy="actions")
     * @ORM\JoinColumn(name="fk_feedback", referencedColumnName="id", nullable=FALSE )
     */
    private $fkFeedback;
}

Note mappedBy and inversedBy properties. Also on the One side I have added information regarding how Doctrine should persist the associated entities: cascade={'persist'}.
Also 'addAction()' function makes sure the changes are reflected in both sides of the relation.

 Forms

<?php

class FeedbackType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            ->add('name','text',array('label' => false, 'required' => false, 'attr'=> array('placeholder'=> 'Last name')))
            ->add('email','email',array('label' => false, 'required' => false, 'attr'=> array('placeholder'=> 'Email')))
            ->add('actions', 'collection', array(
                'type' => new ActionType(),
                'allow_add'    => true,
                'allow_delete' => true,
                'by_reference' => false,
                'cascade_validation' => true,
                'error_bubbling' => false,
            ));
    }
    
    /**
     * @param OptionsResolverInterface $resolver
     */
    public function setDefaultOptions(OptionsResolverInterface $resolver)
    {
        $resolver->setDefaults(array(
            'data_class' => 'MyBundle\Entity\Feedback',
            'cascade_validation' => true,   
        ));
    }

    /**
     * @return string
     */
    public function getName()
    {
        return 'mybundle_feedback';
    }
}

On the FeedbackType class (One side from the One-To-Many relation) I set a collection of ActionType forms. Also 'error_bubbling' to false because I want to get errors from Action form under each field and not only at the form level: form_error(form) .

The ActionType class has nothig special in it, I will not listed here.

Allowing the user to dynamically add new Actions means that you'll need to use some JavaScript.
The option 'allow_add' is needed to this flexibility to be possible.

In addition to telling the field to accept any number of submitted objects, the allow_add also makes a "prototype" variable available to you. This "prototype" is a little "template" that contains all the HTML to be able to render any new "tag" forms.

If you render the entire form you will see in a HTML  data-*  attribute the prototype for creating new Actions. From here on I will not follow the example found in official documentation, instead I will render the form and use jQuery code like in this blog article.

Rendering the form

The prototype macro

In the situation when by default I want to render 2 actions to be filled and allow dynamically to be added other there may be differences between how the widget is rendered and how the prototype is rendered. The below Twig macro handles this issue.
This macro renders the prototype and the actual widget the same way. Therefore the resulting usage code is very little and required javascript code just works for all collections.

{% macro widget_prototype(widget, remove_text) %}
        {% if widget.vars.prototype is defined %}
            {% set form = widget.vars.prototype %}
            {% set name = widget.vars.prototype.vars.name %}
        {% else %}
            {% set form = widget %}
            {% set name = widget.vars.full_name %}
    {% endif %}

        <div data-content="{{ name }}"  class="panel panel-default">
            <div class="panel-body">
                {{ form_row(form.actionDesc) }}
                {{ form_row(form.importance) }}
                {{ form_row(form.deadlineDate, {'attr':{'class':'actions-deadline'}}) }}
                <a class="btn btn-danger" data-related="{{ name }}">{{ remove_text }}</a>
            </div>
            
            
        </div>
    {% endmacro %}

As an example I customized the class attribute for deadlineData to be able to identify them for adding  Datepicker jQuery UI plugin.  The values  widget.vars.full_name and widget.vars.name are related to form object not to my actual Entity fields (name, email).

The actual form rendering:

I am not rendering the form using form_start, form_end in order to get more control on what is displayed. If you are using Bootstrap you can use one of the Twig form layouts offered by Symfony team:

        {% form_theme form 'MyBundle:Form:bootstrap_3_layout.html.twig' %}
        <div class="row">
            <div class="col-sm-3"></div>
            <div class="col-sm-6">
                <h3 style="color:white;">Feedback</h3>
                <form name="docsbundle_feedback" method="POST">
                        {{ form_errors(form) }}
                    <fieldset>           
                        {{ form_row(form.name) }}
                        {{ form_row(form.email) }}
                        <hr>
                         <h3 style="color:white">Actions</h3>
                            <div>
                                <div id="actions" data-prototype="{{ _self.widget_prototype(form.actions, 'Remove action')|escape }}">
                                    {% for widget in form.actions.children %}
                                        {{ _self.widget_prototype(widget, 'Remove action') }}
                                    {% endfor %}
                                </div>

                                <a id="add-action" class="btn btn-info" data-target="actions">Add action</a>
                            </div>

                        <hr>               

                        {{ form_widget(form._token) }}

                        {{ form_widget(form.submit, {'attr':{'class':'btn-info'}}) }}

                    </fieldset>
                </form>
            </div>
        <div class="col-sm-3"></div>
        </div>

jQuery snippet:

 The below jQuery take care to allow adding and removing Actions.

{% block javascripts %}
    <script>
        $('#add-action').click(function(event) {
            var collectionHolder = $('#' + $(this).attr('data-target'));
            var prototype = collectionHolder.attr('data-prototype');
            var form = prototype.replace(/__name__/g, collectionHolder.children().length);

            collectionHolder.append(form);

            return false;
        });
        $('#actions').on('click','.btn-danger', function(event) {
            var name = $(this).attr('data-related');
            $('*[data-content="'+name+'"]').remove();

            return false;
        });

    </script>
{% endblock %}

Controller

In controller I am persisting only the Feedback entity, and with the option cascade={'persis'}, Doctrine knows to take care of persisting the associated Action objects

<?php

public function feedbackAction(Request $request)
    {
        
        $entity = new Feedback();
        
        $empty_action = new Action();
        $entity->addAction($empty_action);
        
        $form = $this->feedbackCreateForm($entity);
        
        $form->handleRequest($request);           
        
        if ($form->isValid()) {  
            .....
            // persist the $entity
            $em = $this->getDoctrine()->getManager();
            $em->persist($entity); 
            $em->flush();
            ...

Neo4j intro

Neo4j is a graph database management system developed by Neo Technology, Inc. Described by its developers as an ACID-compliant transactional database with native graph storage and processing,Neo4j is the most popular graph database according to db-engines.com ^[wikipedia]

I am interested in Neo4j for modelling hierarchies as the RDBMS are not really suited for this ( http://www.codeproject.com/Articles/22824/A-Model-to-Represent-Directed-Acyclic-Graphs-DAG-o)

 Installing Neo4j Community Edition on Windows is a piece of cake: http://neo4j.com/download/
Also there is plenty of documentation and many answered questions on StackOverflow :)

A good place to start: https://www.airpair.com/neo4j/posts/getting-started-with-neo4j-and-cypher

Some useful queries:
--------------------------------
Match Node by id:

MATCH (n)
WHERE id(n) = 14
RETURN n;
---------------------------------
Match relationship by Id

MATCH ()-[r]-() 
WHERE ID(r)=1 
RETURN r 

------------------------------
Find path between 2 nodes, replace "REPORTS_TO" with your relation

START a=node(11), d=node(12)
MATCH p=a-[r:REPORTS_TO*..]-d
RETURN p;
----------------------------
Delete node and all its relationships

MATCH (n { name:'Andres' })
DETACH DELETE n
-----------------------------------------
DELETE all relationships

MATCH  ()-[r:REPORTS_TO]->() DELETE  r;

Importing data from CSV

Official docs: http://neo4j.com/docs/stable/query-load-csv.html

I copied the files to be imported next to the database file because I had issues with other paths. My path looks like: C:\Users\my_user\Documents\Neo4j\file_to_import.csv

Loading nodes:

LOAD CSV FROM 'file:///C:/Users/my_user/Documents/Neo4j/nodes.csv' AS line
CREATE (:User { some_id: line[0], user_name: line[1], link: line[2] })

Add an index on some_id property:

CREATE CONSTRAINT ON (n:User) ASSERT n.some_id IS UNIQUE

Load relations from CSV file, replace PART_OF with your relation:

LOAD CSV  FROM "file:///C:/Users/my_user/Documents/Neo4j/relations.csv" AS line
MATCH (p:User { some_id:line[0] })
MATCH (f:Group { group_id:line[1] })
CREATE (f)-[:PART_OF { some_property: line[2] } ]->(p);

Save logs to Database and access them from web UI with LexikMonologBrowserBundle

Monolog: Monolog is a logging library for PHP used by Symfony.

Cookbook:  http://symfony.com/doc/current/cookbook/logging/monolog.html

LexikMonologBrowserBundle:  https://github.com/lexik/LexikMonologBrowserBundle

This Symfony2 bundle provides a Doctrine DBAL handler for Monolog and a web UI to display log entries.

I am using an existing Doctrine configuration in config.yml:

#config.yml

    default:
                    driver:   "%database_driver%"
                    host:     "%database_host%"
                    port:     "%database_port%"
                    dbname:   "%database_name%"
                    user:     "%database_user_prod%"
                    password: "%database_password_prod%"
                    charset:  UTF8
                    logging:   false
                    profiling: false

If you do not have the last two lines, you should add them to avoid circular reference error.
And also  add lexik_monolog_browser config to use the existing Doctrine connection:

    lexik_monolog_browser:
        doctrine:
            connection_name:  default
            table_name: monolog_entries

Generatate the table in the database:

    >php app/console lexik:monolog-browser:schema-create
   

Add the below to config_dev.yml and/or config_prod.yml in order for Monolog to use our handler to save into database.

    # app/config/config_prod.yml # or any env
    monolog:
        handlers:
            main:
                type:         fingers_crossed # or buffer
                level:        error
                handler:      lexik_monolog_browser
            app:
                type:         buffer
                action_level: info
                channels:     app
                handler:      lexik_monolog_browser
            deprecation:
                type:         buffer
                action_level: warning
                channels:     deprecation
                handler:      lexik_monolog_browser
            lexik_monolog_browser:
                type:         service
                id:           lexik_monolog_browser.handler.doctrine_dbal

Import routes to app/config/routing.yml:

    # app/config/routing.yml
    lexik_monolog_browser:
        resource: "@LexikMonologBrowserBundle/Resources/config/routing.xml"
        prefix:   /admin/monolog



More:

It would be good to save in logs also the current user. For this you can add in your controller:

       //log current user in Monolog
        $logger = $this->get('logger');
        $logger->info('Current user is : ' . $this->getUser()->getUsername());