Sunday, March 20, 2016

CPANAHierarchyBundle - Symfony and Neo4j

I've just uploaded to Github my last project: HierarchyBundle


This Symfony bundle manages a company hierarchy retrieving and stroring information to a Neo4j database. The hierarchy is based on groups: PHP group is part of Software Developement group, which is part of IT, etc. The users are members of these groups having different kind of roles (manager, employee etc).

I've got the idea for this bundle from the Neo4j documentation: http://neo4j.com/docs/stable/examples-user-roles-in-graphs.html . Basically it says that relational database are not really suited for representing hierarchical data :)

HierarchyBundle is using the Neo4jPHP library https://github.com/jadell/neo4jphp . On top of this I've created a class called GroupHierarchyManagement which contains specific functions for this scope, like retrieving the complete hierarchy above a user:



    /**
  * Return the hierarchy of of groups above one user node
  *
  * @param  Everyman\Neo4j\Node   node
  * @return row| null
  */
    public function getGroupHierarchyOfUser($node)
    {
        $id = $node->getId();

        $queryString =  ' MATCH n-[rel:'. $this->defRelTypeUserToGroup .']->group '.
                        ' WHERE id(n)='.$id.' '.
                        ' WITH group '.
                        ' MATCH p=group-[r:PART_OF*..]->d '.
                        ' WHERE ID(d)='.$this->rootGroupId.' '.
                        ' RETURN nodes(p) as nodes';

        $query = new Query($this->client, $queryString);
        $result = $query->getResultSet();

        if ($result->count() != 0) {
            return $result->current()->current();
        } else {
            return;
        }
    }


This class is configured as a service with the Symfony Dependency Injection Container. There are several parameters to be configured in order to work correctly:


#app/config/config.yml

cpana_hierarchy:
    group_hierarchy_manager_neo4j:
        neo4j_user:  'neo4j'
        neo4j_password:  'parola'
        def_rel_type_group_to_group: 'PART_OF'
        def_rel_type_user_to_group: 'MEMBER_OF'
        root_group_id: '69374'
        manager_role_property: 'manager'
        default_property_group:  'name'
        default_property_user: 'name'


Th first two are the user and password in order to login to Neo4j database. Next are the types (names) of the relations between Groups and Groups or between Users and Groups.
We can say that the team "PHP " is "PART_OF" departmenet "Software developement".
Also we can say employee Kenny is "MEMBER_OF" team "PHP".

You need to specify which is the root group node of your hierarchy by prodiving the Neo4j Id of that node in parameter "root_group_id".

The relation "MEMBER_OF" between Users and Groups has a property called "role", you can define which is the value of this property for users which are managers of a certain group: it can be a number, or explicitly  "manager", or "master" :) etc.

A Group or a User node can have many properties, like name, description for Groups, or age, salary for Users, from these we need to configure which to be displayed by default when we talk about that node, the name should be the most obvious value.

With this service called " group_hierarchy_manager_neo4j" I've built an application with a front side where regular users can browse the data and an Admin area to edit  the data.
Below some photos to get the idea:
Front side:

  

Admin side :


Tuesday, March 15, 2016

Symfony: How to embed a Collection of Forms and customize the form field Prototype

This article is based on the following doc articles:

http://symfony.com/doc/current/cookbook/form/form_collections.html
http://symfony.com/doc/2.7/reference/forms/types/collection.html
http://toni.uebernickel.info/2012/03/15/an-example-of-symfony2-collectiontype-form-field-prototype.html

In my example below I will be using Doctrine's  One-To-Many Bidirectional relation. Let's say we have a Feedback form to which you can add none or many Actions to be taken. Feedback form contains information like name and email of the person and Action form contains information like action description, deadline date, importance level. I will eliminate any details which are not relevant to the subject (like all the mappings, validations etc)/

Entities:

Feedback
<?php

class Feedback
{
    
    /**
     * @ORM\OneToMany(targetEntity="\MyBundle\Entity\Action", mappedBy="fkFeedback", cascade={"persist","remove"})
     */
    private $actions;
    
    private $name;
    
    private $email;


    public function __construct() {
        $this->actions = new ArrayCollection();
    }

    public function getActions()
    {
        return $this->actions;
    }

    public function addAction(\MyBundle\Entity\Action $action) 
    {
        $action->setFkIdFeedback($this);
        $this->actions->add($action);
        return $this;
    }
 
    public function removeAction($action) 
    {
        if ($this->actions>contains($action)) {
            $this->actions->removeElement($action);
        }
        return $this;
    }

}

Action

<?php

class Action

{
    private $actionDesc;

    private $importance;

    private $deadlineDate;

    /**
     * @ORM\ManyToOne(targetEntity="Feedback", inversedBy="actions")
     * @ORM\JoinColumn(name="fk_feedback", referencedColumnName="id", nullable=FALSE )
     */
    private $fkFeedback;
}


Note mappedBy and inversedBy properties. Also on the One side I have added information regarding how Doctrine should persist the associated entities: cascade={'persist'}.
Also 'addAction()' function makes sure the changes are reflected in both sides of the relation.

 Forms


<?php

class FeedbackType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            ->add('name','text',array('label' => false, 'required' => false, 'attr'=> array('placeholder'=> 'Last name')))
            ->add('email','email',array('label' => false, 'required' => false, 'attr'=> array('placeholder'=> 'Email')))
            ->add('actions', 'collection', array(
                'type' => new ActionType(),
                'allow_add'    => true,
                'allow_delete' => true,
                'by_reference' => false,
                'cascade_validation' => true,
                'error_bubbling' => false,
            ));
    }
    
    /**
     * @param OptionsResolverInterface $resolver
     */
    public function setDefaultOptions(OptionsResolverInterface $resolver)
    {
        $resolver->setDefaults(array(
            'data_class' => 'MyBundle\Entity\Feedback',
            'cascade_validation' => true,   
        ));
    }

    /**
     * @return string
     */
    public function getName()
    {
        return 'mybundle_feedback';
    }
}

On the FeedbackType class (One side from the One-To-Many relation) I set a collection of ActionType forms. Also 'error_bubbling' to false because I want to get errors from Action form under each field and not only at the form level: form_error(form) .

The ActionType class has nothig special in it, I will not listed here.

Allowing the user to dynamically add new Actions means that you'll need to use some JavaScript.
The option 'allow_add' is needed to this flexibility to be possible.

In addition to telling the field to accept any number of submitted objects, the allow_add also makes a "prototype" variable available to you. This "prototype" is a little "template" that contains all the HTML to be able to render any new "tag" forms.

If you render the entire form you will see in a HTML  data-*  attribute the prototype for creating new Actions. From here on I will not follow the example found in official documentation, instead I will render the form and use jQuery code like in this blog article.

Rendering the form

The prototype macro

In the situation when by default I want to render 2 actions to be filled and allow dynamically to be added other there may be differences between how the widget is rendered and how the prototype is rendered. The below Twig macro handles this issue.
This macro renders the prototype and the actual widget the same way. Therefore the resulting usage code is very little and required javascript code just works for all collections.


{% macro widget_prototype(widget, remove_text) %}
        {% if widget.vars.prototype is defined %}
            {% set form = widget.vars.prototype %}
            {% set name = widget.vars.prototype.vars.name %}
        {% else %}
            {% set form = widget %}
            {% set name = widget.vars.full_name %}
    {% endif %}

        <div data-content="{{ name }}"  class="panel panel-default">
            <div class="panel-body">
                {{ form_row(form.actionDesc) }}
                {{ form_row(form.importance) }}
                {{ form_row(form.deadlineDate, {'attr':{'class':'actions-deadline'}}) }}
                <a class="btn btn-danger" data-related="{{ name }}">{{ remove_text }}</a>
            </div>
            
            
        </div>
    {% endmacro %}

As an example I customized the class attribute for deadlineData to be able to identify them for adding  Datepicker jQuery UI plugin.  The values  widget.vars.full_name and widget.vars.name are related to form object not to my actual Entity fields (name, email).

The actual form rendering:

I am not rendering the form using form_start, form_end in order to get more control on what is displayed. If you are using Bootstrap you can use one of the Twig form layouts offered by Symfony team:


        {% form_theme form 'MyBundle:Form:bootstrap_3_layout.html.twig' %}
        <div class="row">
            <div class="col-sm-3"></div>
            <div class="col-sm-6">
                <h3 style="color:white;">Feedback</h3>
                <form name="docsbundle_feedback" method="POST">
                        {{ form_errors(form) }}
                    <fieldset>           
                        {{ form_row(form.name) }}
                        {{ form_row(form.email) }}
                        <hr>
                         <h3 style="color:white">Actions</h3>
                            <div>
                                <div id="actions" data-prototype="{{ _self.widget_prototype(form.actions, 'Remove action')|escape }}">
                                    {% for widget in form.actions.children %}
                                        {{ _self.widget_prototype(widget, 'Remove action') }}
                                    {% endfor %}
                                </div>

                                <a id="add-action" class="btn btn-info" data-target="actions">Add action</a>
                            </div>

                        <hr>               

                        {{ form_widget(form._token) }}

                        {{ form_widget(form.submit, {'attr':{'class':'btn-info'}}) }}

                    </fieldset>
                </form>
            </div>
        <div class="col-sm-3"></div>
        </div>

jQuery snippet:

 The below jQuery take care to allow adding and removing Actions.


{% block javascripts %}
    <script>
        $('#add-action').click(function(event) {
            var collectionHolder = $('#' + $(this).attr('data-target'));
            var prototype = collectionHolder.attr('data-prototype');
            var form = prototype.replace(/__name__/g, collectionHolder.children().length);

            collectionHolder.append(form);

            return false;
        });
        $('#actions').on('click','.btn-danger', function(event) {
            var name = $(this).attr('data-related');
            $('*[data-content="'+name+'"]').remove();

            return false;
        });

    </script>
{% endblock %}

Controller

In controller I am persisting only the Feedback entity, and with the option cascade={'persis'}, Doctrine knows to take care of persisting the associated Action objects


<?php

public function feedbackAction(Request $request)
    {
        
        $entity = new Feedback();
        
        $empty_action = new Action();
        $entity->addAction($empty_action);
        
        $form = $this->feedbackCreateForm($entity);
        
        $form->handleRequest($request);           
        
        if ($form->isValid()) {  
            .....
            // persist the $entity
            $em = $this->getDoctrine()->getManager();
            $em->persist($entity); 
            $em->flush();
            ...

Sunday, March 6, 2016

Neo4j intro

Neo4j is a graph database management system developed by Neo Technology, Inc. Described by its developers as an ACID-compliant transactional database with native graph storage and processing,Neo4j is the most popular graph database according to db-engines.com [wikipedia]

I am interested in Neo4j for modelling hierarchies as the RDBMS are not really suited for this ( http://www.codeproject.com/Articles/22824/A-Model-to-Represent-Directed-Acyclic-Graphs-DAG-o)

 Installing Neo4j Community Edition on Windows is a piece of cake: http://neo4j.com/download/
Also there is plenty of documentation and many answered questions on StackOverflow :)

A good place to start: https://www.airpair.com/neo4j/posts/getting-started-with-neo4j-and-cypher

Some useful queries:
--------------------------------
Match Node by id:

MATCH (n)
WHERE id(n) = 14
RETURN n;

---------------------------------
Match relationship by Id

MATCH ()-[r]-() 
WHERE ID(r)=1 
RETURN r 

------------------------------
Find path between 2 nodes, replace "REPORTS_TO" with your relation

START a=node(11), d=node(12)
MATCH p=a-[r:REPORTS_TO*..]-d
RETURN p;

----------------------------
Delete node and all its relationships

MATCH (n { name:'Andres' })
DETACH DELETE n

-----------------------------------------
DELETE all relationships

MATCH  ()-[r:REPORTS_TO]->() DELETE  r;

Importing data from CSV

Official docs: http://neo4j.com/docs/stable/query-load-csv.html

I copied the files to be imported next to the database file because I had issues with other paths. My path looks like: C:\Users\my_user\Documents\Neo4j\file_to_import.csv

Loading nodes:

LOAD CSV FROM 'file:///C:/Users/my_user/Documents/Neo4j/nodes.csv' AS line
CREATE (:User { some_id: line[0], user_name: line[1], link: line[2] })


Add an index on some_id property:

CREATE CONSTRAINT ON (n:User) ASSERT n.some_id IS UNIQUE

Load relations from CSV file, replace PART_OF with your relation:

LOAD CSV  FROM "file:///C:/Users/my_user/Documents/Neo4j/relations.csv" AS line
MATCH (p:User { some_id:line[0] })
MATCH (f:Group { group_id:line[1] })
CREATE (f)-[:PART_OF { some_property: line[2] } ]->(p);