Showing posts with label pagination. Show all posts
Showing posts with label pagination. Show all posts

Tuesday, November 24, 2015

Symfony and Doctrine - One-to-Many Bidirectional associations

As you could saw in the two previous posts I am using 2 entities, Ads and SpecificItems, found in a One-to-Many relationship unidirectional. One Ad can have none or several SpecificItems.
What makes it unidirectional is the fact that the Many side(specific items) contain "ads" objects, but the "ads" do not know which are their specific items.

Example of specific items :  2012 (year of manufacturing  of a car etc. SpecificItems holds only the value, the type name (Year of manufacturing, Color etc) is hold in a different class (entity).

Because of the unidirectional relation at this moment I am retrieving the data with 2 different queries, first query get the ads, the second one retrieve the specific items which have ads in the first query result. Bot quires return data as arrays and not as object (as I selected only some fields).

$ads = $em->getRepository('CPANAClassifiedsBundle:Ads')
->findAllAdsMyPagination($currentPage, $limit);
        
 $allSpecificItems = $em->getRepository('CPANAClassifiedsBundle:SpecificItems')

->findItemsForAdsList($ads);

And in the view I am combining the two arrays.

I feel that I am not using really the Doctrine's capabilities by retrieving arrays  instead of actual objects. It is also more natural for an Ad to contain the list of specific items.



Official documentation Doctrine associations:
http://doctrine-orm.readthedocs.org/projects/doctrine-orm/en/latest/reference/association-mapping.html
http://doctrine-orm.readthedocs.org/projects/doctrine-orm/en/latest/reference/unitofwork-associations.html

1. First step - modify the entities

 In the One side of the One to Many, in the Ads entity, I've added $items, which is a collection object, it will hold all the associated SpecificItems objects.


use Doctrine\Common\Collections\ArrayCollection;
........

/**
     * @var integer
     *
     * @ORM\OneToMany(targetEntity="CPANA\ClassifiedsBundle\Entity\SpecificItems", mappedBy="idAds")
     * @ORM\JoinColumn(name="items_id", referencedColumnName="id")
     */

    private $items;

    public function __construct() {
        $this->items = new ArrayCollection();
    }

 
    public function getItems()
    {
        return $this->items;
    }
  
    public function addItem(\CPANA\ClassifiedsBundle\Entity\SpecificItems $item) {
        $this->items[] = $item;
        $item->setIdAds($this);
        return $this;
    }
  
    public function removeItems($item) {
        $this->items->removeElement($item);
    }


In the Many side (SpecificItems entity) I make sure to specific the inversedBy clause:

 /**
     * @var \CPANA\ClassifiedsBundle\Entity\Ads
     *
     * @ORM\ManyToOne(targetEntity="CPANA\ClassifiedsBundle\Entity\Ads", inversedBy="items")
     * @ORM\JoinColumns({
     *   @ORM\JoinColumn(name="id_ads", referencedColumnName="id_ads")
     * })
     */

    private $idAds;


  2. Modify the Repository

Make sure to add all the classes in the SELECT statement otherwise they will be loaded lazy resulting in a large number of  SELECTS into the database (N+1 queries issue in ORMs). I am using the pagination capabilities from Doctrine.
 
 public function findAllAdsCustom($currentPage = 1, $limit)
    {
        $query= $this->getEntityManager()
            ->createQuery(
                'SELECT  a,p,u,cat,c,ca,city,type
                FROM CPANAClassifiedsBundle:Ads a
                LEFT JOIN a.items p
                LEFT JOIN a.idUser u
                LEFT JOIN a.idCategory cat
                LEFT JOIN a.idCurrency c
                LEFT JOIN a.idCityArea ca
                LEFT JOIN ca.idCity city
                LEFT JOIN p.idSpecificItemType type
                ORDER BY a.idAds'
               
            );
           
        $paginator = $this->paginate($query, $currentPage, $limit);

        return $paginator;
    }
   
    public function paginate($dql, $page, $limit)
    {
        $paginator = new Paginator($dql);
        $paginator->setUseOutputWalkers(false);
          
        $paginator->getQuery()
            ->setFirstResult($limit * ($page - 1)) // Offset
            ->setMaxResults($limit); // Limit

        return $paginator;
    }

3. The controller

No need to call a special query for retrieving the SpecificItems, evrything will be now return in the all_ads array which will contain objects of the Ads class. And these Ads objects contain now the $items collection.
 
 public function showAllAction($currentPage)
    {
        $limit = 1;  // limit variable holds the number of ads per page
        $em = $this->getDoctrine()->getManager();
        $ads = $em->getRepository('CPANAClassifiedsBundle:Ads')->findAllAdsCustom($currentPage, $limit);
      
        $totalAds = $ads->count();
        $maxPages = ceil($totalAds/$limit);
        $thisPage = $currentPage;
      
      
        if (!$ads) {
            throw $this->createNotFoundException('Unable to find ads.');
        }
      
        return $this->render('CPANAClassifiedsBundle:Pub\Ads:show_all.html.twig', array(
            'all_ads' => $ads,
            'maxPages'=>$maxPages,
            'thisPage' => $thisPage,
        ) );
    }


4. The view

 {% extends 'CPANAClassifiedsBundle::layout.html.twig' %}

{% block leftcolumn %} {{ render(controller('CPANA\ClassifiedsBundle:Pub/Categories:index')) }} {% endblock %}

{% block content %}

<table style="width:100%">
    {% for ad in all_ads %}  
        <tr><hr></tr>
        <tr><h1> {{ ad.title }}</h1></tr>
        <tr><p><b>Posted by: </b> {{ ad.idUser.name }}</p></tr>
        <tr><p><b>Category: </b> {{ ad.idCategory.categoryName }}</p></tr>
        <tr><p><b>Area: </b> {{ ad.idCityArea.cityAreaName }}</p></tr>
        <tr><p><b>City: </b> {{ ad.idCityArea.idCity.cityName }}</p></tr>
        <tr><h2>Description</h2></tr>
        <tr><p>{{ ad.description }} </p></tr>
      
        {% for item in ad.items %}
            <tr><p><b> {{ item.idSpecificItemType.itemTypeName  }} </b> - {{ item.itemValue }} </p> </tr>
        {% endfor %}
      
        <tr><p><b>Price: </b> {{ ad.price }} -- <b> {{ ad.idCurrency.currencyName }} </b> </p></tr>
      
      
    {% endfor %}

</table>

    {% if maxPages > 1 %}
        <ul>
            {%if thisPage > 1 %}
                <li >
                    <a href="{{ path('cpana_classifieds_public_ads_show_all', {currentPage: thisPage-1 < 1 ? 1 : thisPage-1}) }}">«</a>
                </li>
            {% endif %}
     
            {# Render each page number #}
            {% for i in 1..maxPages %}
                <li>
                    <a href="{{ path('cpana_classifieds_public_ads_show_all', {currentPage: i}) }}">{{ i }}</a>
                </li>
            {% endfor %}

            {# `»` arrow #}
            {%if thisPage < maxPages %}
                <li>
                    <a href="{{ path('cpana_classifieds_public_ads_show_all', {currentPage: thisPage+1 <= maxPages ? thisPage+1 : thisPage}) }}">»</a>
                </li>
            {% endif %}
        </ul>
    {% endif %}

{% endblock %}


5. See the results

 When loading the "Show all ads" I can see them nicely paginated and also having the associated SpecificItems listed. In total 3 queries were executed to obtain the result, same as before when I had One-To-Many unidirectional relationship between ads and specific items. 




Thursday, November 12, 2015

Symfony and Doctrine: Pagination

In my previous post I've wrote a custom DQL to retrieve all the ads from the database. This is not really the situation in a normal application with thousands of classifieds ads, pagination should be used to limit the number of rows returned from database and keep track of the next sets of rows.

I will first implement pagination in a similar way as I did for CPANABasicBlogBundle in this post.

Documentation:
http://doctrine-orm.readthedocs.org/projects/doctrine-orm/en/latest/tutorials/pagination.html
http://anil.io/post/41/symfony-2-and-doctrine-pagination-with-twig  - tutorial

Starting with version 2.2 Doctrine ships with a Paginator for DQL queries. It has a very simple API
and implements the SPL interfaces Countable and IteratorAggregate.

I. Pagination using Paginator class

1. First modify the path.
The path should have a parameter "currentPage" which can be optional, and the default value is 1:

cpana_classifieds_public_ads_show_all:
    path:     /classifieds/ads/showall/{currentPage}
    defaults: { _controller: CPANAClassifiedsBundle:Pub\Ads:showAll,currentPage:1}


       
2. Modify the repository :

    public function findAllAdsCustom($currentPage = 1, $limit)
    {
        //retrieve the associated Query object with the processed DQL
        $qb= $this->getEntityManager()
            ->createQuery(
                'SELECT  a.idAds, a.title, a.description, a.price
                ,u.name
                ,cat.categoryName
                ,c.currencyName
                ,ca.cityAreaName
                ,city.cityName
                FROM CPANAClassifiedsBundle:Ads a
                JOIN a.idUser u
                JOIN a.idCategory cat
                JOIN a.idCurrency c
                JOIN a.idCityArea ca
                JOIN ca.idCity city'
               
            )
            ->getQuery();
           
        $paginator = $this->paginate($query, $currentPage, $limit);

        return $paginator;
    }
   
    public function paginate($dql, $page = 1, $limit = 3)
    {
        $paginator = new Paginator($dql);
        $paginator->setUseOutputWalkers(false);
       
        $paginator->getQuery()
            ->setFirstResult($limit * ($page - 1)) // Offset
            ->setMaxResults($limit); // Limit

        return $paginator;
    }

-------------------------------------------
If you try to run the code without this line " $paginator->setUseOutputWalkers(false);" you get this  error "Error: Not all identifier properties can be found in the ResultSetMapping:idAds"

http://www.doctrine-project.org/jira/browse/DDC-1927
http://stackoverflow.com/questions/24837913/doctrine-2-pagination
https://groups.google.com/forum/#!topic/doctrine-user/AVbeIyaGvfQ

3. In the controller add:
....
use Doctrine\ORM\Tools\Pagination\Paginator
...

    public function showAllAction($currentPage)
    {
        $limit = 3;
        $em = $this->getDoctrine()->getManager();
        $ads = $em->getRepository('CPANAClassifiedsBundle:Ads')->findAllAdsCustom($currentPage, $limit);
       
        $allSpecificItems = $em->getRepository('CPANAClassifiedsBundle:SpecificItems')->findAllItemsCustom();
       
         $maxPages = ceil($ads->count()/$limit);
        $thisPage = $currentPage;
       
       
        if (!$ads) {
            throw $this->createNotFoundException('Unable to find ads.');
        }
       
        return $this->render('CPANAClassifiedsBundle:Pub\Ads:show_all.html.twig', array(
            'all_ads' => $ads,
            'maxPages'=>$maxPages,
            'thisPage' => $thisPage,
            'all_items' => $allSpecificItems
        ) );
    }
----------------------

4. View
-------
{% block content %}

<table style="width:100%">
    {% for ad in all_ads %}
        <tr><hr></tr>
        <tr><h1> {{ ad['title'] }}</h1></tr>
        <tr><p><b>Posted by: </b> {{ ad['name'] }}</p></tr>

        .....

</table>

    {% if maxPages > 1 %}
        <ul>
            {%if thisPage > 1 %}
                <li >
                    <a href="{{ path('cpana_classifieds_public_ads_show_all', {currentPage: thisPage-1 < 1 ? 1 : thisPage-1}) }}">«</a>
                </li>
            {% endif %}
      
            {# Render each page number #}
            {% for i in 1..maxPages %}
                <li>
                    <a href="{{ path('cpana_classifieds_public_ads_show_all', {currentPage: i}) }}">{{ i }}</a>
                </li>
            {% endfor %}

            {# `»` arrow #}
            {%if thisPage < maxPages %}
                <li>
                    <a href="{{ path('cpana_classifieds_public_ads_show_all', {currentPage: thisPage+1 <= maxPages ? thisPage+1 : thisPage}) }}">»</a>
                </li>
            {% endif %}
        </ul>
    {% endif %}

{% endblock %}


I have only 6 ads so I modified the limit=1 to get the following:

--------------------------------------------------------------------------------------------------

 II. Pagination directly from DQL


http://doctrine-orm.readthedocs.org/projects/doctrine-orm/en/latest/reference/dql-doctrine-query-language.html

14.7.6.5. First and Max Result Items (DQL Query Only)

You can limit the number of results returned from a DQL query as well as specify the starting offset, Doctrine then uses a strategy of manipulating the select query to return only the requested number of results:

    Query::setMaxResults($maxResults)
    Query::setFirstResult($offset)

If your query contains a fetch-joined collection specifying the result limit methods are not working as you would expect. Set Max Results restricts the number of database result rows, however in the case  of fetch-joined collections one root entity might appear in many rows, effectively hydrating less than  the specified number of results.
---------------------------------------------------------------------------------------------------------

1. Modify AdsRepository.php by adding:

    public function findAllAdsMyPagination($currentPage = 1, $limit)
    {
       
        $query= $this->getEntityManager()
            ->createQuery(
                'SELECT  a.idAds, a.title, a.description, a.price
                ,u.name
                ,cat.categoryName
                ,c.currencyName
                ,ca.cityAreaName
                ,city.cityName
                FROM CPANAClassifiedsBundle:Ads a
                JOIN a.idUser u
                JOIN a.idCategory cat
                JOIN a.idCurrency c
                JOIN a.idCityArea ca
                JOIN ca.idCity city'
               
            );
        $query->setFirstResult($limit * ($currentPage - 1)) // Offset
            ->setMaxResults($limit); // Limit
       
        $result = $query->getResult();
        return $result;
    }
   
    public function countAllAds()
    {
        $query= $this->getEntityManager()
            ->createQuery(
                'SELECT  count(a.idAds) FROM CPANAClassifiedsBundle:Ads a');
               
        $count = $query->getSingleScalarResult();;
        return $count;
    }



Note:  As you can see I've added a new method which is counting all the records in ads table.

2. Modify controller to use the new  method findAllAdsMyPagination():

public function showAllAction($currentPage)
    {
        $limit = 3;
        $em = $this->getDoctrine()->getManager();
       
        $ads = $em->getRepository('CPANAClassifiedsBundle:Ads')->findAllAdsMyPagination($currentPage, $limit);
        $totalAds = $em->getRepository('CPANAClassifiedsBundle:Ads')->countAllAds();
        $allSpecificItems = $em->getRepository('CPANAClassifiedsBundle:SpecificItems')->findAllItemsCustom();
       

        $maxPages = ceil($totalAds/$limit);
        $thisPage = $currentPage;

        if (!$ads) {
            throw $this->createNotFoundException('Unable to find ads.');
        }
       
        return $this->render('CPANAClassifiedsBundle:Pub\Ads:show_all.html.twig', array(
            'all_ads' => $ads,
            'maxPages'=>$maxPages,
            'thisPage' => $thisPage,
            'all_items' => $allSpecificItems
        ) );
    }
-----------------------
 Nothing changes in the view
 -------------------------------

III. Loading information from the many side of the Many-To-One relation.

 Now let's improve also the way I retrieve the associated SpecificItems. At this moment I retrieve the entire table,  but I should load only the values for the set of ads listed on one page (after applying pagination).

 1. In "SpecificItemsRepository" I created a new method which receive as parameter a list of ads as an array of arrays.

 

    public function findItemsForAdsList($ads)
    {
        $ads_id_list = array();
        foreach($ads as $ad) {
            $ads_id_list[]=$ad['idAds'];
        }
        $query = $this->getEntityManager()
            ->createQuery(
                'SELECT  si.itemValue
                        ,IDENTITY(si.idAds)
                        ,ty.itemTypeName
                FROM CPANAClassifiedsBundle:SpecificItems si
                JOIN si.idSpecificItemType ty
                WHERE IDENTITY(si.idAds) IN (:list)'
            );
        $query->setParameter('list', $ads_id_list);
      
        return $query->getResult();
    }

 ------
 2. In controller just replace FindAllItemsCustom() with 

 $allSpecificItems = $em->getRepository('CPANAClassifiedsBundle:SpecificItems')->findItemsForAdsList($ads);

3.  Nothing changes in the view.

The SQL queries generated by Doctrine are the following:

Wednesday, July 8, 2015

PHP basic pagination with MySQL

First step: create new branch in GitHub from the GitHub. Code. Download the code, from the GitHub application on local machines. Modify the code, commit changes to this branch and synchronize with GitHub server.  When I finished with the development I make a Pull Request to merge the branch with the repository.

I will display a certain number of posts per page (now is hard coded to 2), and 2 buttons (links) "Older posts" and "Newer Posts".

I am using MySQL "LIMIT" clause to specify the number of records to be returned. From http://www.w3schools.com/php/php_mysql_select_limit.asp
     
     The SQL query below says "return only 10 records, start on record 16 (OFFSET 15)":
        $sql = "SELECT * FROM Orders LIMIT 10 OFFSET 15";

It seems that the default type for bindParam is string so you will get an error if you do not specify to be an integer:   PDO:PARAM_INT:
    http://stackoverflow.com/questions/4544051/sqlstate42000-syntax-error-or-access-violation-1064-you-have-an-error-in-you
       
        $sth=$this->db->prepare("SELECT * FROM blogposts LIMIT :per_page OFFSET :page_number ;");
        $sth->bindParam(':per_page', $per_page, PDO::PARAM_INT);
        $sth->bindParam(':page_number', $page_numb, PDO::PARAM_INT);
        $sth->execute();



-added pagination.xml file in \model to store current page number and number of posts per page (this should be changed in the future as having one current page for all the users accessing the website is not right.)  Later edit: problem solved in this post http://phpbeginnertoadvanced.blogspot.co.uk/2015/07/fix-pagination.html

-added routes for /blog/older and /blog/newer 

You will have access to the code files when releasing the v0.4