Collection Filters - Readme Driven Development

README.md

Filter Language for Collections

Why?

You often need subsets of objects in a collection and want to access them efficiently in your domain model. But you certainly don't want to access the EntityManager or any other object manager here to craft a query. FilterExpressions for collections allow to go back to the database and query for all objects matching the crafted expression. Additionally they also work against in meemory ArrayCollection exactly the same. This way you don't (except for the SQL performance when it haunts you ;)) have to think about the context and can focus on your domain logic.

In Doctrine ORM this will be done by building DQL under the hood, in memory it will be done using Collection#filter(Closure $closure);

Technical Requirements:

1. Should allow filtering depending on the "persistence" backend, i.e. in memory for Arraycollection and using sql for PersistentCollection
2. Should be very simple to be adoptable in many persistence providers
3. Are always either accepting "Expr op Expr" xor "Field op value". A new Expression Language is needed for that, cannot reuse the ORM one.
4. Assumes that for "Field" a getter "getField" exists on target object and that the field is mapped in any corresponding persistence provider.

example.php

<?php

class Post
{
    /**
     * @OneToMany(targetEntity="Comment", mappedBy="post", fetch="EXTRA_LAZY")
     * @var ArrayCollection
     */
    private $comments;
    
    public function __construct()
    {
        $this->comments = new ArrayCollection();
    }
    
    public function getRecentComments()
    {
        $expr = new ExpressionBuilder();
        return $this->comments->select(
            $expr->gt("created", new \DateTime("-7 days"))
        );
    }
    
    public function getCommentsByAuthor($author)
    {
        $expr = new ExpressionBuilder();
        return $this->comments->select(
            $expr->equals("author", $author)
        );
    }
    
    public function getAllRecentSpamComments()
    {
        $expr = new ExpressionBuilder();
        return $this->comments->select(
            $expr->and(
                $expr->equals("status", Comment::SPAM),
                $expr->gt("created", new \DateTime("-7 days"))
            )
        );
    }
}

class Comment
{
    /**
     * @ManyToOne(targetEntity="Post", inversedBy="comments")
     * @var Post
     */
     private $post;
     
     /**
      * @ManyToOne(targetEntity="User")
      */
     private $author;
     
     /**
      * @Column(type="datetime")
      * @var DateTime
      */
     private $created;
     
     /**
      * @Column(type="integer")
      * @var integer
      */
     private $status = self::PUBLISHED;
}

// Both ArrayCollection and PersistentCollection will implement this.
interface FilteredCollection extends Collection
{
    /**
     * Match all objects against the given expression return a NEW collection.
     *
     * @return Collection
     */
    public function select(Expression $expr);
}

@bschussek, well seems legit.

I completely agree with @l3pp4rd as far as ensuring your backend queries are highly optimized, but this feature would be amazing to have in a lot of cases:

For example, in @l3pp4rd's example, you would have the query:

SELECT p, c, l FROM Entity\Post p
LEFT JOIN p.comments c
LEFT JOIN c.likes l
WHERE c.createdAt > :timestamp 

and in your controller, do:

<?php

    // ...

    $topComments = $this->comments->select(
        $expr->gt('c.likes', 1)
    );

    $comments = $this->comments->select(
        $expr->isNull('c.likes')
    );

Of course, for speed, it makes sense to do the sorting in one go-around... I'm not sure how hard it would be to make it so that it just iterates through the collection only one time and does all the expression matches, but that would be pretty rad if it works this way too.. for example:

<?php

    // ...

    // iterate through all the comments in one go-around
    list($topComments, $comments) = $this->comments->select(array(
        $expr->gte('c.likes', 1),  // get comments with likes
        $expr->isNull('c.likes')  // get comments with no likes
    );

Also, having limits and orderBys, limits, notIns would be even more powerful then having the ability to do::

<?php

    // ...

    // get the top 5 comments
    $topFiveExpr = $expr->gte('c.likes', 1)->limit(5)->orderBy('c.likes ASC');  // get the top 5 comments

    // get the rest of the comments excluding the top 5
    $commentsExpr = $expr->notIn($topFiveExpr);

    list($topFiveComments, $comments) = $this->comments->select(array(
        $topFiveExpr,
        $commentsExpr
    );

I see a pretty awesome twig extension coming out of this too ;P