README.md

Issue

Projectors fail with contraint errors when rebuilding read models.

Setup description

Implementations of Prooph\EventStore\Projection\AbstractReadModel are used to build read-models. We are using Doctrine Entities for these models. ReadModelProjector::OPTION_PERSIST_BLOCK_SIZE is set to 1000 The persist method of Prooph\EventStore\Projection\AbstractReadModel has been adapted to wrap the stack of changes inside a transaction. (see ReadModelTrait.php)

The Problem

In regular production this works fine, as the stack of changes is usually small since the projector is up-to-date. However when rebuilding the read models I reguarly and relyable get constraint exceptions.

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '24a4e8ed-de31-5574-bed9-2da25ff29070' for key 'PRIMARY'

These errors seem to come from cascading one-2-many relationships.

Analises

As said, these exceptions seem to occure for cascading one-2-many relationships. In our case a single Order may have many Properties. But only one of the same type (or name).

For the appropiate events I determain if I should create, update or remove a property based on its name and existence on the order.

The code for this typically looks like;

            case $event instanceof Event\ProductGroupWasUpdated:
                /** @var Order $order */
                if ($order = $this->entityManager->find(Order::class, $event->aggregateId())) {
                    $order->setProductGroupId((string) $event->productGroup());
                    $order->setProductionMethod((string) $event->productionMethod());

                    foreach ($event->initialOrderProperties()->toArray() as $name => $value) {
                        if ($order->getOrderProperties()->containsKey($name)) { // existing property, we'll update it
                            /** @var OrderProperty $property */
                            $property = $order->getOrderProperties()->get($name);

                            if (null !== $value) {
                                $property->setValue($value);

                                $this->entityManager->persist($property);
                            } else {
                                $order->removeOrderProperty($property);
                                $this->entityManager->remove($property);
                            }
                        } else {
                            OrderProperty::createAndAddToOrder($order, $name, $value);
                        }
                    }

                    $this->entityManager->persist($order);
                }

                break;
            case $event instanceof Event\OrderProductGroupOptionsWhereUpdated:
                /** @var Order $order */
                if ($order = $this->entityManager->find(Order::class, $event->orderId())) {
                    foreach ($event->properties()->toArray() as $name => $value) {
                        if ($order->getOrderProperties()->containsKey($name)) { // existing property, we'll update it
                            /** @var OrderProperty $property */
                            $property = $order->getOrderProperties()->get($name);

                            if (null !== $value) {
                                $property->setValue($value);

                                $this->entityManager->persist($property);
                            } else {
                                $order->removeOrderProperty($property);
                                $this->entityManager->remove($property);
                            }
                        } else {
                            $property = OrderProperty::createAndAddToOrder($order, $name, $value);

                            $order->addOrderProperty($property);
                        }
                    }

                    $this->entityManager->persist($order);
                }

                break;

To be complete I use this to create a new property;

    public static function createAndAddToOrder(Order $order, string $name, $value)
    {
        // cause of polymorhic properties
        switch ($name) {
            case OrderPropertiesPath::SANDALS_CUSTOM_MADE_MODEL_COMPOSITION:
                $property = new SandalsCustomMadeModelComposition();
                break;
            default:
                $property = new OrderProperty();
        }

        $property
            ->setPropertyId((string) Uuid::uuid5($order->getOrderId(), $name))
            ->setName($name)
            ->setValue($value);

        $order->addOrderProperty($property);

        return $property;
    }

Somehow $order->getOrderProperties() seems to not return the correct list of properties. Perhaps from before the transaction started.

Inspecting the property table I find that an order property exists before the transaction is started.

SELECT x.* FROM plhw_application_api_development.read_dossier_order_property x
WHERE property_id='24a4e8ed-de31-5574-bed9-2da25ff29070'

property_id                         |property      |order_id                            |name          |value|
------------------------------------|--------------|------------------------------------|--------------|-----|
24a4e8ed-de31-5574-bed9-2da25ff29070|order.property|b999faa0-2eac-4355-ad6a-4b1b2ab1345c|options.remark|""   |

Performance difference

OPTION_PERSIST_BLOCK_SIZE = 1     => 100%
OPTION_PERSIST_BLOCK_SIZE = 1000  => 16%

So would like to keep using 1000...

ReadModelTrait.php

<?php

public function persist(): void
    {
        if ($this->entityManager) {
            $this->entityManager->beginTransaction();

            try {
                parent::persist();
                
                // parent::persist() calls the following
                //
                // foreach ($this->stack as list($operation, $args)) {
                //    $this->{$operation}(...$args);
                // }
                //
                // $this->stack = [];

                $this->entityManager->flush();
                $this->entityManager->commit();
            } catch (\OutOfBoundsException $e) {
                $this->entityManager->rollback();

                //throw $e;
            } catch (\Exception $e) {
                $this->entityManager->rollback();

                throw $e;
            }

            $this->entityManager->clear();
        } else {
            parent::persist();
        }
    }