ZF2 How to add fieldset to form and remove element from it

CreateContactForm.php

<?php
namespace Application\Form;

use Zend\Form\Form;
use Zend\ServiceManager\ServiceLocatorAwareInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

class CreateContractForm extends Form
    implements ServiceLocatorAwareInterface
{
    protected $serviceLocator;

    public function init()
    {
    	$sm = $this->getServiceLocator();
    	$contractFieldset = $sm->get('ContractFieldset');
    	$contractFieldset->setUseAsBaseFieldset(true);
    	$this->add($contractFieldset);

        // Remove `comment` element from `ContractFieldset`
    	$contractFieldset->remove('branch');
    }

    public function getServiceLocator ()
    {
        return $this->serviceLocator;
    }

    public function setServiceLocator (ServiceLocatorInterface $serviceLocator)
    {
        $this->serviceLocator = $serviceLocator;
    }
}

EditContractForm.php

<?php
namespace Application\Form;

use Zend\Form\Form;

class EditContractForm extends Form
{
    public function init()
    {
	$this->add(array(
       	    'type' => 'ContractFieldset',
       	    'name' => 'contract',
	    'options' => array(
		'use_as_base_fieldset' => true,
    	    ),
	));

        // Remove `comment` element from `ContractFieldset`
    	$this->get('contract')->remove('branch');
    }
}

In my ‪#‎Form‬ I have ‪#‎Fieldset‬ named 'contract'. I would like to remove an element 'branch' that is added inside this Fieldset.

CreateContractForm.php shows the easy way when adding Fieldset via:

$contractFieldset = new ContractFieldset();

But how to remove the element when the Fieldset was added the way in EditContractForm.php?