PHP Group associative Array duplicates and make subarrays of different values

I have this type of table:

$this->data = (
  0 => (
    'reference' => 'ABC123',
    'title' => 'lorem',
    'parent' => 12,
  ),
  1 => (
    'reference' => 'ABC456',
    'title' => 'ipsum',
    'parent' => 42,
  ),
  2 => (
    'reference' => 'ABC789',
    'title' => 'dolor',
    'parent' => 36,
  ),
  3 => (
    'reference' => 'ABC123',
    'title' => 'lorem',
    'parent' => 18,
  ),
  4 => (
    'reference' => 'ABC789',
    'title' => 'dolor',
    'parent' => 26,
  )
);

And I made this script to remove duplicates with keeping different known keys as subarrays:

// Get number of duplicates for each 'reference' of $this->data
$occurences = array_count_values(array_column($this->data, 'reference'));

// Array to put index to remove of $this->data
$to_unset = ();
foreach($occurences as $reference => $count) {
  // If item unique, continue
  if($count == 1) continue;

  // Get all indexes of 'reference' inside $this->data
  $indexes = array_keys(array_column($this->data, 'reference'), $reference);

  // Keep first index of occurence
  $first_index = $indexes(0);
    // Remove it from indexes
    unset($indexes(0));

  // Keep the first occurence
  $model = $this->data($first_index);
    // Convert different known keys as array
    $model('parentId') = ($model('parentId'));
      
  foreach($indexes as $index){
    // Group 'parentId' in same array
    array_push($model('parentId'), $this->data($index)('parentId'));
      
    // Put index to remove then
    array_push($to_unset, $index);
  }

  // Replace the first item occurence by model
  $this->data($first_index) = $model;
}

// Remove all occurences
$this->data = array_diff_key($this->data, array_flip($to_unset));

// Reindex
$this->data = array_values($this->data);

To get this:

$array = (
  0 => (
    'reference' => 'ABC123',
    'title' => 'lorem',
    'parent' => (12, 18),
  ),
  1 => (
    'reference' => 'ABC456',
    'title' => 'ipsum',
    'parent' => 42,
  ),
  2 => (
    'reference' => 'ABC789',
    'title' => 'dolor',
    'parent' => (36, 26),
  )
);

But my script is very slow (20 seconds for +13k items), how can I improve it ?