Magento: Working with large collection

This piece of code will loop through each customers in the database base, but it will run out of memory with if you have a large number of customers in your database. It will throw PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 513088680 bytes)

$customers = Mage::getModel('customer/customer')->getCollection()
			->addAttributeToSelect(array('email','firstname','lastname'));
foreach($customers as $c)
{
	echo $c->getEmail();
}

Pagination technique allows you to divide the collection in pages(chunks), each page contains the number of objects you specified. This way it only loads the number of objects specified in each page iteration.

$customerCollections = Mage::getModel('customer/customer')->getCollection()
->addAttributeToSelect(array('email','firstname','lastname'))
->addAttributeToFilter('store_id', "1")
->setPageSize(1000);
$pages = $customerCollections->getLastPageNumber();
echo "$pages number of customers:".$customerCollections->getSize();
 
 
for($i=1; $i<=$pages; $i++)
{
	$customerCollections->setCurPage($i);
	$customerCollections->load();
	foreach($customerCollections as $cust)
	{
		echo $cust->getEmail()."\n";
	}
	$customerCollections->clear();
}

Resource iterator technique allows you to get each item one by one through the function callback using the walk function in core/resource_iterator

public function resourceIteratorMethod()
{
	$customerCollections = Mage::getModel('customer/customer')->getCollection()
	->addAttributeToSelect("entity_id","email")
	->addAttributeToFilter('store_id', "1");
	
	Mage::getSingleton('core/resource_iterator')->walk(
	$customerCollections->getSelect(),
	array(array($this,'customerCallback'))
	);
}
public function customerCallback($args)
{
	$cust = Mage::getModel('customer/customer');
	$cust->setData($args['row']);
	echo $cust->getEmail()."\n";
}

Search within Codexpedia

Custom Search

Search the entire web

Custom Search