RapNet diamond searching Magento Extension
RapNet is the Rapport Diamond Trading Network connecting thousands of the best diamond suppliers and buyers around the world. RapNet is an exclusive professional trading community where members deal directly with each other without any commission or transaction fees.
RapNet has its presence in 89 countries and there are more than 1,586,116 Diamonds trading on the site. RapNet is the world’s largest online marketplace for certified diamonds and reputable diamond professionals.
The challenge was to develop a Magento site that fetches the huge data and displays products information available on RapNet in more presentable and comprehensive manner
Challenge
- Managing such a huge product data on normal hosting environment.
- Need of Buy parcel features for bulk buying.
- Effective search for Color diamond.
- Data update to get real time pricing.
- UV Glow diamond selection.
Magewix Technologies has specialization in integrating Rapnet API diamond feed with Magento. Below is a white paper on how we do it
A Quick technical walkthrough on how we integrated Rapnet Diamond feeds with Magento.
Magento is the largest e-commerce installations world-wide. Magewix Technologies being leader in jewelry diamond portals; routinely receive requests to integrate Rapnet or IDEX diamond data feeds with existing Magento based jewelry websites.
However unlike Opencart or Prestashop, integrating Rapnet feeds (often going into 50,000+ diamond records) is complex programming. It took us considerable time and development effects to perfect it.
How we do it
1) Initial Analysis
a) Type of Rapnet Subscription
On receiving project proposal to integrate Rapnet data feed with Magento the initial check is on client’s Rapnet subscription to determine if they have Rapnet DLS services subscribed or not. For pulling data in CSV format from Rapnet and re inserting in Magento (or in any shopping cart database) Rapnet Diamond Listing Services ( DLS) is required. DLS is not switched on by default with subscription. If not available a request has to be put with Rapnet. Rapport/Rapnet also provides inventory link services to pull data directly from its servers . However to have full data control and customized mark-ups DLS is an absolute must.
b) Server configuration
Second aspect is, what configuration server the client has for its Magento hosting. Magento and Rapnet combination requires powerful server config. A Dedicated server or cloud hosting on Amazon or Rackspace is recommended.
Once both the above issues are discussed integration work proceeds.
2) Data pulling from Rapnet servers in CSV format
Client needs to provide us details on what type of diamond they are interested in. For example some may be interested in diamonds from a particular country only. Or some may want to pull stones which are of particular carat weight e.g. 0.3 to 2 carat.
Based of client’s requirements Rapnet DLS services are logged into and appropriate URL is generated which is supposed to download the diamond data in csv.
3) Filtering data
Diamond data pulled has lot of impurity as it’s uploaded directly by diamond wholesalers from across the globe. Magewix Technologies has perfected filters which weed out lot of inaccurate data. These may be wrong GIA cert numbers or records without weight , shape or empty records etc. After data scrubbing it is ready for reinserting in clients Magento database.
4) Deleting unwanted data
Daily Rapnet data is updated by its members. Appropriately data on Magento needs to be refreshed by removing unavailable or sold out stones. It is a complicated process but perfected by Magewix Technologies through its extensively tested scripts.
5) Updating / inserting data
The process of inserting data consist of parsing the csv files and inserting the data under appropriate categories and attributes.
6) First time Preparation of database Creating category and subcategories
Initially” Diamond” as main category is created manually.
Inside diamond category sub-categories of shape is created manually for example Round, Pear, Emerald, Heart, Cushion, Asscher, Princess, Marquise, Oval, Radiant
At the time of data insertion we fetch above category ids by passing shape.
Attribute insertion through script:
If the requirement is of large number of attributes and creating them manually is cumbersome we use, Magento function to bulk create attributes: $installer->addAttribute(‘catalog_product’, ‘manufacturer’, $attribute);
7) Price markup
This is an important functionality which we inbuilt with data insertion. Client provides us his markup percentage (could be flat say 20% on all diamonds or may be staggering as per carat weight).
8) Cron job setting
To eliminate any manual intervention cron job is created to repeat the process of data pulling and inserting at required frequency (daily/ twice in a week / once in a week). A log file is created to maintain the log on cron job executed.
9) On front Creating Ajax based search interface
Magewix Technologies has also specialized in AJAX based search interface for diamonds. The search interface filters through thousands of diamond records at blazing fast speed to pull out the filter records. Search interface is extremely user friendly, multi browser compatible and searches Diamonds on all core parameters including carat weight, price, clarity, florescence and shape.
Commonly Occurring issue while integrating rapnet data with magentoMagento Index management.
Issue : Magento indexes most of its data in order to access it faster. When you make changes to your Store such as updating some of your products, you need to re-index the data so that the changes can show up on frontend. Larger database requires more time to re-index the data. So re-indexing data by Magento admin panel could time out with thousands of Diamond records.
Solution : There is another way to reindex your magento data and it is to do it by cron jobs. You can set cron job by Cpanel or SSH.Maximum execution time of 30 seconds exceeded.
Issue : Magento’s default max_execution_time is of 30 seconds but to insert large data like Rapnet CSV it will often time out.
Solution : There is a way to increase max execution time. Either by in php .ini or in htaccess files. Change max_execution_time=18000.
CSV files read or write permission.
To read or write from CSV file you have to mention the read/write permission to CSV file.
10) Here is a live demo of Rapnet IDEX diamond Integrated on Magento
Commonly Occurring issue while integrating rapnet data with magentoMagento Index management.
Issue : Magento indexes most of its data in order to access it faster. When you make changes to your Store such as updating some of your products, you need to re-index the data so that the changes can show up on frontend. Larger database requires more time to re-index the data. So re-indexing data by Magento admin panel could time out with thousands of Diamond records
Solution : There is another way to reindex your magento data and it is to do it by cron jobs. You can set cron job by Cpanel or SSH.
Maximum execution time of 30 seconds exceeded.
Issue : Magento’s default max_execution_time is of 30 seconds but to insert large data like Rapnet CSV it will often time out.
Solution : There is a way to increase max execution time. Either by in php .ini or in htaccess files. Change max_execution_time=18000.
CSV files read or write permission.
To read or write from CSV file you have to mention the read/write permission to CSV file.
