Sites such as eBay, Etsy and AliBaba have enormous reach and support with 100,000’s of sellers on just one site.

However, multi-vendor sites are also really difficult to build and maintain. E-commerce itself is difficult enough, but when you multiply the number of transactions and vendors, you multiply all of the potential issues.

Still, we often get asked how to set up a multi-vendor site. For our members and our readers, here are 9 options for building a multi-vendor site with open source software.

Drupal with Drupal Commerce

• Price: Free.
• Description: Drupal Commerce is a powerful shopping cart system for Drupal. Drupal 7 and Drupal Commerce are free to companies that want to deploy it themselves.

Commerce Kickstart

Commerce Kickstart is the quickest way to get up and running with Drupal Commerce. It includes the latest versions of Drupal core, Drupal Commerce, and the other modules it depends on.

Drupal Commerce Multiseller

What is eCommerce Multiseller?

One Website, multiple sellers a.k.a. stores

Think Etsy, Amazon Marketplace, Ebay

Unified Customer Experience

Can buy at multiple stores at once

Stores have their own dashboard for managing products, orders, promotions, etc.

Third-party Modules:

1. Commerce Store
2. Commerce Marketplace

PHP Environment

see here: http://www.drupalcommerce.org/questions/8356/ajax-http-error-occurred-http-result-code-500

vim /etc/php5/apache2/php.ini 

max_execution_time = -1 (default is 60 seconds, -1 is unlimited)
max_input_time = -1
max_input_nesting_level = 300
memory_limit = 256M

Download & Extract

You can download the lastest Commerce Kickstart here: https://drupal.org/project/commerce_kickstart

cd /var/www
wget http://ftp.drupal.org/files/projects/commerce_kickstart-7.x-2.12-core.tar.gz
tar xzf commerce_kickstart-7.x-2.12-core.tar.gz
mv commerce_kickstart-7.x-2.12/* .
rm -rf commerce_kickstart-7.x-2.12 commerce_kickstart-7.x-2.12-core.tar.gz

Requirements

cd /var/www
mkdir -p sites/default/files
cp sites/default/default.settings.php sites/default/settings.php
chown -R www-data:www-data sites

Install guide

Go to http://your-server-ip/ , you will see the WELCOME SCREEN.

If you see the welcome message, Congrats! Follow the guide.

You can check your current timezone by just running

$ date

Or checking the timezone file at

$ more /etc/timezone
America/New_York

So to change it just run

$ sudo dpkg-reconfigure tzdata

And follow on screen instructions. Easy.

Install Apache

sudo apt-get update

sudo apt-get install apache2

vim /etc/apache2/httpd.conf Add this line: ServerName localhost

To check if Apache is installed, direct your browser to your server’s IP address (eg. http://12.34.56.789). The page should display the words “It works!”

Install MySQL

sudo apt-get install mysql-server libapache2-mod-auth-mysql php5-mysql

During the installation, MySQL will ask you to set a root password.

Once you have installed MySQL, we should activate it with this command: sudo mysql_install_db

Finish up by running the MySQL set up script:

sudo /usr/bin/mysql_secure_installation

It’s easiest just to say Yes to all the options. At the end, MySQL will reload and implement the new changes.

Create a new database for your site. mysqladmin -u root -p create commerce

Install PHP

sudo apt-get install php5 libapache2-mod-php5 php5-mcrypt

sudo apt-get install php5-gd

sudo apt-get install php5-curl

查看系统内安装的locale

locale -a

查看当前系统语言环境

locale

安装语言包: zh_CN.utf8

cd /usr/share/locales

./install-language-pack zh_CN

设为默认

sudo vim /etc/default/locale

修改为:

LANG="zh_CN.UTF-8"

LANGUAGE="zh_CN:zh"

重启shell，OK

Background

Hbase Version: 0.90.6

RegionSplitter

Usage

The org.apache.hadoop.hbase.util.RegionSplitter class provides several utilities to create a table with a specified number of pre-split regions.

create a table named ‘myTable’ with 60 pre-split regions containing 2 column families ‘test’ & ‘rs’

bin/hbase org.apache.hadoop.hbase.util.RegionSplitter -c 60 -f test:rs myTable

• -c 60, specifies the requested number of regions as 60
• -f specifies the column families you want in the table, separated by “:”

SplitAlgorithm

MD5StringSplit is the default RegionSplitter.SplitAlgorithm for creating pre-split tables. The format of MD5StringSplit is the ASCII representation of an MD5 checksum. Row are long values in the range “00000000” => “7FFFFFFF” and are left-padded with zeros to keep the same order lexographically as if they were binary.

public static class MD5StringSplit implements SplitAlgorithm {
    final static String MAXMD5 = "7FFFFFFF";
    final static BigInteger MAXMD5_INT = new BigInteger(MAXMD5, 16);

    public byte[][] split(int n) {
        BigInteger[] splits = new BigInteger[n - 1];
        BigInteger sizeOfEachSplit = MAXMD5_INT.divide(BigInteger.valueOf(n));
        for (int i = 1; i < n; i++) {
            // NOTE: this means the last region gets all the slop.
            // This is not a big deal if we're assuming n << MAXMD5
            splits[i - 1] = sizeOfEachSplit.multiply(BigInteger.valueOf(i));
        }
        return convertToBytes(splits);
    }

    ...

}

Rowkey Design

    String id = "your_id";

    String md5 = org.apache.commons.codec.digest.DigestUtils.md5Hex(id);
    BigInteger x1 = new BigInteger(md5, 16);
    BigInteger x2 = new BigInteger("7FFFFFFF", 16);
    BigInteger x3 = x1.and(x2);
                      
    String rowkey_hash = x3.toString(16);
    while(rowkey_hash.length() < 8){
        rowkey_hash = "0" + rowkey_hash;
    }

    String rowkey = rowkey_hash + "_" + id;
    System.out.println(rowkey);

Scenario

• Data Size: 4TB
• HBase Cluster: 10 nodes
• hbase.hregion.max.filesize: 2GB
• Cluster Region Count: 4TB/2GB = 2048
• Node Region Count: 2048/10 ~= 200

command

• To compile Sqoop, run ant package. There will be a fully self-hosted build provided in the build/sqoop-(version)/ directory.
• You can build just the jar by running ant jar.
• You can specify the hadoopversion, ant -Dhadoopversion=20 package

hadoopversion

hadoopversion. Can only be 20, 23, 100, 200 or 210

20

arg1="${hadoopversion}" arg2="20"
name="hadoop.version" value="0.20.2-cdh3u1"
name="hbase94.version" value="0.90.3-cdh3u1"
name="zookeeper.version" value="3.3.3-cdh3u1"

23

arg1="${hadoopversion}" arg2="23"
name="hadoop.version" value="0.23.1"
name="hbase94.version" value="0.92.0"
name="zookeeper.version" value="3.4.2"

100

arg1="${hadoopversion}" arg2="100"
name="hadoop.version" value="1.0.0"
name="hbase94.version" value="0.92.0"
name="zookeeper.version" value="3.4.2"

200

name="hadoop.version" value="2.0.4-alpha"
name="hbase94.version" value="0.94.2"
name="zookeeper.version" value="3.4.2"

210

name="hadoop.version" value="2.1.0-beta"
name="hbase94.version" value="0.94.2"
name="zookeeper.version" value="3.4.2"

MySQL->HBase:HBaseImportJob

MySQL->HBase:HBaseBulkImportJob

MySQL->HDFS(Avro files)–>HBase

Sqoop from MySQL into Avro files in HDFS, this will result in a bunch of avro files in your target directory. You can check that they’re there with hadoop fs -ls $TARGET_DIR.

sqoop import --connect jdbc:mysql://$MYSQL_SERVER/$DATABASE --driver com.mysql.jdbc.Driver --username $USER --password $PASSWORD --table $TABLE --target-dir $TARGET_DIR --as-avrodatafile

Error in hbase regionserver log

2014-02-13 11:38:13,028 INFO org.apache.hadoop.hdfs.DFSClient: Failed to connect to /192.168.101.21:50010, add to deadNodes and continue
java.net.SocketException: Too many open files

About ulimit

http://hbase.apache.org/book.html#ulimit

To be clear, upping the file descriptors and nproc for the user who is running the HBase process is an operating system configuration, not an HBase configuration.

Raising ulimit on centos-server

fs.file-max

1. vim /etc/sysctl.conf
2. Add this line fs.file-max = 122880 into /etc/sysctl.conf
3. You can execute the command sysctl -p or reboot to permanently add the kernel parameter above

   ulimit

4. vim /etc/security/limits.conf
5. Add the lines below:
   • soft nofile 122880
   • hard nofile 122880
6. logout and login, execute the command ulimit -n inside your terminal. If you had done everything correctly, you should now see a value returns as 122880.

CM users

For CM users, since there’s a supervisor in effect that monitors the launched daemons, the ulimits are inherited from it and are hence pre-set in /usr/sbin/cmf-agent (look for lines that start with “ulimit”.

1. Edit /usr/sbin/cmf-agent and change the ulimit -n setting.
2. shutdown hdfs/mapreduce/hbase services on them.
3. On these stopped nodes run: service cloudera-scm-agent hard_restart
4. Restart the services.

How to confirm

To confirm, you can go to any CM-managed node, ps -ef | grep hdfs, grab the pid of a hdfs process. cat /proc/<pid>/fd to make sure it’s sticking.

Limit                     Soft Limit           Hard Limit           Units     
Max processes             65536                65536                processes 
Max open files            122880               122880               files 

1. System support

Before you install python 2.7 in centos 5.*, first check https/zlib/bzip2 support

• Compression requires the (missing) zlib module
  yum -y install zlib zlib-devel
• unknown url type: https — Some packages may not be found! yum -y install openssl openssl-devel
• python -c “import bz2; print bz2.doc” ==> CompressionError: bz2 module is not available yum -y install bzip2 bzip2-devel

2. Install Python 2.7.5

When you are with CentOS 5.*, you will see that Python is at version 2.4.3, it is reserved for system’s use. Next we are going to install Python 2.7.5 without breaking the system’s default. It is rather important to not to get involved with that as critical system tools such as YUM depend on it.

2.1 download & install

wget http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tgz -P /opt
cd /opt
tar xvf Python-2.7.5.tgz
cd Python-2.7.5
./configure --prefix=/usr/local
# make altinstall is used to prevent replacing the default python binary file /usr/bin/python
make && make altinstall

2.2 Adding New Python Installation Location to $PATH

export PATH="/usr/local/bin:$PATH"
or you can(recommended): echo "export PATH=\"/usr/local/bin:\$PATH\"" >> /etc/profile && source /etc/profile

2.3 check python command

python2.7 -c "import sys; print sys.version"

2.4 install setuptools

wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz -P /opt
tar xvf setuptools-1.4.2.tar.gz
cd setuptools-1.4.2
# Be careful, python2.7 here
python2.7 setup.py install

2.5 install pip

curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python2.7 -

3. Scrapy install

3.1 Pre-requisites

• Python 2.7
• lxml
• pip or easy_install

3.2 lxml

@see http://lxml.de/installation.html

yum -y install libxml2 libxml2-devel libxslt libxslt-devel

3.3 scrapy

@see http://stackoverflow.com/questions/7340784/easy-install-pyopenssl-error pip install pyOpenSSL==0.12

pip install Scrapy==0.16.4

4. Optional: Redis/Hiredis/MySQL-Python/SQLAlchemy/PIL/DBUtils

pip install redis hiredis

yum install mysql-devel
pip install MySQL-python

pip install SQLAlchemy

yum install libjpeg libjpeg-devel freetype freetype-devel libpng libpng-devel
pip install http://effbot.org/media/downloads/Imaging-1.1.7.tar.gz

pip install DBUtils