1. System support

Before you install python 2.7 in centos 5.*, first check https/zlib/bzip2 support

• Compression requires the (missing) zlib module
  yum -y install zlib zlib-devel
• unknown url type: https — Some packages may not be found! yum -y install openssl openssl-devel
• python -c “import bz2; print bz2.doc” ==> CompressionError: bz2 module is not available yum -y install bzip2 bzip2-devel

2. Install Python 2.7.5

When you are with CentOS 5.*, you will see that Python is at version 2.4.3, it is reserved for system’s use. Next we are going to install Python 2.7.5 without breaking the system’s default. It is rather important to not to get involved with that as critical system tools such as YUM depend on it.

2.1 download & install

wget http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tgz -P /opt
cd /opt
tar xvf Python-2.7.5.tgz
cd Python-2.7.5
./configure --prefix=/usr/local
# make altinstall is used to prevent replacing the default python binary file /usr/bin/python
make && make altinstall

2.2 Adding New Python Installation Location to $PATH

export PATH="/usr/local/bin:$PATH"
or you can(recommended): echo "export PATH=\"/usr/local/bin:\$PATH\"" >> /etc/profile && source /etc/profile

2.3 check python command

python2.7 -c "import sys; print sys.version"

2.4 install setuptools

wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz -P /opt
tar xvf setuptools-1.4.2.tar.gz
cd setuptools-1.4.2
# Be careful, python2.7 here
python2.7 setup.py install

2.5 install pip

curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python2.7 -

3. Scrapy install

3.1 Pre-requisites

• Python 2.7
• lxml
• pip or easy_install

3.2 lxml

@see http://lxml.de/installation.html

yum -y install libxml2 libxml2-devel libxslt libxslt-devel

3.3 scrapy

@see http://stackoverflow.com/questions/7340784/easy-install-pyopenssl-error pip install pyOpenSSL==0.12

pip install Scrapy==0.16.4

4. Optional: Redis/Hiredis/MySQL-Python/SQLAlchemy/PIL/DBUtils

pip install redis hiredis

yum install mysql-devel
pip install MySQL-python

pip install SQLAlchemy

yum install libjpeg libjpeg-devel freetype freetype-devel libpng libpng-devel
pip install http://effbot.org/media/downloads/Imaging-1.1.7.tar.gz

pip install DBUtils