Tuesday, July 28, 2015

Openstack data_processing deployed in virtual environment for sahara kilo release

prerequisites
  • Cinder  storage_availability_zone = nova
  • Heat 


1:Install  sahara into a virtual environment
    $sudo apt-get install python-setuptools python-virtualenv python-dev
   $ virtualenv sahara-venv
   $ sahara-venv/bin/pip install 'http://tarballs.openstack.org/sahara/sahara-stable-kilo.tar.gz'
   $ mkdir sahara-venv/etc
   $ cp sahara-venv/share/sahara/sahara.conf.sample-basic sahara-venv/etc/sahara.conf

2:install local mysql for sahara

3: Sahara Configuration sahara.conf
    [DEFAULT]
    use_neutron=true
    use_namespaces=True
    [database] 
    connection=mysql://username:password@host:port/database
    [keystone_authtoken]
   auth_uri=http://127.0.0.1:5000/v2.0/

   identity_uri=http://127.0.0.1:35357/

4: Policy configuration
    cat sahara-venv/etc/policy.json
   {
    "default": ""
   }
   By default sahara will search for a policy.json file in the    same directory as the configuration file.

5: Create the database schema
     $ sahara-venv/bin/sahara-db-manage --config-file sahara-venv/etc/sahara.conf upgrade head
6: start sahara
   $ sahara-venv/bin/sahara-all --config-file sahara-venv/etc/sahara.conf

7:  register sahara in the Identity service catalog    
$openstack service create --name sahara --description "Sahara Data Processing" data-processing
$openstack endpoint create --region RegionOne \
--publicurl "http://16.158.50.211:8386/v1.1/%(tenant_id)s" \
--adminurl "http://16.158.50.211:8386/v1.1/%(tenant_id)s" \
--internalurl "http://16.158.50.211:8386/v1.1/%(tenant_id)s"\

8:Building Images for sahara Plugin
As of now the sahara plugin works with images with pre-installed versions of Apache Hadoop. To simplify the task of building such images we use Disk Image Builder




  • Clone repository “https://github.com/openstack/sahara-image-elements” locally( sudo bash diskimage-create.sh -h )
  • tox -e venv -- sahara-image-create -p [vanilla|spark|hdp|cloudera|storm|mapr]
    tox -e venv -- sahara-image-create -i [ubuntu|fedora|centos]

  • glance image-create --name=ubuntu_sahara_vanilla_hadoop_2_6 \--disk-format=qcow2 --container-format=bare \--file ./ubuntu_sahara_vanilla_hadoop_2_6_latest.qcow2 --progress
  •   
  •   
  • $sahara image-register --id $IMAGE_ID --username ubuntu
  • No comments:

    Post a Comment