Skip to main content

Home

Installing Unravel Server on an EC2 instance

This topic explains how to create a new EC2 instance, set up RDS (optionally), install and configure Unravel Server on the new EC2 instance, and connect it to the EMR cluster you want to monitor.

Important

If you have not already done so, confirm your cluster meets Unravel's hosting requirements.

1. Create an EC2 instance
2. Configure the EC2 instance

  1. Disable selinux.

    sudo setenforce Permissive

  2. Edit /etc/selinux/config to make sure the setting persists after reboot and make sure SELINUX=permissive.

    sudo vi /etc/selinux/config

  3. Install libaio.x86_64, lzop.x86_64, and ntp.x86_64.

    sudo yum install -y libaio.x86_64
sudo yum install -y lzop.x86_64
sudo yum install -y ntp.x86_64

  4. Start ntpd and check the system time.

    sudo service ntpd start
sudo ntpq -p

  5. Create a new user named hadoop.

    sudo useradd hadoop
4. Install the Unravel RPM on the EC2 instance

The installation creates the following items:

  • Virtualization type: HVM

  • User unravel (if it doesn't exist already).

  • /etc/init.d/unravel_* scripts for controlling services, and /etc/init.d/unravel_all.sh which you can use to manually stop, start, and get status of all daemons in proper order.

  1. Download the Unravel Server RPM.

  2. Install the Unravel Server RPM.

    The precise filename varies depending on the version, how it was fetched, or copied.

    sudo rpm -Uvh unravel-version.rpm

  3. Append the following line to /usr/local/unravel/etc/unravel.properties:

    com.unraveldata.onprem=false
com.unraveldata.cluster.type=EMR

  4. For monitoring EMR Spark service, add the following properties to unravel.properties:

    com.unraveldata.spark.live.pipeline.enabled=true
com.unraveldata.spark.hadoopFsMulti.useFilteredFiles=true
com.unraveldata.spark.events.enableCaching=true

  5. Run the switch_to_user.sh script..

    usr/local/unravel/install_bin/switch_to_user.sh hdfs hadoop
6. Log into Unravel UI

  1. Start Unravel daemons.

    sudo /etc/init.d/unravel_all.sh start

  2. Create an SSH tunnel from your workstation to the Unravel EC2 instance.

    ssh -i ssh_key.pem centos@unravel-host-ip -L 3000:127.0.0.1:3000

  3. Using a supported web browser (see Unravel's ​​Amazon EMR​ compatibility matrix), navigate to http://127.0.0.1:3000 and log in with username admin with password unraveldata.

    signin.png
In this section: