Skip to main content

Home

Setting up Amazon RDS (optional)

Unravel's default installation uses a bundled database for part of its storage. For better performance and ease of management, we recommend replacing the bundled database with an Amazon RDS instance.

1. Create an RDS security group, subnet group, and parameter group

  1. Create an RDS security group on the VPC of Unravel Server and allow inbound access from Unravel Server security group. For more information refer to Controlling access with security groups

  2. Create a new database subnet group named unravel.

    A new database subnet group is required for "multiple availability zone" (multi-AZ) deployment. The VPC should at least contains two subnets in at least two availability zones in a given region, in the same VPC. For more information, see AWS documentation.

    aws-emr-subnet.png

  3. Create a new database parameter group with custom settings. Choose a MySQL version compatible with Unravel.

    emr-rds-parameter.png
    Custom database parameters
     key_buffer_size = 268435456 
 max_allowed_packet = 33554432 
 table_open_cache = 256 
 read_buffer_size = 262144 
 read_rnd_buffer_size = 4194304 
 max_connect_errors=2000000000 
 net_read_timeout = 300 
 net_write_timeout = 600 
 innodb_open_files=9000 
 character_set_server=utf8 
 collation_server = utf8_unicode_ci 
 innodb_autoextend_increment=100 
 innodb_log_file_size = 134217728 
 innodb_log_buffer_size = 33554432 
 innodb_flush_log_at_trx_commit = 2 
 innodb_lock_wait_timeout = 50

    Caution

    It is not possible to modify a parameter group that is associated with a running RDS instance. To modify a parameter group that is in use, you can do either one of the following actions:

    • Create a copy of your desired parameter group, make the required modifications, and assign this new parameter group to your RDS instance.

    • Stop the RDS instance associated with your parameter group, make the change, and start the RDS instance again.

2. Create an Amazon RDS instance

  1. In the Amazon RDS portal, click Create database and enter the following details:

    • Choose a database creation method: Standard create

    • In Engine options, select the following options:

      • Engine options: MySQL

      • Edition: MySQL Community

      • Version: MySQL 5.7.33

    • In Templates, select Production.

    • In Settings, enter the following information:

      • DB instance identifier: unravel-mysql-prod

      • In Credential Settings, add the following information:

        • Master username: unravel

        • Master password: Change_Password

        • Auto generate a password: Select the checkbox to auto-generate a password.

    • In DB instance class, enter the following information:

      • DB instance class: Select Memory optimized classes (includes r and x classes) and then choose db.r5.large ( 2 vCPU, 16 GiB RAM)

    • In Storage, enter the following information:

      • Storage type: Provisioned IOPS (SSD)

      • Allocated Storage: 100

      • Provisioned IOPS:3000

      • Storage autoscaling: Enable storage autoscaling

        • Select Enable storage autoscaling.

        • Maximum storage threshold: 1000

    • In Additional configuration, enter the following information:

      • Initial database name: unravel-mysql-prod

      • DB parameter group: unravel-rds

      • Option group: default:mysql-5-7

  2. In the Advanced Settings page change the following settings.

    You can leave all other settings with default values or specify values suitable to your requirements.

    Network & Security Settings

    • Virtual Private Cloud: Select the VPC that contains minimally two subsets and on the same region that you plan to deploy Unravel and the EMR cluster.

    • Subnet group: Select the new database subnet group you created, named unravel.

    • Public accessibility: No

    • Availability zone : No Preference

    • VPC security group: Select the new VPC security group you created.

  3. Click Create database.

3. Connect Unravel to the RDS instance

By default, the security group created for the unravel RDS has no network access granted on port 3306 on the subnet connected. You must modify the security group applied on Unravel RDS.

  1. In the Amazon RDS dashboard, locate the MySQL database endpoint.

  2. Look for the security group used for the Unravel RDS instance.

  3. Modify the inbound rule of the security group by adding a new rule to allow connections from either:

    • The Unravel EC2 instance's security group.

    • The subnet IP block in which the Unravel EC2 instance located.

    Either the security group or IP block works, provided the RDS instance is located on the same region as the VPC.

  4. Install MySQL client on the Unravel server and then verify the MySQL connection.

    yum install mysql
mysql  -h <RDS-Hostname> -u unravel -p
mysql> show database

    For example,

    aws-emr-cli-mysql.jpg

  5. Verify that the database unravel_mysql_prod has been created; if not create it.

    CREATE DATABASE IF NOT EXISTS unravel_mysql_prod;
4. Create a schema for the RDS instance

  1. Stop Unravel

    <Unravel installation directory>/unravel/manager stop

  2. Set or add the following properties in the table as follows: 

    <Unravel installation directory>/unravel/manager config properties set property key
##For example: 
/opt/unravel/manager config properties set unravel.jdbc.username unravel 
/opt/unravel/manager config properties set unravel.jdbc.password P@sSworD
/opt/unravel/manager config properties set unravel.jdbc.url jdbc:mysql://unravelmysqlprod.csfw1hkmlpgh.us-east-1.rds.amazonaws.com:3306/unravel_mysql_prod

    You can use a hostname; but to avoid DNS lookups use an IP address.

    The database password can be encrypted.

  3. Refresh database.

    <Unravel installation directory>/unravel/manager refresh db

  4. Apply the changes

    <Unravel installation directory>/unravel/manager config apply

  5. Start Unravel

    <Unravel installation directory>/unravel/manager start
In this section: