Skip to content

Configure InfinStor Starter with Mlflow service

The InfinStor Starter with Mlflow service is configured by downloading and running the infinstor-config.sh. This script is included in the zip: Configuration Script

Summary

Configure your root AWS Account user's access key ID and secret access key as the default profile in ~/.aws/credentials, download and run the configure program infinstor-config.sh

Before you begin

Your AWS credentials must be correctly configured as the default. You cannot use a non default profile by setting the AWS_PROFILE environment variable. This means that the file ~/.aws/credentials must have a section similar to:

[default]
aws_access_key_id = AKIAAAAAAAAAAAAAAAAA
aws_secret_access_key = 3499GNNJGUH5Y85HNJGSRNSNHJ5TUI78

Note also that the credentials above must be for your AWS Account root user - IAM accounts will not work

AWS Resources required

The InfinStor Starter with Mlflow service uses two resources in your AWS Account. These resources are listed below:

  • A DynamoDB Table called infinstor-mlflow-server-[your-username]
  • An S3 bucket infinstor-mlflow-artifacts-[your-username]

IAM Roles and Permissions

In addition, the InfinStor Starter service requires some roles and permissions created. These are listed below:

  • An IAM Role called infinstor-mlflow-ec2-role-[your-username]. This role is for EC2 Instances started by InfinStor's 'Run Transform in a SingleVM' function. The following permissions are allowed to this role:
  • Allow access to SQS
  • Allow access to the S3 bucket infinstor-mlflow-artifacts-[your-username] for storing artifacts and logs
  • An instance profile that uses the above IAM role. This is for the EC2 instances started by InfinStor's 'Run Transform in a SingleVM' function
  • An IAM Role for the InfinStor service called infinstor-service-role-[your-username]. This role permits the InfinStor service to perform the following actions on your behalf in your AWS account:
  • Access EC2 for starting up VMs used for the 'Run Transform in a SingleVM' function
  • Allow access to the DynamoDB Table called infinstor-mlflow-server-[your-username]
  • Get and Pass the IAM Role called infinstor-mlflow-ec2-role-[your-username] to the EC2 instances started up by the 'Run Transform in a SingleVM' function
  • Allow access to SQS
  • Allow access to the S3 bucket infinstor-mlflow-artifacts-[your-username] for storing artifacts and logs

The infinstor-config.sh script configures the above resources in your AWS Account

Downloading

Example downloading and unzipping the configuration scripts

(base) jagane@jagane-i9:~/tmp$ unzip infinstor-config.zip 
Archive:  infinstor-config.zip
  inflating: infinstor-config.sh     
  inflating: upgrade-to-premium.sh   

Running

This is a screen capture of an example run, the example InfinStor service username is jagane-testcustomer: Example run of the infinstor-config.sh script. In this example, the InfinStor Service username is jaganesundar

(base) jagane@jagane-i9:~/tmp$ ./infinstor-config.sh 
Logging to /tmp/infinstor-config.sh.log
InfinStor service username: jaganesundar
InfinStor service password: 
Creating dynamodb table infinstor-mlflow-server-jaganesundar for InfinStor mlflow server
Creating bucket infinstor-mlflow-artifacts-jaganesundar for infinstor mlflow artifacts
Creating IAM role for EC2 Instances
Creating Instance Profile for EC2 Instances
Creating IAM Role for the InfinStor service
Saving AWS Config information in the InfinStor service
successfully saved customer aws config

You can delete the IAM Roles and Policies created by these scripts by running:

$ ./infinstor-config.sh -undo