Adding/Modifying/Deleting a bucket to InfinStor

Concepts

InfinStor maintains a table of buckets. A bucket known to InfinStor may be used for InfinSnap/InfinSlice or for MLflow artifact storage. This table of buckets has access information for each bucket, i.e. access key id and secret access key or role ARN and role external ID.

REST API

InfinStor provides an API for managing entries in this buckets table

add-mod-del-bucket

URL: https://your_service_server/add-mod-del-bucket

Method: POST

Data:

• op: required. must be add, modify or delete
• bucketname: required for the add operation.
• cloud: required for add operation. Must be aws, azure, minio or other
• endpoint: optional
• sharedSecret: optional. used by minio storage for sending infinsnap events from minio to InfinStor object_store_event lambda
• accessKeyId: optional
• secretAccessKey: optional
• accessRole: optional
• accessRoleExt: optional

Notes:

Calls to the API must be authenticated. Only the administrator for the service has permissions to make this call.

Example 1:

In this example, a new buckets table entry is created for the bucket named testb3. This bucket uses the default endpoint and has access role called aaaa with external id bbbb. The following is the data sent in the POST call

op=add&bucketname=testb3&endpoint=&accessKeyId=&secretAccessKey=&accessRole=aaaa&accessRoleExt=bbbb

Here are some hints if the bucket testb3 belongs to another AWS account.

The bucket must have CORS enabled

Go to the bucket, click on permissions, scroll down to the CORS section and add a policy that might look like:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3600
    }
]

Role Details

The role described as aaa in the example above is a role ARN. The role must have a permissions policy that enables access to the bucket testb3. For example, the following policy would work

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::testb3"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::testb3/*"
            ]
        }
    ]
}

Example 2:

In this example, an existing buckets table entry is modified for the bucket named testb3. A non default endpoint https://minio.isstage1.bb.com/ is specified in this call. Note that the endpoint URL is urlencoded

op=modify&bucketname=testb3&endpoint=https%3A%2F%2Fminio.isstage1.bb.com%2F

Example 3:

In this example, an existing buckets table entry for the bucket named testb3 is deleted

op=delete&bucketname=testb3

Example 4:

In this example, an existing buckets table entry for the bucket named testb3 is converted from role based access to secret access key based access

op=add&bucketname=testb3&endpoint=&accessKeyId=iiii&secretAccessKey=ssss&accessRole=&accessRoleExt=