1. Quick Start¶
The objective is to configure your client environment (your laptop, CLI shell, Notebook or other environments) to talk to Infinstor MLFlow server. Use the steps below:
- Infinstor MLflow server configuration:
- for the free MLflow service, use the following
- MLFlow UI: https://mlflowui.free.infinstor.com
- MLFLOW_TRACKING_URI environment variable:
infinstor://mlflow.free.infinstor.com
- for the SaaS version purchased from the AWS Marketplace, use the following
- Infinstor service Dashboard UI: https://service.infinstor.com
- MLFlow UI: https://mlflowui.infinstor.com
- MLFLOW_TRACKING_URI environment variable:
infinstor://mlflow.infinstor.com
- for the special SaaS version provided to certain users (we would have let you know this), use the following
- Infinstor service Dashboard UI: https://service.saas.infinstor.com
- MLFlow UI: https://mlflowui.saas.infinstor.com
- MLFLOW_TRACKING_URI environment variable:
infinstor://mlflow.saas.infinstor.com
- for Enterprise version (single tenant version), please contact your administrator for the above configuration details.
- for the free MLflow service, use the following
- Install infinstor-mlflow-plugin
- Configure environment variable MLFLOW_TRACKING_URI
- Obtain a token file for authentication
- Configure MLFLOW_EXPERIMENT_ID environment variable
- Use MLflow cli or run python code with MLflow api calls. An MLflow example with xgboost is shown here.
- Use MLflow UI to view the MLflow runs under the above MLflow experiment
1.1. Install infinstor-mlflow-plugin¶
pip install --upgrade infinstor-mlflow-plugin
- Installs infinstor-mlflow-plugin in your client environment
1.2. Configure MLFLOW_TRACKING_URI environment variable¶
InfinStor MLflow backend is activated by setting the environment variable MLFLOW_TRACKING_URI to the value provided by your administrator. See configuration details above for details. Example
> export MLFLOW_TRACKING_URI=<admin_provided_tracking_uri>
1.3. Obtain a Token file for authentication¶
Use either the UI or CLI (as described below) to generate an authentication token to talk to Infinstor MLflow service
1.3.1. Token file generation using a UI¶
Use your InfinStor service dashboard UI URL (see configuration details described above) to login. Use your provided credentials. Then click on Configuration -> Manage Token
in the sidebar
1.3.1.1. Create and Download Token File¶
Pressing the 'Create Token File' button causes a new tab to be opened, and the user will flow through the InfinStor main service' cognito authetication. In the case of Enterprise Licenses, users will have to complete the authentication system configured for that particular Enterprise.
1.3.1.2. Use Token File for authenticating CLI programs¶
Once authentication is complete, the browser will download a file named token. This token file must be placed in a sub-directory called .infinstor in the user's home directory.
> mkdir -p ~/.infinstor
> cp ~/Downloads/token ~/.infinstor
1.3.2. Token file generation using the CLI¶
- Install
infinstor-mlflow-plugin
as described here - Configure your MLFLOW_TRACKING_URI using the steps described below.
login_infinstor
- This will prompt for your credentials and will automatically create the tokenfile in
~/.infinstor/token
if authentiation is successful
- This will prompt for your credentials and will automatically create the tokenfile in
1.4. Configure MLFLOW_EXPERIMENT_ID environment variable¶
The environment variable MLFLOW_EXPERIMENT_ID must be set in order to record runs in the correct experiment ID. If this environment variable is not set, then runs will be recorded to the default experiment id 0. This is particularly important when Authorization is enabled, because the user may not have access to experiment id 0. The following example sets the experiment id to 7
> export MLFLOW_EXPERIMENT_ID=7
1.5. Xgboost Example¶
Note
To run the example below, you will need Infinstor MLflow server with compute extensions (and not just the Infinstor MLflow server).
- The infinstor backend (the -b
argument to mlflow run
, as shown below) will run the MLproject
in Infinstor's compute platform. This platform allows the MLproject
to be run in any supported public cloud.
- Without the infinstor backend, the MLproject
will run in your local laptop or VM (where the mlflow run
is issued) instead.
The example xgboost supplied with open source MLflow is a good quick example. It can be found here: https://github.com/mlflow/mlflow/tree/master/examples/xgboost/xgboost_sklearn
Download the files from this directory and edit the conda.yaml by adding infinstor_mlflow_plugin
and boto3
to the pip dependencies. The edited file is shown here:
name: xgboost-example
channels:
- defaults
- anaconda
- conda-forge
dependencies:
- python=3.6
- xgboost
- pip
- pip:
- mlflow>=1.6.0
- matplotlib
- infinstor_mlflow_plugin
- boto3
Now you can run the example as follows:
> export MLFLOW_TRACKING_URI=<admin_provided_tracking_uri>
> cd <directory with downloaded files>
> export MLFLOW_EXPERIMENT_ID=<exp_id>
> mlflow run -b infinstor-backend --backend-config '{"instance_type": "t3.large"}' .
1.6. MLflow UI¶
The MLflow UI is available at the URL provided by your administrator. See configuration details above for details. Use the MLflow UI to see the tracked MLflow experiments' and MLflow runs*.