Tutorial 11: Deploy your optimization model with elytica and Streamlit

Introduction and setup

This tutorial focuses on the integration of elytica and Streamlit for solving optimization problems. Specifically, we will show you how to create a Streamlit application that interacts with elytica to solve the classic diet problem.

While some familiarity with linear programming is assumed, the focus of the tutorial is on using Streamlit to create a user-friendly interface that allows users to input their dietary requirements and obtain an optimal food list generated by elytica’s optimization capabilities. We will walk you through the process of building the application step by step, including how to connect elytica and Streamlit, create an input table, and display the results.

By the end of this tutorial, you will have a practical understanding of how to use elytica and Streamlit to create data-driven applications that solve optimization problems. So let’s dive in and get started!

To get started with the integration of elytica and Streamlit on Windows, you can download the latest version of Python from the official Python website at https://www.python.org/downloads/. Once downloaded, run the installation file and follow the instructions to install Python on your computer. Make sure you add python to the environment variables.

To install Streamlit, open a command prompt by typing “cmd” into the search bar in the Windows start menu and hitting enter. In the command prompt, type the following command:

pip install streamlit

After downloading Python, you can install Streamlit on your Windows computer by opening a command prompt and running the command pip install streamlit. This will install Streamlit and its dependencies on your machine.

Next, create a folder where you will store your project files, and within that folder create a new file called app.py. Once you have done this, navigate to your project directory in the command prompt and run the following command to launch your Streamlit application:

streamlit run app.py

This will launch your Streamlit application on a local server, which you can access in your web browser at http://localhost:8501. With Python and Streamlit installed, you are now ready to integrate elytica and Streamlit to solve optimization problems.

In addition to Streamlit, there are a few other packages we need for this project. You can install them by running the following command in your command prompt:

pip install elytica-dss pandas

This will install the required packages: elytica-dss package for elytica integration and pandas. Once you have installed these dependencies, you will be ready to use them in your Streamlit app for the diet problem using elytica.

Step 1: Import required libraries and load elytica credentials

import streamlit as st
import elytica_dss as edss
import time
import json
import pandas as pd

# Load elytica credentials
elytica_token = st.secrets["elytica"]["token"]

# Authenticate the user and access the DSS (Data Science Studio) instance
dss = edss.Service(elytica_token)
dss.login()

In this code snippet, we import the required libraries, which include streamlit, elytica_dss, time, json, and pandas. The code shown in the snippets should be placed in app.py.

Next, we load the elytica credentials from the secrets.toml file (create a folder called .streamlit and place it there) using st.secrets["elytica"]["token"]. The secrets.toml file is a file that contains sensitive information such as API keys, passwords, and other secrets. It is used to securely store sensitive information and is not included in the code repository.

In this example, the secrets.toml file contains the following:

[elytica]
token = "YOUR_ELYTICA_TOKEN"

You should replace YOUR_ELYTICA_TOKEN with your actual elytica token. Once the token is loaded, we authenticate the user and access the elytica service by creating a new instance of the edss.Service class and passing the elytica_token as an argument. We then call the login() method on the instance to authenticate the user.

Step 2: Setup the elytica project for diet optimization

Now that we have loaded the elytica credentials, we can proceed to set up the project for optimizing our diet. The first step is to check if there is an existing project with the name “Diet Problem.” We can do this using the getProjects() function of the dss object that we defined earlier.

st.write("# Optimize your diet")
projects = dss.getProjects()
name = "Diet Problem"
description = "The diet problem"
project = list(filter(lambda p: p.name == name, projects))
job = None
job_completed = False
output = ""
input_filename = "diet_problem.json"
latex_out = ""
results = ""

In the above code snippet, we first use the st.write() function to display a title for our project. Then we use dss.getProjects() to retrieve a list of all existing projects. We specify the name of our project as “Diet Problem” and set a default description. We initialize some variables that we will use later on.

Next, we define a helper function selectJob() which takes a project as input and sets the selected project and job.

def selectJob(project):
  dss.selectProjectById(project.id)
  jobs = dss.getJobs()
  job = jobs[0]
  dss.selectJobById(job.id)
  return job

This function will be used to select the project and job whenever we need to work on them.

Now we need to check if the project exists. If not, we will create a new project. We can do this using the following code:

if not project:
  st.write("No project named Diet. Please proceed to create a project.")
  applications = dss.getApplications()
  option = st.selectbox(
      'What type of application should the diet problem be?',
      tuple([a.display_name for a in applications]))
  if st.button('Create Project'):
    app = [a for a in applications if a.display_name==option][0]
    projects = dss.createProject(name, description, app)
    project = list(filter(lambda p: p.name == name, projects))[0] 
    job = selectJob(project)
else:
  project = project[0]
  job = selectJob(project)

In the above code, we check if the project variable is empty. If it is, we display a message asking the user to create a new project. We use the getApplications() function to retrieve a list of all available applications and display them in a dropdown using the st.selectbox() function. Once the user selects an application, they can click the “Create Project” button to create the project using the createProject() function. We then retrieve the newly created project and set the job by calling the selectJob() function.

If the project already exists, we simply retrieve it using project[0] and set the job using selectJob(project).

That’s it for setting up the elytica project for our diet optimization task! In the next step, we will define the optimization problem and set up the input data.

Step 3: loading and editing the diet problem data

With elytica, we need to specify the input files required for the optimization problem. This includes the linear programming model, which can be found in the model.hlpl file. You can either copy the model.hlpl file to your Streamlit project folder or paste the contents of the file in the newly created project on compute.elytica.com. You can create a project beforehand named “Diet Problem” on compute.elytica.com or reuse the project created in Tutorial 10. Similarly, you can reuse diet_problem.json from Tutorial 10 or download it from the provided link. We will be using a helper function assignAndUploadFile to upload the files to the elytica service. The function takes four arguments:

1. name: The name of the file.
2. contents: The contents of the file as a string.
3. arg: The argument to assign the file to.
4. replace (optional): A boolean value indicating whether to replace the file if it already exists. The default value is True.

The assignAndUploadFile function first checks if the file already exists in the input files using a lambda function and the filter method. If the file doesn’t exist or replace is True, the function uploads the file using the dss.uploadFileContents method and assigns it to the specified argument using dss.assignFile.

Here is the implementation of the assignAndUploadFile function:

input_files = dss.getInputFiles()

def assignAndUploadFile(name, contents, arg, replace=True):
    file = list(filter(lambda f: f.name == name, input_files))
    if not file or replace:
        input_file = dss.uploadFileContents(name, contents)
        dss.assignFile(input_file, arg)

To get started, we will load the data from the diet_problem.json file using the json.load() method. We will then use the pd.DataFrame() function from the pandas library to convert the "Food" and "Nutrients" keys into dataframes.

with open(input_filename, 'r') as f:
  diet_data = json.load(f)

food_edited_df = st.experimental_data_editor(pd.DataFrame(diet_data["Food"]), num_rows="dynamic")
nutrients_edited_df = st.experimental_data_editor(pd.DataFrame(diet_data["Nutrients"]), num_rows="dynamic")

Here, we use Streamlit’s experimental_data_editor component to allow the user to edit the dataframes. This component provides an interface for editing tabular data, allowing the user to add or remove rows and edit the existing values.

We also include a Save button that, when clicked, will save the edited data back to the diet_problem.json file. To achieve this, we update the "Food" and "Nutrients" keys in the diet_data dictionary with the edited data, and then write the updated dictionary back to the file.

if st.button('Save'):
  diet_data["Food"] = json.loads(food_edited_df.to_json(orient="records"))
  diet_data["Nutrients"] = json.loads(nutrients_edited_df.to_json(orient="records"))
  with open(input_filename, 'w') as f:
    json.dump(diet_data, indent=2, ensure_ascii=False, fp=f)

In this way, we can load and edit the data used for the diet problem, making it easier to experiment with different inputs and run the optimization problem with customized data.

The Streamlit page should display two tables: one for the food items and one for the nutrients. The tables are editable, and the user can add, remove, or modify entries as needed. Once the changes have been made, the user can click the “Save” button to update the data in the diet_problem.json file. The changes will then be reflected in the optimization results when the user runs the optimization later in the tutorial.

Step 4: Running the application and getting results

In this step, we will run the application and get the results. The application uses the elytica service to run the linear programming model and optimize the diet based on the user’s input data. The optimization process will be done asynchronously, meaning that we can continue interacting with the streamlit application while the optimization is in progress. Once the optimization is complete, we can view the results and see the optimal diet plan.

def getStdOut(data):
  global output
  output = output + json.loads(data)['stdout'].replace("\n", " \n\r ")

This is a helper function that is used to get the standard output from the job. The getStdOut function takes in data as an argument which is the response from the server, and updates the global variable output with the standard output of the job.

def finished(data):
  global job_completed, latex_out, results
  output_files = dss.getOutputFiles()
  for f in output_files:
    if f.name == "latex_out":
      latex_out = dss.downloadFile(f) 
    if f.name == "results":
      results = dss.downloadFile(f)
  job_completed = True

This is another helper function that is used to get the output files from the job. The finished function takes in data as an argument which is the response from the server, and updates the global variables job_completed, latex_out, and results. It uses the dss.getOutputFiles() function to get a list of output files and downloads the latex_out and results files if they exist.

if st.button('Run'):
  model = open('model.hlpl', 'r').read()
  assignAndUploadFile(f"{job.id}.hlpl", model, 1, replace=False)
  assignAndUploadFile("diet_problem.json", json.dumps(diet_data, indent=2, ensure_ascii=False), 2, replace=True)
  dss.queueJob(finished_callback=finished, stdout_callback=getStdOut) 

This is the main code that is executed when the “Run” button is pressed. It first reads the model.hlpl file and assigns it to the job using assignAndUploadFile. It then uploads the diet_problem.json file using assignAndUploadFile. Finally, it adds the job to the job queue using dss.queueJob. The finished_callback and stdout_callback parameters are set to finished and getStdOut respectively, which are called when the job is finished or when there is standard output available.

placeholder = st.empty()
while (True):
  if job_completed:
    time.sleep(0.2)
  with placeholder.container():
    st.code(output)
  if job_completed:
    break
  time.sleep(0.05)

r = results.decode() if type(results) == bytes else results
final = json.loads(str(r))
results_df = st.table(pd.DataFrame(final["Diet"]))

latex_out = latex_out.decode() if type(latex_out) == bytes else latex_out
st.latex(latex_out)

if st.button('Do another run'):
  st.experimental_rerun()

The final snippet in Step 4 is used to display the results of the optimization problem. First, a placeholder is created to display the output, and a loop continuously checks if the job has completed. If the job has completed, the output is displayed using the st.code() function. Once the job is complete, the results are extracted from the results variable and loaded into a Pandas DataFrame using the pd.DataFrame() function. The DataFrame is then displayed using the st.table() function, which allows viewing the results.

The latex_out variable contains the LaTeX code generated by the optimization problem, which is displayed using the st.latex() function. Finally, a button is provided to allow for running the optimization problem again, using the st.experimental_rerun() function.

If all went well, you should have something similar to:

The code can also be found on the GitHub repository.