+++
title = 'Apply the prediction pipeline in Google Cloud'
date = 2024-07-30T22:06:27Z
draft = true
weight = 17
+++
Apart from applying the prediction models to analyze images, the script that runs [the prediction pipeline](./learn-about-pipeline) on Google Cloud ingests data into [BigQuery](https://cloud.google.com/bigquery) to store all the image analysis details.

After [setting up a server](../deploy-cn/start-server) in a Google Cloud account, you can start recording videos of objects passing on your conveyor belt to gather data for analysis. The next step is transferring those video or image files to a [Cloud Storage](https://cloud.google.com/storage) bucket, where the prediction pipeline processes the images.

The results of each video or image prediction are stored and appended to two BigQuery tables, ensuring efficient data management. After systematically processing each file in the Cloud Storage bucket, the pipeline exports and stores the results in another bucket for further use and analysis.

This page explains how to create and manage Cloud Storage buckets on Google Cloud for the videos you record, run the prediction pipeline to apply the models, and process images for further analysis. You can then connect a visualization dashboard to the BigQuery tables to [display results as charts and reports](../view-data/).<br/><br/>

{{< table_of_contents >}}

---

## Store videos in Cloud Storage buckets

To effectively manage and process the videos or images recorded by the [machine vision camera](../system-req/choose-camera/) capturing objects on the conveyor belt, you need the following storage devices:

-  **Local disk storage**: Temporarily cache or store files locally from the camera.
-  **Cloud Storage input bucket**: Store your recorded videos or images. You can automate uploads using a storage transfer agent.
-  **Cloud Storage output bucket**: Store the prediction results.

By using these storage options, you ensure an organized, automated, and efficient workflow for managing and processing your recorded videos or images and prediction results.

### Create the Cloud Storage input and output buckets

[Create a Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) in your Google Cloud account that serves as the _input_ repository for the recorded videos. This bucket stores all the videos or images captured by the machine vision camera and centralizes the storage of your recorded files, making them easily accessible for further processing.

Before uploading to the input bucket, store the files temporarily on your local disk in a predetermined location. This temporary storage ensures you have a local backup and can manage uploads efficiently.

To streamline the upload process to the input bucket, you can schedule a storage job to run at your expected frequency. This automated job takes care of transferring videos or images from the local disk to the Cloud Storage input bucket, relieving you of the manual task. Additionally, implementing a storage transfer agent further facilitates the transfer process, ensuring reliable and efficient uploads. For more information, see [Manage transfer agents](https://cloud.google.com/storage-transfer/docs/managing-on-prem-agents). Otherwise, manually upload your files stored or cached in your local storage to the input bucket before [running the prediction pipeline](#run-the-prediction-pipeline). For more information about manual uploads, see [Upload objects from a file system](https://cloud.google.com/storage/docs/uploading-objects).

In addition to the input bucket, create another Cloud Storage _output_ bucket to store the results generated by your prediction pipeline. This output bucket serves as a well-organized repository for the processed data, such as predictions and analysis results, ensuring that the output from your models is easily accessible and you can manage your pipeline's output.

## Grant required permissions

The [service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account) from your NVIDIA T4 GPU virtual machine (VM) instance must have access to Cloud Storage buckets and BigQuery tables to let the VM instance run the prediction pipeline and upload the results to BigQuery.

The VM you created when [deploying CircularNet](../deploy-cn/) has a service account that must act as a _principal_ with the following two roles:  

-  Grant the Storage Admin role to access your Cloud Storage input and output buckets. [Add the VM service account as a principal to a bucket-level policy](https://cloud.google.com/storage/docs/access-control/using-iam-permissions#bucket-add) for both buckets.
-  Grant the BigQuery Admin role to access your BigQuery tables. [Manage roles of the VM service account using the Google Cloud console](https://cloud.google.com/iam/docs/manage-access-service-accounts).

## Run the prediction pipeline

Follow these steps to run the prediction pipeline and process your input files on Google Cloud:

1. [Open the Google Cloud console](https://cloud.google.com/cloud-console).
1. [Create the Cloud Storage input and output buckets](#create-the-cloud-storage-input-and-output-buckets).
1. Upload the images or videos your machine vision camera captured to the Cloud Storage input bucket.
1. [Grant  the required permissions to the VM service account](#grant-required-permissions).
1. From the **Navigation menu** on the Google Cloud console, select **Compute Engine** > **VM instances**.
1. On the **VM instances** page, find the VM instance you created with the NVIDIA T4 GPU when [deploying CircularNet](../deploy-cn/).
1. If you stopped your VM instance, restart it by clicking **More actions** > **Start / Resume** in the row of the instance that you want to restart.  
  
    **Note:** Stop the instance after you finish using it to avoid unnecessary expenses.  

1. Click **SSH** in the row of the instance that you want to connect to. The **SSH-in-Browser** tool opens. For more information, see [Connect to VMs](https://cloud.google.com/compute/docs/connect/standard-ssh#connect_to_vms).
1. On the **SSH-in-browser** window, [start the server](../deploy-cn/start-server).
1. Display the names of the models you loaded to the Triton inference server:

    ```
    cat triton_server.sh
    ```

	The first lines of the output show the names of the loaded models in square brackets. You can call any of these models when running the prediction pipeline.  
	  
	**Important:** Run the previous command in the `server` folder, which contains the `triton_server.sh` script.  

1. Exit the `server` folder and open the client folder in the `prediction_pipeline` directory:

    ```
    cd ..
    cd client/
    ```

	This folder contains the `pipeline_images.py` and `pipeline_videos.py` Python files that store the complete prediction pipelines for input images or videos, respectively. The `run_gcp_images.sh` and `run_gcp_videos.sh` scripts run these Python files automatically.  

1. If you have to modify the scripts to provide your specific paths and values for the prediction pipeline, edit the corresponding parameter values on the script. The following example modifies the video pipeline script:

    ```
    vim run_gcp_videos.sh
    ```

	The Vim editor displays the following parameters:

    ```
    --input_directoy=<path-to-input-bucket>
    --output_directory=<path-to-output-bucket>
    --fps=<frames-per-second>
    --height=<height>
    --width=<width>
    --material_model=<material-model>
    --material_form_model=<material-form-model>
    --score=<score>
    --search_range=<search-range>
    --memory=<memory>
    --project_id=<project-id>
    --dataset_id=<dataset-id>
    --table_id=<table-id>
    ```

    Replace the following:

    -  `<path-to-input-bucket>`: The path to [the Cloud Storage input bucket you created](#create-the-cloud-storage-input-and-output-buckets), for example `gs://my-input-bucket/`. 
    -  `<path-to-output-bucket>`: The path to [the Cloud Storage output bucket you created](#create-the-cloud-storage-input-and-output-buckets), for example `gs://my-output-bucket/`. 
    -  `<frames-per-second>`: The rate at which you want to capture images from videos to split videos into frames, for example, 15.
    -  `<height>`: The height in pixels of the image or video frames that the model expects for prediction, for example, 512.
    -  `<width>`: The width in pixels of the image or video frames that the model expects for prediction, for example, 1024.
    -  `<material-model>`: The name of the material model in the Triton inference server that you want to call, for example, `material_resnet_v2_512_1024`.
    -  `<material-form-model>`: The name of the material form model in the Triton inference server that you want to call, for example, `material_form_resnet_v2_512_1024`.
    -  `<score>`: The threshold for model prediction, for example, 0.40.
    -  `<search-range>`: The pixels up to which you want to track an object for object tracking in consecutive frames, for example, 100.
    -  `<memory>`: The frames up to which you want to track an object, for example, 20.
    -  `<project-id>`: The ID of your Google Cloud project, for example, `my-project`.
    -  `<dataset-id>`: The ID that you want to assign to a BigQuery dataset to store prediction results, for example, `circularnet_dataset`.
    -  `<table-id>`: The ID that you want to assign to a BigQuery table to store prediction results, for example, `circularnet_table`. If the table already exists in your Google Cloud project, the pipeline appends results to that table.

    **Note:** If your input files are not videos but images, replace `run_gcp_videos.sh` on the command with `run_gcp_images.sh` and remove the `--fps=<frames-per-second>` parameter.

    Save changes and exit the Vim editor. To do this, press the **Esc** key, type `:wq`, and then press **Enter**.  

1. Run the prediction pipeline:

    ```
    bash run_gcp_videos.sh
    ```

    **Note:** If your input files are not videos but images, replace `run_gcp_videos.sh` on the command with `run_gcp_images.sh`.

The script also creates a `logs` folder inside the `client` folder that saves the logs with the troubleshooting results and records from the models.

You have finished running the prediction pipeline and applying the prediction models to your files for further analysis. You can find the image results with the applied masks in your output bucket. You can also open the generated [BigQuery](https://cloud.google.com/bigquery) table to see the model analytics results and [preview table data](https://cloud.google.com/bigquery/docs/quickstarts/load-data-console#preview_table_data). To create new results, repeat the steps in this section every time you modify the files in your input bucket.

**Important:** If you rerun the prediction pipeline on the same video or image file, you must delete the results created the first time you ran the script from the output bucket to avoid conflicting issues. [Manage the lifecycle of objects in your Cloud Storage buckets](https://cloud.google.com/storage/docs/lifecycle) to help manage costs.