GPU CI Tests Using Kaggle. How to make sure your code will work in… | by yousefnami | May, 2022

How to verify your code will work in a GPU atmosphere

Picture due to Roman Synkevych taken from Unsplash.

TLDR: you need to use action-kaggle-gpu-test in your workflows. That is an Early Entry Launch model. To comply with modifications, please watch the repo on GitHub.

In case your solely entry to GPUs is thru Kaggle and Colab then it’s probably that you just by no means write unit exams that use a GPU atmosphere. It is because a) it’s tough emigrate a complete codebase to Kaggle/Colab and b) it’s even tougher to automate this.

On this article I current a easy resolution for automating GPU integration exams utilizing GitHub actions and Kaggle. The article begins with a quick description of the Drawback Assertion, adopted by the Proposed Answer and eventually ending with some Concluding Remarks. Data of unit testing and GitHub Actions will likely be assumed all through.

With out a native GPU system, I sometimes use Colab/Kaggle solely on the final step when coaching. Which means that whereas creating, I can’t confirm that the coaching loop will work on a GPU runtime. Even when I’m going by way of the effort of debugging by way of Colab/Kaggle, any additional modifications that I make to the codebase offline will have to be examined once more. Working in a group multiplies all of those issues. At this stage, automated exams on a GPU runtime turn out to be actually useful.

Ideally, there could be a CI/CD service that permits you to connect with a GPU runtime for free. Nevertheless, on the time of writing there aren’t any such companies. The closest one is Gradient which allegedly permits you to connect with a free M4000 community GPU, nevertheless the documentation was not clear on whether or not the free GPU can also be accessible on their Workflow service. Once I tried this, it required a bank card so I made a decision to go for different strategies.

This left me with an answer that interfaces with Colab/Kaggle immediately. There have been distinctive challenges related to each:

  • Colab: you possibly can ssh into Colab and use the GPU accessible, nevertheless this can’t be automated since you possibly can’t launch a Colab runtime programmatically. You need to do that manually by way of your browser.
  • Kaggle: you too can ssh right into a Kaggle VM, however you gained’t be capable to see the GPU. It is because the run button on Kaggle initialises sure instructions earlier than you need to use the GPU. I used to be not in a position to determine how to do that programmatically.

The ultimate resolution I settled on was to launch a GPU-enabled Kaggle script to run integration exams.

The answer is to do the next inside a GitHub Actions runtime:

  • Authenticate Kaggle CLI: use a Kaggle API token to setup the Kaggle CLI. This may let you immediately interface with Kaggle for working scripts/notebooks utilizing a GPU runtime
  • Create and Ship Exams: create a Python script that may run on a Kaggle GPU runtime. This Python script will clone the repository you want to check, setup the digital atmosphere and run pytest. The Kaggle CLI is then used to ship a script with the title and title of the repository you’re testing. That is carried out utilizing kaggle kernels push $REPO_NAME
  • Examine Kaggle Standing: use kaggle kernels standing $REPO_NAME to test the standing of the check. If the standing has an error, because of this the check failed and the logs are returned (utilizing kaggle kernels output $REPO_NAME). If the check has handed, then a hit message is returned. In any other case the standing will likely be checked periodically.

The diagram under exhibits how the answer works end-to-end:

All of that is written as a separate motion.yml that exists as a standalone merchandise. I’ve referred to as it action-kaggle-gpu-test. You’ll be able to then use this in any repository GitHub Workflow information to allow testing utilizing the GPU runtime. That is carried out as follows:

The motion file itself is proven under:

  • GPU testing is critical to guarantee that your fashions will utilise assets accurately.
  • It’s exhausting to automate this free of charge, given the shortage of CI/CD GPU clusters
  • It’s doable to attain GPU testing utilizing the Kaggle API to programmatically create notebooks/script utilizing a GPU runtime
  • This technique isn’t significantly environment friendly or elegant, nevertheless it achieves its objective.

The Motion could be discovered within the GitHub market:

To see a venture that’s utilizing this motion for GPU exams, see this repository:

In case you’re excited about updates to this motion, watch the repository:

All photographs by writer except specified in any other case.

More Posts