mirror of https://github.com/Lightning-AI/pytorch-lightning.git synced 2026-04-02 02:07:46 +00:00

Files

Akihiro Nitta e47d5a2376 CI: Combine conda and full testing into a single workflow (#14387 )

* Remove conda job

* Remove conda job from readme

* Remove conda jobs from checkgroup

* Remove conda from docker builds

* Remove base-conda dockerfile

* Rewrite the strategy matrix while keeping equivalent

* Run the workflow on this branch

* Revert "Rewrite the strategy matrix while keeping equivalent"

This reverts commit e54298d60e57cffbf8107890987be3fe4a006c77.

* Add PyTorch versions

* Run on draft and disable unrelated costly CI

* Revert "Run the workflow on this branch"

This reverts commit 51ed8b905d8926b630dce4817124bd486135d3ec.

* tmp: Lightweight relevant CI

* Fix CI pathfilter

* Update matrix

* Drop skipping logic

* pip list

* reorder pip list

* tmp: lightweight ci

* Install specified pytorch

* Fix torch installation

* Uncomment steps

* Increase timeout

* bad merge

* Revert "Run on draft and disable unrelated costly CI"

This reverts commit eb5dc5e6bd.

* Update checkgroup

* Update docs and remove Python/PyTorch versions

* Remove pip-list

* Fail if wrong pytorch version installed

* Add Python 3.8, PyTorch 1.9 job

* tmp: remove azure jobs

* tmp: remove dockers

* tmp: remove others

* Run all combinations

* Include oldest

* Exclude no Python 3.10 distributions

* tmp: no concurrency

* tmp: double timeout

* Add pytest log reporter

* Add pytest-reportlog

* Fewer jobs

* Revert "tmp: no concurrency"

This reverts commit 4a7978dcb3.

* fix artifact name

* Revert test reports

* Revert unrelated changes

* Revert unrelated changes

* Add the combination of ex-conda jobs

* Update checkgroup

* revert timeout

* remove conda job

* revert docker build workflow file

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

2022-09-29 22:39:04 -04:00

7.6 KiB

Raw Blame History

Continuous Integration and Delivery

Unit and Integration Testing

workflow name	workflow file	action	accelerator*
Test PyTorch full	.github/workflows/ci-pytorch-test-full.yml	Run all tests except for accelerator-specific, standalone and slow tests.	CPU
Test PyTorch slow	.github/workflows/ci-pytorch-test-slow.yml	Run only slow tests. Slow tests usually need to spawn threads and cannot be speed up or simplified.	CPU
pytorch-lightning (IPUs)	.azure-pipelines/ipu-tests.yml	Run only IPU-specific tests.	IPU
pytorch-lightning (HPUs)	.azure-pipelines/hpu-tests.yml	Run only HPU-specific tests.	HPU
pytorch-lightning (GPUs)	.azure-pipelines/gpu-tests.yml	Run all CPU and GPU-specific tests, standalone, and examples. Each standalone test needs to be run in separate processes to avoid unwanted interactions between test cases.	GPU
PyTorchLightning.Benchmark	.azure-pipelines/gpu-benchmark.yml	Run speed/memory benchmarks for parity with pure PyTorch.	GPU
test-on-tpus	.circleci/config.yml	Run only TPU-specific tests.	TPU

*Accelerators used in CI
- GPU: 2 x NVIDIA Tesla V100
- TPU: Google GKE TPUv3
- IPU: Colossus MK1 IPU
- HPU: Intel Habana Gaudi SYS-420GH-TNGR which has 8 Gaudi accelerators
To check which versions of Python or PyTorch are used for testing in our CI, see the corresponding workflow files or checkgroup cofig file at .github/checkgroup.yml.

Documentation

workflow file	action
.github/workflows/docs-checks.yml	Run doctest.

Code Quality

workflow file	action
.codecov.yml	Measure test coverage with codecov.io
.github/workflows/code-checks.yml	Check Python typing with MyPy.
.github/workflows/ci-schema.yml	Validate the syntax of workflow files.

Others

workflow file	action
.github/workflows/cicd-pytorch-dockers.yml	Build docker images used for testing in CI. If run on nightly schedule, push to the Docker Hub.
.github/workflows/ci-pkg-install.yml	Test if pytorch-lightning is successfully installed using pip.
.github/workflows/events-recurrent.yml	Terminate TPU jobs that live more than one hour to avoid possible resource exhaustion due to hangs.

Deployment

workflow file	action
.github/workflows/release-pypi.yml	Publish a release to PyPI.
.github/workflows/release-docker.yml	Build Docker images from dockers/*/Dockerfile and publish them on hub.docker.com.
.github/workflows/legacy-checkpoints.yml	App on request generate legacy checkpoints and upload them to AWS S3.
.github/workflows/events-nightly.yml	Publish the package to TestPyPI. Publish Docker images on hub.docker.com.

Bots

workflow file	action
.github/mergify.yml	Label PRs as conflicts or ready, and request reviews if needed.
.github/stale.yml	Close inactive issues/PRs sometimes after adding the "won't fix" label to them.
.github/workflows/probot-auto-cc.yml, .github/lightning-probot.yml	Notify maintainers of interest depending on labels added to an issue We utilize lightning-probot forked from PyTorch’s probot.
.github/workflows/probot-check-group.yml, .github/checkgroup.yml	Checks whether the relevant jobs were successfully run based on the changed files in the PR
.pre-commit-config.yaml	pre-commit.ci runs a set of linters and formatters, such as black, flake8 and isort. When formatting is applied, the bot pushes a commit with its change. This configuration is also used for running pre-commit locally.

7.6 KiB Raw Blame History Unescape Escape

Continuous Integration and Delivery

Unit and Integration Testing

Documentation

Code Quality

Others

Deployment

Bots

7.6 KiB

Raw Blame History